Tunnel structure deformation identification and risk grading method based on multi-modal large model

Abstract

The invention relates to the field of tunnel deformation monitoring, in particular to a tunnel structure deformation identification and risk grading method based on a multi-modal large model, and the method comprises the steps: collecting and preprocessing multi-modal data: processing the collected data; large model analysis and feature fusion: modeling a long-term dependency relationship and a change mode of sensor time sequence data by a multi-modal large model, extracting structural damage features of image data, and fusing modal representations to form comprehensive vector representation of a tunnel structure state; trend prediction: performing time sequence trend prediction on the key deformation parameters of the tunnel structure by using the large model to obtain a deformation development trend; and abnormal inflection point identification: identifying the abnormal inflection point in the deformation process based on the deviation condition of the actual monitoring data, and marking the corresponding time point as the abnormal inflection point. According to the method, multi-modal sensing information is fused, and trend prediction, abnormal inflection point intelligent identification and automatic risk grading early warning of tunnel structure deformation are realized through a deep learning model.

Description

Technical Field

[0001] This invention relates to the field of tunnel deformation monitoring, specifically to a method for identifying and classifying the deformation of tunnel structures based on a multimodal large model. Background Technology

[0002] The long-term stability of tunnel structures is crucial for the safety of transportation and underground engineering. During tunnel operation or construction, the surrounding rock and support structure may experience slow deformation or even sudden displacement. Without timely monitoring and early warning, this could lead to serious structural safety accidents. Therefore, monitoring and risk assessment of tunnel structural deformation has always been a key focus in the field of civil engineering. Traditional tunnel deformation monitoring typically relies on manual periodic measurements or simple sensor threshold alarms. For example, measuring points are manually set up on the tunnel wall and settlement and convergence data are obtained using total stations, force gauges, etc., or a single sensor is used to monitor a specific physical quantity (such as strain or displacement). An alarm is triggered when the monitored value exceeds a preset threshold.

[0003] However, these traditional methods have many shortcomings: First, manual monitoring consumes a lot of manpower and resources and cannot achieve continuous real-time monitoring; second, single sensor or threshold judgment methods are easily affected by environmental noise, which may lead to false alarms or missed alarms; third, various sensor data are often analyzed independently, lacking comprehensive utilization of multi-source information, and cannot fully reflect the true state of the tunnel structure. Especially in cases of complex geological conditions or frequent construction disturbances, traditional methods are unable to capture the trend and inflection point of abnormal deformation in a timely and accurate manner, resulting in risk warnings lagging behind the development of hazards.

[0004] With the development of sensing technology and data communication, tunnel structural health monitoring can acquire multimodal data sources, including strain gauge readings on the structural surface, convergence settlement monitored by displacement sensors, changes in ambient temperature, and tunnel structural images acquired by surveillance cameras. This lays the foundation for more comprehensive intelligent analysis. Meanwhile, the application of artificial intelligence and deep learning technologies in structural health monitoring is increasing. In particular, recurrent neural networks such as Long Short-Term Memory (LSTM) have outstanding advantages in time series prediction, while one-dimensional convolutional neural networks (1D-CNN) perform excellently in extracting local features and patterns from time series data. Combining these two technologies is expected to improve the prediction accuracy and anomaly detection capabilities for structural deformation data.

[0005] However, existing technologies still lack solutions for multi-source data fusion and intelligent risk assessment in tunnels. Some methods only predict based on single sensor data without utilizing other relevant information; others, while mentioning multi-source data, do not fully employ deep learning for end-to-end intelligent identification, resulting in risk level assessments still relying on human experience or simple rules.

[0006] Therefore, there is an urgent need for an intelligent early warning method that can integrate multimodal data and has the ability to predict trends and automatically identify abnormal inflection points, so as to improve the accuracy and timeliness of tunnel structure deformation monitoring and realize automatic graded early warning of tunnel structure safety risks. Summary of the Invention

[0007] Based on the above description, this invention provides a method for tunnel structure deformation identification and risk classification based on a multimodal large model. It addresses the shortcomings of existing tunnel structure deformation monitoring, such as the lack of fusion analysis of multi-source data, untimely anomaly identification, and reliance on manual risk level judgment. It provides a more intelligent technical solution that can integrate multimodal sensor information and achieve trend prediction of tunnel structure deformation, intelligent identification of anomaly inflection points, and automatic risk classification and early warning through a deep learning model.

[0008] On the one hand, the technical solution of the present invention to solve the above-mentioned technical problems is as follows: a method for tunnel structure deformation identification and risk classification based on a multimodal large model, comprising the following steps:

[0009] Multimodal data acquisition and preprocessing: Acquire multi-source monitoring data of the tunnel structure, including strain, displacement, temperature and image data. Perform noise filtering, outlier removal, missing data imputation, normalization and timestamp alignment on the data to form a unified input format.

[0010] Large-scale model analysis and feature fusion: The preprocessed multimodal data is input into a multimodal large-scale model. Based on pre-trained cross-modal semantic understanding capabilities, the large-scale model models the long-term dependencies and change patterns of sensor time-series data, extracts structural damage features from image data, and fuses the representations of each modality to form a comprehensive vector representation of the tunnel structure state.

[0011] Trend prediction: Using the large model, the key deformation parameters of the tunnel structure are predicted over time series to obtain the deformation development trend within a predetermined time range in the future;

[0012] Anomaly inflection point identification: Based on the deviation of actual monitoring data from model prediction results, anomaly inflection points in the deformation process are automatically identified. When the deviation exceeds the dynamic threshold or abrupt changes occur continuously, the corresponding time point is marked as an anomaly inflection point.

[0013] Risk level assessment: Combining abnormal inflection point characteristics, deformation trend prediction results and damage information from image recognition, the risk level of the tunnel structure is automatically assessed according to preset safety assessment standards, and an early warning is triggered when a high risk level is reached.

[0014] The following technical effects are achieved through a closed loop of "unified data → cross-modal fusion → trend prediction → inflection point identification → risk classification": 1) By eliminating noise filtering, anomaly removal, missing data filling, normalization, and time alignment, data drift caused by different sensor dimensions and sampling mismatch is eliminated, significantly improving input consistency and robustness, and reducing false alarms and missed alarms; 2) Multimodal large-scale modeling of time series and images is used to cross-validate visual features such as cracks / stripping / seepage with the slow-changing relationships of strain, displacement, and temperature, overcoming the vulnerability of single modes and improving the generalization ability to new work sites, complex environments, and extreme working conditions; 3) Based on long-term dependency learning, forward predictions and uncertainty intervals of key deformation parameters are given, distinguishing between reversible fluctuations and structural progress caused by seasons / temperature, providing quantifiable lead time for maintenance scheduling, traffic organization, and resource preparation; 4) Anomaly inflection points are automatically marked with dynamic thresholds of predicted residuals and abrupt change continuity criteria, and combined with multimodal evidence to locate the location and possible mechanism, shortening the discovery-disposal link; 5) By integrating trends, inflection points, and damage quantification characteristics into a unified state vector, the system automatically classifies data according to safety standards and triggers early warnings, forming a traceable and consistent assessment caliber. This reduces subjectivity and operational costs, and supports threshold adaptation and coordinated response strategy deployment.

[0015] Based on the above technical solution, the present invention can be further improved as follows:

[0016] Furthermore, the multimodal large model is a cross-modal pre-trained model based on an improved attention mechanism, and its feature interaction weights are defined by the following formula:

[0017] ;

[0018] in, For the first A query vector for modal features. For the first Key vectors of modal features, The dimension of the key vector. For nonlinear modulation coefficients, It is a dynamic equilibrium factor.

[0019] Furthermore, the multi-source monitoring data includes:

[0020] Strain data acquired by strain gauges;

[0021] Converging displacement or settlement data collected by displacement sensors;

[0022] Ambient temperature data collected by a temperature sensor;

[0023] Image data of tunnel lining and internal structure collected by camera equipment.

[0024] Furthermore, the risk level determination is based on a comprehensive risk index. The calculation formula is as follows:

[0025] ;

[0026] in, To monitor the amount of deformation, The deformation threshold, For the rate threshold, To calculate the window length, The number of abnormal inflection points. This represents the total number of time periods. To identify damage factors in images, For model parameters, This is a nonlinear function used to normalize the image damage results.

[0027] Furthermore, the identification of the abnormal inflection point is based on an improved residual change rate criterion, wherein the residual is defined as: ;

[0028] When satisfied When that time point is determined to be an abnormal inflection point, then... These are measured values. For predicted values, For time step, For historical residual weights, The rate of change threshold, This is the threshold value for the residual amplitude.

[0029] Furthermore, the method further includes an intelligent early warning and response generation step, specifically:

[0030] Trend prediction based on the output of the multimodal large model Compared with measured values Calculate the weighted historical residuals

[0031] ;

[0032] in Historical residual weights;

[0033] Calculate the comprehensive risk index based on the fused multimodal features

[0034] ;

[0035] in This represents the current deformation. The deformation threshold, For the rate threshold, For window length, The number of abnormal inflection points. This represents the total number of time periods. Image damage factor, For model parameters, It is a nonlinear normalization function;

[0036] Construct an adaptive joint discrimination threshold, where the magnitude threshold and the rate of change threshold are respectively...

[0037] ;

[0038] in The prediction uncertainty estimate given for the large model. For near absolute deviation of the midpoint, For threshold parameters;

[0039] When satisfied When ; it is automatically determined to be the first The system assesses risk level and generates a comprehensive early warning report, which includes at least the risk level, future forecast trend, and timeframe. The system collects abnormal inflection points and recommends appropriate measures, which are then pushed to the tunnel management terminal or remote monitoring platform via a human-machine interface or network communication module. This enables automatic graded early warning of tunnel structural safety risks based on multi-source data fusion, trend prediction, and automatic identification of abnormal inflection points.

[0040] Secondly, the technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0041] A tunnel structure deformation identification and risk classification system based on a multimodal large model includes:

[0042] The multimodal data acquisition module is used to acquire multi-source monitoring data such as strain, displacement, temperature and images of the tunnel structure, and to perform preprocessing and synchronization.

[0043] The large model analysis module has a built-in cross-modal large model, which is used to receive the multimodal data and perform feature modeling and fusion, and output deformation trend prediction results and abnormal inflection point identification results.

[0044] The risk assessment and early warning module is used to assess the tunnel risk level based on the analysis results of the large model and output alarm or visual early warning information.

[0045] This system offers the following technical advantages: By integrating multimodal data acquisition and preprocessing (filtering, anomaly removal, missing data filling, normalization, and clock synchronization) to form a unified input, it eliminates biases caused by dimensionality, sampling frequency, and clock drift, significantly reducing false alarms and missed alarms. The cross-modal large model integrates numerical time series data with image semantics end-to-end, overcoming long-term reliance while extracting damage features such as cracks, spalling, and water seepage, achieving "numerical-visual" mutual verification and maintaining robustness and transferability under various operating conditions, seasonal changes, and equipment variations. It outputs forward-looking trends and uncertainty ranges for key deformation parameters, distinguishing between temperature / seasonal reversible fluctuations and structural advancements. It provides quantifiable lead time for maintenance scheduling, material preparation, and traffic organization; it automatically marks abnormal inflection points with dynamic thresholds driven by predictive residuals and continuous mutation criteria, taking into account both sudden jumps and slow accelerations, and locating the location and time of occurrence, shortening the discovery-response link; it integrates trend, inflection point, and image quantification indicators, automatically classifies risks according to safety standards, and triggers multi-channel alarms and visualizations, forming a unified and traceable assessment caliber to support coordinated response and closed-loop management; its modular architecture and time alignment mechanism improve the system's scalability and fault tolerance, making it easy to connect new sensors and model versions, and maintain stable operation in complex field conditions.

[0046] Furthermore, the system can be deployed on edge computing devices at the tunnel site for real-time processing, or deployed on a cloud platform to centrally receive and analyze sensor data, thereby supporting flexible deployment in different application scenarios.

[0047] Furthermore, the large model analysis module further includes a multimodal feature fusion unit and a temporal prediction unit, wherein:

[0048] The multimodal feature fusion unit is used to extract and uniformly represent cross-modal features of strain, displacement, temperature and image data collected by different sensors, so as to avoid the problem of insufficient prediction accuracy of a single sensor.

[0049] The time-series prediction unit is used to automatically model the deformation development trend based on fusion features and dynamically identify abnormal inflection points in combination with actual monitoring values, thereby achieving automatic risk level classification without the need for manual experience rules.

[0050] Thirdly, a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for identifying and classifying tunnel structure deformation based on a multimodal large model as described above.

[0051] Compared with existing technologies, this invention fully integrates multimodal monitoring information of tunnel structures and utilizes deep learning models to achieve comprehensive analysis of deformation behavior. Compared with existing technologies, this invention has the following advantages:

[0052] 1. Improved data consistency and robustness: Through filtering, anomaly removal, missing data imputation, normalization and timestamp alignment, heterogeneous data such as strain, displacement, temperature and images are unified into a stable input distribution, eliminating the deviation caused by differences in dimensions and sampling and clock drift, significantly reducing false alarms / false negatives, ensuring the continuity of time sequence, and providing a reliable data foundation for end-to-end learning and online inference;

[0053] 2. Cross-modal attention fusion enhances generalization and interpretability: Improved attention weights By introducing nonlinear modulation and dynamic balancing, the model adaptively allocates "numerical-visual" contributions, which not only overcomes long-term sensor dependence but also suppresses visual noise and spurious correlations. It remains robust across scenes, seasons, and device changes, and the visualization of weights supports interpretable diagnosis of key modalities and channels.

[0054] 3. Multi-source sensing mutual verification and fault tolerance: Strain / displacement reflects internal response, temperature reveals environmental coupling, and images characterize cracks, spalling and seepage; multi-source joint modeling can distinguish between reversible fluctuations and structural progress caused by seasons / temperature, and mutual verification reduces the blind zone of single modes; when individual sensors fail or drift, other modes form compensation and verification, improving system integrity and continuous operation capability.

[0055] 4. Risk index R achieves objective and consistent classification judgment: Deformation amplitude, rate fluctuation, inflection point intensity and image damage factor are unified into a calibrable comprehensive index, supporting the same caliber assessment and traceability for different sections and working conditions; continuous scoring and discrete level mapping facilitate threshold adaptation and strategy optimization, reduce the subjectivity of human experience, and provide a quantitative basis for maintenance scheduling and resource allocation.

[0056] 5. Adaptive Inflection Point Identification and Executable Early Warning Closed Loop: Based on predicting residuals and their rate of change, it combines dynamic thresholds composed of uncertainty and robust statistics. Simultaneously, it captures sudden jumps and gradual accelerations; automatically generates risk levels and future ranges. The report includes anomaly inflection point sets and handling suggestions, supports edge / cloud linkage, shortens the discovery-handling latency, and improves operation and maintenance efficiency and security redundancy. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of the overall system architecture and data flow of Embodiment 1 of the present invention;

[0058] Figure 2 This is a flowchart of the preprocessing and unified time base in Embodiment 1 of the present invention;

[0059] Figure 3 This is a schematic diagram of cross-modal fusion and attention weights in Embodiment 1 of the present invention;

[0060] Figure 4 This is a flowchart of trend prediction and uncertainty output in Embodiment 1 of the present invention;

[0061] Figure 5 This is a flowchart of the dual-threshold logic for identifying abnormal inflection points in Embodiment 1 of the present invention;

[0062] Figure 6 This is a schematic diagram of the risk index R classification and treatment mapping in Embodiment 1 of the present invention;

[0063] Figure 7 This is a diagram illustrating the dual-track deployment and linkage sequence of Embodiment 1 of the present invention;

[0064] Figure 8 This is a diagram of the multimodal data acquisition module in Embodiment 2 of the present invention;

[0065] Figure 9 This is a flowchart of the data preprocessing and synchronization process in Embodiment 2 of the present invention;

[0066] Figure 10 This is a structural diagram of the large model analysis module in Embodiment 2 of the present invention;

[0067] Figure 11 This is a cross-modal feature fusion map of Embodiment 2 of the present invention;

[0068] Figure 12 This is a schematic diagram of time series modeling and prediction in Embodiment 2 of the present invention;

[0069] Figure 13 This is a risk heat map and alarm information output diagram of Embodiment 2 of the present invention. Detailed Implementation

[0070] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0072] Example 1:

[0073] refer to Figure 1 and Figure 4 A method for tunnel structure deformation identification and risk classification based on a multimodal large model includes the following steps:

[0074] Multimodal data acquisition and preprocessing: Acquire multi-source monitoring data of the tunnel structure, including strain, displacement, temperature and image data. Perform noise filtering, outlier removal, missing data imputation, normalization and timestamp alignment on the data to form a unified input format.

[0075] Large-scale model analysis and feature fusion: The preprocessed multimodal data is input into a multimodal large-scale model. Based on pre-trained cross-modal semantic understanding capabilities, the large-scale model models the long-term dependencies and change patterns of sensor time-series data, extracts structural damage features from image data, and fuses the representations of each modality to form a comprehensive vector representation of the tunnel structure state.

[0076] Trend prediction: Using the large model, the key deformation parameters of the tunnel structure are predicted over time series to obtain the deformation development trend within a predetermined time range in the future;

[0077] Anomaly inflection point identification: Based on the deviation of actual monitoring data from model prediction results, anomaly inflection points in the deformation process are automatically identified. When the deviation exceeds the dynamic threshold or abrupt changes occur continuously, the corresponding time point is marked as an anomaly inflection point.

[0078] Risk level assessment: Combining abnormal inflection point characteristics, deformation trend prediction results and damage information from image recognition, the risk level of the tunnel structure is automatically assessed according to preset safety assessment standards, and an early warning is triggered when a high risk level is reached.

[0079] Overall technical logic (closed loop of device → edge → cloud)

[0080] Data source layer (end): Strain, displacement (convergence / settlement), temperature and images are synchronized with a unified timestamp;

[0081] Edge layer: Completes preprocessing, feature extraction, first-level rapid discrimination, and local early warning;

[0082] Cloud layer: Performs deep fusion of cross-modal large models, trend prediction and uncertainty estimation, anomaly inflection point identification, and comprehensive risk index. Calculation and hierarchical processing;

[0083] Governance and Feedback: Parameter / model versions and data lineage are governed in a unified manner, and thresholds and strategies are distributed in reverse to form a closed loop of "monitoring-analysis-early warning-handling-verification".

[0084] Replaceable base models: Cloud-based default models are Qwen2.5-VL / InternVL2 (choose one, either can be replaced with LLaVA-NeXT / PaliGemma 2); Edge-based default models are MiniCPM-V. Changing the base will not affect the formulas and discrimination logic in this paper.

[0085] Terms and Symbols

[0086] : Observe / predict deformation; Current deformation amplitude;

[0087] Amplitude and rate thresholds; Statistical / robust / forecasting window;

[0088] : Number of abnormal inflection points / Total number of time periods; Image damage factor;

[0089] Risk index parameters;

[0090] Prediction uncertainty; Adaptive / Fixed threshold;

[0091] Cross-modal attention weight variables; : Grading threshold;

[0092] : Residual and threshold coefficient.

[0093] As a preferred option, the overall system flow and functional modules are as follows:

[0094] S101 Monitoring Deployment: Strain gauges, convergence meters / settlement meters, temperature points, and industrial cameras are deployed at typical cross-sections (arch / arch waist / sidewall / track bed); each sensor / camera is assigned a unique ID and time-stamped with a unified clock (such as PTP).

[0095] S102 Data Access and Caching: End-side caching for 1–5 minutes per ring, edge nodes transmit data in batches; automatic retransmission in case of disconnection and control of packet loss retransmission.

[0096] S103 Preprocessing: Performs denoising, outlier removal, missing data completion, normalization, and time alignment to produce uniform output. Temporal tensors and calibrated image batches (see data acquisition and preprocessing module).

[0097] S104 Large Model Fusion: Temporal features are encoded by TS-Encoder (Transformer); images are encoded into tokens via the visual tower of the pedestal model; these tokens then enter the cross-modal co-attention layer, using weights... Fusion (see Cross-modal Model Structure and Alignment Module).

[0098] S105 Trend Forecasting and Uncertainty: Predicting the Future Step and (See the Training and Fine-tuning Process module).

[0099] S106 Abnormal Inflection Point Identification: Press , , The double-threshold rule is used to label mutations (see the trend prediction and abnormal inflection point identification module).

[0100] S107 Comprehensive Risk Classification: Calculation and Comparison of output levels and trend bands Recommendations for handling the situation (see the Comprehensive Risk Index and Tiered Early Warning Module).

[0101] S108 Closed-loop linkage: Edge triggers local alarms and increases sampling frequency; cloud generates reports, archives and distributes new policies (see Engineering Deployment, Parameter Calibration and Operation and Maintenance Module).

[0102] Key effect: Without changing the discrimination formula, the visibility of "weak early signs" is enhanced by cross-modal attention, and with the uncertainty-driven adaptive threshold, the low false alarm and early detection are balanced.

[0103] As a preferred option, the data acquisition and preprocessing module:

[0104] S201 sensor sampling strategy: strain, displacement: 1–5 min / time; temperature: 5–10 min / time; image: 5–10 min / frame or event trigger (such as displacement rate exceeding limit).

[0105] Image calibration: internal / external parameter calibration, distortion correction, ROI anchoring of circumferential seams / joints / water seepage-prone areas.

[0106] S202 Denoising and Outlier Removal, Timing: Savitzky–Golay or wavelet denoising; Outliers are removed by a combination of quantile (p1 / p99) and MAD.

[0107] Image: Non-local means / bilateral filtering; strong reflections / shadows are corrected with brightness threshold + morphological adaptive correction.

[0108] S203 missing data completion and alignment, short missing data (≤3 consecutive points) spline / Kalman smoothing; long missing data is estimated using "multimodal regression" (using adjacent sections and temperature);

[0109] Alignment: Standardize to fixed Interpolation / convergence is used to maintain energy conservation (avoiding spurious fluctuations).

[0110] S204 Normalization and Scale Unification, Time Series: z-score or min–max;

[0111] Image: Proportional scaling and block-by-block reading are performed according to the multi-scale scheme supported by the base (ensuring that high-resolution details are not lost).

[0112] Output: Feature matrix, image tokens, required standardized image batches, complete timestamps, and data lineage records.

[0113] As a preferred option, the cross-modal model structure and alignment module are:

[0114] S301 Vision Encoder (Replaceable Base)

[0115] Cloud-based (choose one): Qwen2.5-VL / InternVL2 Visual Tower (ViT series) + Projector Head;

[0116] Edge: MiniCPM-V integrates a lightweight visual tower;

[0117] Output: Multi-scale image token sequence and location information.

[0118] S302 Timing Encoder (TS-Encoder)

[0119] A multi-head self-attention Transformer is adopted; considering the periodicity of "day-night temperature change-structural response", a learnable periodic temporal embedding is introduced;

[0120] Output: A sequence of time-series tokens that is aligned with the number of visual tokens.

[0121] S303 Cross-modal fusion and weighting

[0122] ;

[0123] (Nonlinear modulation) and The (balance factor) is calibrated on the validation set to give sufficient weight to weak anomalies (early minor cracks, small rate surges);

[0124] A gating unit is introduced to suppress the negative transfer of "modal missing / low quality" to the fusion results (the gating value comes from the data quality score).

[0125] S304 Task Head

[0126] Prediction Header: Output and ;

[0127] Risk head: calculation (See the Comprehensive Risk Index and Tiered Early Warning Module).

[0128] Visual impairment head: estimation (Sub-factors such as cracks / misalignment / detachment / water stains are weighted and summed after piecewise linear or Sigmoid normalization, and the weights are fitted on the validation set).

[0129] As a preferred option, the training and fine-tuning process module is:

[0130] S401 Initialization and Freeze Strategy

[0131] Load open-source weights; freeze the visual tower and base LLM backbone, making only minor adjustments: visual projection head (small steps), TS-Encoder, cross-modal fusion layer, and task head;

[0132] LoRA / Adapter is used to facilitate fast domain adaptation and training with small video memory.

[0133] S402 Multitasking Loss

[0134] Trend Forecast: ;

[0135] Uncertainty: (Gaussian hypothesis or quantile regression alternative);

[0136] Cross-modal alignment: Window-level matching loss (Image fragment <-> temporal fragment consistency);

[0137] Visual impairment: BCE / IoU combined loss with manually labeled or weakly supervised pseudo-labels;

[0138] Total loss .

[0139] S403 Data Arrangement and Command Alignment

[0140] Organize samples using "event slices" (before and after the anomaly) (Click window);

[0141] Optional command fine-tuning (cloud): Used to generate readable reports and interpretable explanations, but without changing the core discrimination head and formula system.

[0142] S404 Evaluation Indicators and Early Stop

[0143] Abnormal inflection points: recall / false alarm rate, alarm lead time;

[0144] Risk grading: Macro / micro average F1, confusion matrix;

[0145] Prediction range: Coverage (PICP), Range width (PINAW);

[0146] Early termination should be based on the optimal overall metric of the validation set.

[0147] As a preferred option, the trend prediction and abnormal inflection point identification module is:

[0148] S501 Residual Definition (Maintain Consistency)

[0149] ;

[0150] S502 Dual-threshold triggering and adaptive threshold (maintaining consistency)

[0151] ; ;

[0152] S503 confidence-driven noise reduction mechanism

[0153] when Increased environmental fluctuations (diurnal temperature range / passenger flow), The adjustment was subsequently made upwards to reduce false alarms;

[0154] like If the limit is reduced and there are consecutive minor violations, the system will trigger a mild warning (operational prompt) based on "cumulative minor violations" and will not directly upgrade the system.

[0155] S504 Missing / Drift Robust

[0156] Sensor drift is eliminated by "adjacent section comparison + temperature regression correction";

[0157] When a single mode is missing, the gating mechanism reduces its weight to avoid negatively impacting the fusion result.

[0158] As a preferred option, the comprehensive risk index and tiered early warning module...

[0159] S601 Risk Index (maintain consistency)

[0160] ;

[0161] Weighted average of sub-factors such as crack length / width, misalignment height, detachment area / volume, water stain area, and humidity gradient;

[0162] parameter Fitting from historical samples and validation sets (see Engineering Deployment, Parameter Calibration and Operation and Maintenance Modules).

[0163] S602 Classification and Disposal Mapping

[0164] : Prompt level (increase sampling frequency, increase inspection frequency);

[0165] Level II (Schedule a 24–48 hour review, and recommend temporary speed limits / load restrictions);

[0166] Level III (Immediate on-site verification and temporary reinforcement, shortening the operating window);

[0167] Level IV (Emergency Response / Close-off, Emergency Hazard Relief).

[0168] S603 Reporting and Interpretability

[0169] Output: Risk level, trend line , List of inflection points, contribution (modal / factor proportion), recommended actions, and version number of the model / threshold used (for audit traceability).

[0170] As a preferred option, the engineering deployment, parameter calibration, and operation and maintenance module...

[0171] S701 Dual-track Deployment

[0172] Lightweight edge computing (MiniCPM-V + TS-Encoder): Industrial PC / small GPU; completes first-level discrimination within 1–3 seconds and can provide local audio and visual alarms;

[0173] High-precision cloud-based (Qwen2.5-VL / InternVL2 + fusion head): Centralized secondary review, report archiving, and strategy orchestration.

[0174] S702 Parameter Calibration Procedure

[0175] Threshold class Based on historical "normal / abnormal" samples from multiple lines, optimization is performed using ROC-Youden metrics or cost-sensitive weighting.

[0176] Weight Class Bayesian optimization / genetic algorithm;

[0177] window : Combining sampling frequency with operation and maintenance response time limits (e.g.) =1440 points / day window, =96 points / Stable window, =60–120 minute forecast window).

[0178] Version S703 and Bloodline Management

[0179] Lineage is recorded using a quintuple of "model version - parameters - data slices - training script - evaluation report";

[0180] Each threshold change generates a new policy ID, which is cited in the report for traceability.

[0181] S704 Fault and Safety Redundancy

[0182] Single point of failure: Modal gating with automatic weight reduction;

[0183] Communication failure: edge caching and resume interrupted downloads;

[0184] False alarm control: When drastic environmental changes lead to The system automatically switches to "stable mode" when the price spikes. (Upgrade, only prompt, no upgrade).

[0185] Typical application results of S705 (example)

[0186] Running for 30 consecutive days: The temperature difference increases significantly on days 12–14. Upward adjustment to suppress false alarms; Day 21 When the rate of change exceeds the limit and the image crack factor increases, it triggers... On days 24–25, a displacement rate jump and significant image damage occurred, triggering… It was temporarily reinforced in conjunction with other measures, and the subsequent trend converged and the level dropped.

[0187] Example 2: A tunnel structure deformation identification and risk classification system based on a multimodal large model aims to automatically identify the deformation trend and classify the risk level of the tunnel structure by collecting and fusing multi-source data of the tunnel structure and combining it with large model analysis. The system includes: a multimodal data acquisition module for collecting multi-source monitoring data such as strain, displacement, temperature, and images of the tunnel structure, and performing preprocessing and synchronization; a large model analysis module with a built-in cross-modal large model for receiving the multimodal data, performing feature modeling and fusion, and outputting deformation trend prediction results and abnormal inflection point identification results; and a risk judgment and early warning module for assessing the tunnel risk level based on the large model analysis results and outputting alarm or visual early warning information.

[0188] I. Multimodal Data Acquisition Module

[0189] 1) The multimodal data acquisition module is used to collect and integrate data from multiple sensors in real time at key parts of the tunnel structure. The collected data includes, but is not limited to, strain data, displacement data, temperature and humidity data, and image data. By fusing multi-source data, it provides a comprehensive basis for risk assessment and early warning of tunnel structure deformation.

[0190] (1) Strain sensor unit: This unit includes multiple strain sensors distributed in different parts of the tunnel structure. It is mainly used to monitor strain changes in the tunnel wall and foundation in real time due to factors such as load, settlement, and temperature changes. The strain sensors are made of high-sensitivity materials, which can accurately reflect minute structural deformations and generate strain signals. The signals collected by the strain sensors are amplified, filtered, and then output digitally for subsequent data analysis.

[0191] (2) Displacement Sensor Unit: This unit is equipped with various types of displacement sensors, including but not limited to fiber optic displacement sensors, laser rangefinders, or laser displacement sensors. The displacement sensors are used to monitor the vertical and horizontal displacement changes of the tunnel structure in real time, particularly monitoring structural deformations such as settlement and displacement between the tunnel floor and walls. The displacement sensors can accurately measure displacements at the micrometer level and have high time resolution, meeting the needs of dynamic monitoring.

[0192] (3) Temperature and humidity sensor unit: This unit includes various environmental monitoring sensors, mainly used to monitor temperature and humidity changes inside the tunnel. The temperature and humidity sensors collect temperature and humidity data inside the tunnel in real time through high-precision sensing elements. Especially in the case of moisture in the tunnel, they can effectively reflect the impact of moisture changes on the tunnel structure. The data collected by the temperature and humidity sensors can provide a basis for subsequent analysis and help identify structural problems caused by temperature and humidity fluctuations.

[0193] (4) Image Acquisition Unit: This unit consists of multiple high-definition industrial cameras, used to capture image data of the tunnel's inner surface and other key areas. The image acquisition unit can monitor the tunnel's interior for defects such as cracks, misalignments, and leaks in real time using high-resolution cameras. The acquired image data is preliminarily analyzed by image processing algorithms and transmitted to subsequent analysis modules through a data interface.

[0194] 2) Data preprocessing and synchronization after acquisition

[0195] During data acquisition, multimodal data will be obtained simultaneously through different sensor modules. To ensure data accuracy and consistency, the system includes a data preprocessing and synchronization unit, which has the following functions:

[0196] Noise removal: This involves using digital signal processing algorithms to remove noise from the raw sensor data, ensuring that only valid signal components are retained. Common noise removal methods include Kalman filtering, mean filtering, and median filtering, which can effectively reduce the impact of the external environment on data quality.

[0197] Missing data imputation: During actual data acquisition, data may be missing due to equipment failure or transmission problems. The system uses interpolation algorithms (such as linear interpolation, spline interpolation, etc.) to supplement the missing data, ensuring data continuity and integrity.

[0198] Normalization: In order to eliminate the differences in the dimensions of different sensors, the system performs normalization processing on the collected data, unifying the data from different sensors to the same order of magnitude range, so as to facilitate comprehensive processing and fusion by the subsequent analysis module.

[0199] Timestamp Alignment: Due to potential time differences in multi-source data acquisition, the system employs a timestamp alignment mechanism to ensure consistency in the time points of various data types, facilitating accurate data alignment during subsequent analysis. For example, image acquisition data and sensor data may have time differences due to different acquisition frequencies; the system uses a time synchronization algorithm to ensure that these data are compared and analyzed on the same timeline.

[0200] Through the above preprocessing and synchronization operations, the system can ensure the consistency and validity of the collected multimodal data in time and space, providing high-quality data support for subsequent data fusion and intelligent analysis.

[0201] 3) Data Interface and Transmission: All pre-processed data collected will be transmitted to the central processing unit or cloud storage system via a high-speed data interface. Encryption protocols are used during data transmission to ensure data security. Furthermore, the system supports multiple transmission methods, including wireless networks and fiber optic communication, to adapt to the needs of different field environments.

[0202] The multimodal data acquisition module, through the collaborative operation of multiple sensor units, acquires multi-dimensional monitoring data within the tunnel in real time, including strain, displacement, temperature and humidity, and image data. Through meticulous data preprocessing and synchronization, high-quality and consistent data are ensured, providing reliable data support for subsequent large-scale model analysis and risk assessment. This module can be widely applied in tunnel structure health monitoring and risk early warning systems, featuring high precision, high timeliness, and high reliability.

[0203] II. Large Model Analysis Module

[0204] The large-scale model analysis module is the core component of the tunnel structure deformation identification and risk classification system of this invention. It is responsible for in-depth analysis and feature fusion of multimodal data from different sensors, automatically generating tunnel structure deformation trend predictions and conducting risk assessments. This module adopts a fusion of multimodal large-scale models, and through the flexible selection and replacement of multiple base models, it can optimize computational efficiency and analysis accuracy in different deployment environments. Different default large-scale models are available for cloud and edge computing devices, ensuring feasibility and efficiency on different computing platforms.

[0205] 1) Model Base and Replacement Strategy

[0206] (1) Cloud-based large model base: By default, the cloud uses Qwen2.5-VL or InternVL2 as the base model, both of which can effectively support multimodal data analysis and large-scale inference tasks.

[0207] Qwen2.5-VL: This model has powerful cross-modal understanding capabilities, can handle the fusion of large amounts of multi-source data, and provides high-quality structural health assessment based on pre-trained large-scale datasets.

[0208] InternVL2: As a cross-modal large model that focuses more on visual and language understanding, InternVL2 can more accurately process the fusion of image data and sensor data inside tunnels.

[0209] In addition, cloud deployment can replace other models that support large-scale multimodal inference, such as LLaVA-NeXT or PaliGemma 2, to flexibly adjust computing resources and accuracy according to specific tasks and hardware requirements.

[0210] (2) Large Model Base at the Edge: In edge computing environments, MiniCPM-V is used by default as the large model base. This model has high-efficiency computing capabilities and can perform fast inference on devices with limited resources. MiniCPM-V is optimized for deployment on low-power embedded devices or edge computing gateways with limited computing resources, supporting real-time data analysis and local risk prediction. Compared with cloud models, MiniCPM-V places greater emphasis on latency optimization and low-power operation, ensuring efficient operation in field deployments.

[0211] (3) Model replacement mechanism: The system supports flexible replacement of the base model on different deployment platforms to meet the needs of specific application scenarios. For example, Qwen2.5-VL can be selected on the cloud platform to process large-scale datasets, while MiniCPM-V can efficiently complete local inference tasks on edge computing devices. The system integrates a model replacement interface, allowing users to select different base models or replace custom models according to their actual needs.

[0212] 2) Multimodal data fusion and analysis: The large model analysis module performs in-depth fusion and analysis of data from multiple sensors through the following steps:

[0213] (1) Multimodal data preprocessing and feature extraction: The multimodal data preprocessing module performs unified processing on data from different sensors (such as strain, displacement, temperature and humidity, images, etc.). After filtering, normalization, noise reduction, missing value imputation and other operations, the data is sent to the large model analysis module for further feature extraction.

[0214] Strain and displacement data are used to extract structural deformation features through statistical methods and machine learning models.

[0215] Image data is used to extract features of defects such as cracks and misalignments using computer vision techniques (such as convolutional neural networks) and transformed into high-level visual representations.

[0216] Temperature and humidity data are fused with sensor data through time series analysis to extract environmental features related to structural deformation.

[0217] (2) Cross-modal feature fusion: The feature fusion unit uses a self-attention mechanism and a deep neural network to jointly model data from different modalities to obtain a high-dimensional multimodal feature vector. The self-attention mechanism can automatically assign weights to features from different modalities, thereby optimizing the contribution of different data sources to the structural health status. In this way, the system can make full use of data from different sensors and improve the accuracy of structural health assessment.

[0218] (3) Temporal Modeling and Prediction: The temporal prediction unit performs temporal modeling based on the features after multimodal fusion, predicts the future deformation trend of the tunnel structure, and identifies possible abnormal inflection points in a timely manner. This unit adopts a temporal modeling method based on the Transformer architecture, which provides high-precision trend analysis for deformation prediction by capturing long-term dependencies. The temporal modeling formula is as follows:

[0219] ;

[0220] in, Indicating a future moment Prediction of deformation trends For attention weights, This represents the corresponding value vector. Attention weights. The calculation formula is:

[0221] ;

[0222] (4) Anomaly Inflection Point Identification and Risk Warning: Based on time-series modeling results and real-time monitoring data, the system can dynamically identify anomalies in tunnel structural deformation. For example, when the tunnel deformation rate suddenly accelerates, the system can quickly identify the anomaly and trigger the warning mechanism by comparing historical deformation trends with real-time data. The warning information not only includes the trend changes in structural deformation but also indicates the risk level, and corresponding emergency response measures are taken according to different risk levels.

[0223] This tunnel structure deformation identification and risk classification system, based on a multimodal large model, achieves in-depth analysis and accurate prediction of multimodal data by flexibly selecting and replacing the base large model (such as Qwen2.5-VL, InternVL2, MiniCPM-V, etc.) at the cloud and edge. Through efficient feature fusion and temporal modeling, the system can assess the deformation trend and risk level of tunnel structures in real time and provide timely warnings. The system features high real-time performance, accuracy, and scalability, making it suitable for various tunnel monitoring and early warning application scenarios.

[0224] III. Risk Assessment and Early Warning Module

[0225] The risk assessment and early warning module is a key component of the tunnel structure deformation identification and risk classification system of this invention. It is primarily used to assess the risk level of the tunnel structure based on large-scale model analysis results and to trigger timely early warnings. This module utilizes a comprehensive judgment of multimodal data analysis results to dynamically assess the health status of the tunnel, providing timely and effective decision support for maintenance personnel.

[0226] 1) Risk scoring model

[0227] The risk assessment and early warning module calculates a risk index. To assess the risk level of the tunnel. This risk index... This formula is derived from a combination of the deformation characteristics of the tunnel structure and environmental changes, and is as follows:

[0228] ;

[0229] in:

[0230] It represents the change in strain, reflecting the degree of deformation of the tunnel structure caused by changes in load or environment;

[0231] It represents the amount of displacement change, indicating the horizontal or vertical displacement of the tunnel structure, and reflecting the direction and magnitude of structural deformation;

[0232] This indicates temperature fluctuations, which can affect the tunnel structure materials and lead to deformation.

[0233] This indicates information about cracks or misalignments identified in the image, including crack width, height, and other surface defects obtained through image analysis technology.

[0234] Weighting coefficient These parameters are obtained through learning and optimization of historical data using a large model. When calculating the risk index, these parameters are adjusted according to the weights of different data sources, thereby effectively reflecting the contribution of various data to the safety of the tunnel structure. The weighting coefficients can be determined based on past monitoring data of tunnel structures and the influence of various environmental factors.

[0235] 2) Risk assessment and classification

[0236] Based on the calculated risk index The system categorizes the risks of tunnel structures. By setting multiple risk level thresholds, the system can dynamically assess the health status of the tunnel and take different response measures for different risk levels. For example:

[0237] When the risk index When the temperature drops below the set safety threshold, the system determines that the tunnel is in a normal and safe state and does not trigger any warnings.

[0238] When the risk index When the tunnel exceeds the attention threshold but does not reach the danger threshold, the system determines that the tunnel is in a high-risk state and triggers a attention warning, recommending that monitoring be strengthened and preventive maintenance measures be taken.

[0239] When the risk index When the danger threshold is exceeded, the system determines that the tunnel structure faces a high risk and an emergency response must be initiated immediately, triggering an emergency warning and notifying relevant personnel to conduct on-site inspections and emergency response.

[0240] 3) Risk warning generation

[0241] Once the risk index is calculated If the preset threshold is exceeded, the system will generate and output a warning message. The warning message includes the following:

[0242] (1) Risk Heat Map: A risk heat map is a visual image generated based on the risk scores of different parts of the tunnel. The heat map uses color coding (e.g., green represents low risk, yellow represents medium risk, and red represents high risk) to show the risk distribution in different areas of the tunnel. In this way, maintenance personnel can clearly understand which areas have potential risks and prioritize their handling.

[0243] (2) Alarm Information: In addition to the risk heat map, the system will also generate detailed alarm information, including:

[0244] Deformation Trend: By comparing historical and real-time data, the deformation trend of the tunnel structure is analyzed and displayed to help maintenance personnel identify the reasons for the increased deformation.

[0245] Risk Level: Based on the calculated risk index, output the current risk level of the tunnel (e.g., safe, concern, dangerous).

[0246] Recommended Response Measures: Based on the current risk level, the system provides corresponding response measures suggestions. For example, in a high-risk situation, the system may suggest emergency measures such as structural reinforcement and construction intervention, and send early warning information to relevant personnel.

[0247] 4) Early warning information push and response mechanism

[0248] Once a risk warning is triggered, the system will push the warning information to relevant personnel through various methods. These methods include:

[0249] SMS / email notification: Send SMS or email notifications to designated operations and maintenance personnel to inform them of the warning level and specific response measures;

[0250] Mobile application push: The tunnel monitoring system pushes early warning notifications through its mobile application to ensure that on-site personnel can receive important information in real time;

[0251] Desktop Alerts: In the monitoring center, the system can alert maintenance personnel via desktop pop-ups and display detailed risk information and early warning charts.

[0252] 5) System response and feedback mechanism

[0253] The system's response mechanism dynamically adjusts based on the warning level. When the risk level reaches a preset danger threshold, the system automatically activates the emergency plan, initiating preliminary arrangements for on-site investigation and structural repair. On-site personnel can reinforce, drain, or implement other protective measures in severely deformed areas based on the warning information to prevent accidents. After the emergency response, the system continues to monitor the tunnel's status and adjusts and provides feedback on risks based on subsequent monitoring data.

[0254] 6) System optimization and adaptive capabilities

[0255] The risk assessment and early warning module is adaptive, capable of optimizing the risk scoring model based on accumulated historical data and new monitoring data. During system operation, users can adjust risk thresholds and weighting coefficients based on real-time data, thereby more accurately assessing the health status of the tunnel structure.

[0256] IV. Multimodal Feature Fusion Unit

[0257] The multimodal feature fusion unit is the core component of the large-scale model analysis and fusion mechanism. Its task is to effectively fuse data from different sensors (such as strain, displacement, temperature and humidity, images, etc.) to generate a comprehensive health characterization of the tunnel structure. By fusing data from multiple modalities, the system can comprehensively understand the deformation state of the tunnel and its influencing factors, thereby providing accurate basis for risk assessment and early warning.

[0258] 1) Cross-modal data coding

[0259] In the initial stage of multimodal feature fusion, data collected by various sensors are encoded and processed separately. Data from different sensors often have different characteristics; for example, strain and displacement data are continuous time-series data, image data have spatial characteristics, and temperature and humidity data have strong environmental dependence. To unify the processing of these heterogeneous data, the system uses the following method for data encoding:

[0260] (1) Cross-modal data coding

[0261] Time-series modeling of strain and displacement data: Strain and displacement data need to be processed according to the deformation of the tunnel structure. Assumptions For strain data, For displacement data, after standardization, an adaptive weighted LSTM model is used for time series modeling. The standardization formula is as follows:

[0262] ;

[0263] in, and They represent the first Strain and displacement data for each node, and These are the mean and standard deviation of the corresponding data. The standardized data is then used for time-series modeling via a recurrent neural network (RNN) or a long short-term memory network (LSTM). LSTM extracts time-series features through gating mechanisms (such as input gate, forget gate, and output gate), capturing long-term accumulated features during deformation, such as crack propagation and settlement trends.

[0264] The improved LSTM update formula is as follows: LSTM is used to model time series data.

[0265] ;

[0266] ;

[0267] ;

[0268] ;

[0269] ;

[0270] in, , , and These represent the changes in displacement, strain, and temperature, respectively. The coefficients are adjusted to enhance the influence of these features in the LSTM model.

[0271] Spatial Feature Extraction from Image Data: Image data (such as images of cracks or faults on the surface of tunnels) possess spatial features that require processing using computer vision techniques. Convolutional Neural Networks (CNNs) are used to extract features from these image data. CNNs effectively capture local features in images through multiple convolutional and pooling layers and transform them into high-level abstract features, such as the size, shape, and location of cracks.

[0272] Image data is processed using a convolutional neural network (CNN), and a local convolution weighting mechanism is employed for feature extraction, specifically targeting features such as cracks and faults on the tunnel surface. The improved convolution formula is as follows:

[0273] ;

[0274] in, For convolution kernel, For the first Image blocks, These are weighting coefficients, representing the importance of different regions in the model. This represents the convolution operation. It is a bias term. Weighting coefficient. This is learned and reflects the influence of image regions on the final features.

[0275] Temporal feature extraction of temperature and humidity data:

[0276] Temperature and humidity data reflect changes in the tunnel's internal environment and exhibit strong time-series characteristics, affecting the stability of the tunnel structure. Time-series analysis methods (such as moving average and weighted average) are used to smooth the temperature and humidity data, and a Long Short-Term Memory (LSTM) network is further used to extract its temporal dependence features.

[0277] Temperature and humidity data were analyzed using time series analysis to extract their features, and then processed using an adaptive weighted moving average method. The moving average formula is as follows:

[0278] ;

[0279] in, For temperature and humidity data, These are weighting coefficients used to control the impact of historical data on the current time step prediction. This represents the sliding window size. By using a weighted average, the system can flexibly adjust the contribution of historical temperature and humidity to the current state prediction.

[0280] (2) Cross-modal self-attention mechanism

[0281] To effectively integrate features from different sensors, the system employs an adaptive weighted self-attention mechanism. This mechanism dynamically learns the correlations between data from different modalities and weights them according to the importance of the data.

[0282] How self-attention works: Self-attention evaluates the relevance between features by calculating the similarity between each feature and generating a weighted sum. Specifically, the system first maps all input features to query, key, and value vectors. Then, it calculates the inner product between the query vector and the key vector to measure their similarity.

[0283] Adaptive weighted attention mechanism: This mechanism builds upon the traditional self-attention mechanism by incorporating an importance assessment for each modal feature. The specific formula is as follows:

[0284] ;

[0285] in, It is the first A query vector with input features It is the first A key vector of input features, It is the attention weight, representing the first... The first feature and the second The correlation between features. New additions. The adaptive adjustment coefficient can be dynamically adjusted according to the data type and importance of different sensors.

[0286] Then, the weighted sum is calculated using the following formula to generate the fused feature representation:

[0287] ;

[0288] in, It is the first A vector of values ​​for each feature. It is the feature representation after fusion.

[0289] Feature weighting and enhancement:

[0290] For example, crack features in an image may be closely related to deformation patterns in displacement data. The system can automatically strengthen this correlation through a self-attention mechanism so that the model can more accurately capture the relationship between crack propagation and structural deformation.

[0291] Similarly, strain data and temperature and humidity data may have a greater impact on structural health in certain situations. The self-attention mechanism will automatically adjust the weight of these data according to the specific context, ensuring that the model fully considers these influencing factors when making predictions.

[0292] (3) Feature fusion and multidimensional feature space mapping

[0293] The features processed by the self-attention mechanism will be further passed to the multi-dimensional feature space mapping layer. In this layer, the features of various sensors will be mapped to a unified multi-dimensional feature space, thereby forming a comprehensive representation of the health status of the tunnel structure.

[0294] Feature fusion: In this stage, data from various sensors (strain, displacement, temperature and humidity, images, etc.) are weighted through a self-attention mechanism to become a high-dimensional feature representation. This feature representation combines information from multiple aspects such as time, space, and environment to comprehensively describe the health status of the tunnel.

[0295] These fused features are then passed to the next layer for further processing. Through multi-layer networks, fusion and dimensionality reduction are performed, so that the final feature representation retains the key information from each data source while simplifying redundant information.

[0296] Deep Neural Network (DNN) Processing and Mapping: The fused features are processed by a deep neural network (DNN). The DNN uses multiple layers of nonlinear transformations to map the fused high-dimensional features to a low-dimensional unified representation.

[0297] This unified representation provides a clear health status characterization for subsequent deformation prediction, risk assessment, and early warning. Through its multi-layered structure, the DNN effectively extracts global patterns and details, ensuring that the model can simultaneously consider structural deformation over long time scales and short periods.

[0298] Feature fusion and optimized representation:

[0299] Through an adaptive weighted attention mechanism, the system is able to dynamically adjust the weights of each modality of data, ensuring that the correlation between different sensor features is fully explored.

[0300] After feature fusion, the system will further process the data using a deep neural network (DNN) to map the fused high-dimensional features to a low-dimensional unified representation. The improved DNN processing formula is as follows:

[0301] ;

[0302] in, It is a high-dimensional feature representation after fusion. and These are the weights and biases of the DNN. It is an activation function (such as ReLU). Through a multi-layered structure, DNN can extract global features and perform dimensionality reduction, ultimately obtaining a low-dimensional representation of the tunnel's health state. .

[0303] Global Pattern and Detail Capture: This low-dimensional representation effectively provides a comprehensive and concise health status characterization for subsequent deformation prediction, risk assessment, and early warning. Through layer-by-layer processing of deep neural networks, the system can simultaneously consider structural deformation over long time scales and short periods, ensuring that the model has efficient global pattern capture capabilities.

[0304] The multimodal feature fusion unit successfully integrates data from different sensors by combining a cross-modal self-attention mechanism and a deep neural network. The self-attention mechanism enables the system to dynamically learn the correlation between various types of data, thereby optimizing the feature fusion effect; while the deep neural network ensures that the fused features can effectively capture the global patterns and details of tunnel deformation, thus providing high-quality input for subsequent deformation prediction and risk assessment.

[0305] Example 3: A computer-readable storage medium for executing the intelligent identification and risk classification method for tunnel structure deformation described in this invention. This medium may be a hard disk, solid-state storage, flash memory, embedded memory card, edge computing device ROM, FPGA onboard RAM, or a virtual disk in a cloud server.

[0306] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for tunnel structure deformation identification and risk classification based on a multimodal large model, characterized in that, Includes the following steps: Multimodal data acquisition and preprocessing: Acquire multi-source monitoring data of the tunnel structure, including strain, displacement, temperature and image data. Perform noise filtering, outlier removal, missing data imputation, normalization and timestamp alignment on the data to form a unified input format. Large-scale model analysis and feature fusion: The preprocessed multimodal data is input into a multimodal large-scale model. Based on pre-trained cross-modal semantic understanding capabilities, the large-scale model models the long-term dependencies and change patterns of sensor time-series data, extracts structural damage features from image data, and fuses the representations of each modality to form a comprehensive vector representation of the tunnel structure state. Trend prediction: Using the large model, the key deformation parameters of the tunnel structure are predicted over time series to obtain the deformation development trend within a predetermined time range in the future; Anomaly inflection point identification: Based on the deviation of actual monitoring data from model prediction results, anomaly inflection points in the deformation process are automatically identified. When the deviation exceeds the dynamic threshold or abrupt changes occur continuously, the corresponding time point is marked as an anomaly inflection point. Risk level assessment: Combining abnormal inflection point characteristics, deformation trend prediction results and damage information from image recognition, the risk level of the tunnel structure is automatically assessed according to preset safety assessment standards, and an early warning is triggered when a high risk level is reached.

2. The method for tunnel structure deformation identification and risk classification based on a multimodal large model according to claim 1, characterized in that, The multimodal large model is a cross-modal pre-trained model based on an improved attention mechanism, and its feature interaction weights are defined by the following formula: ； in, For the first A query vector for modal features. For the first The key vector of each modal feature The dimension of the key vector. For nonlinear modulation coefficients, It is a dynamic equilibrium factor.

3. The method for tunnel structure deformation identification and risk classification based on a multimodal large model according to claim 1, characterized in that, The multi-source monitoring data includes: Strain data acquired by strain gauges; Converging displacement or settlement data collected by displacement sensors; Ambient temperature data collected by a temperature sensor; Image data of tunnel lining and internal structure collected by camera equipment.

4. The method for tunnel structure deformation identification and risk classification based on a multimodal large model according to claim 1, characterized in that, The risk level determination is based on a comprehensive risk index. The calculation formula is as follows: ； in, To monitor the amount of deformation, The deformation threshold, For the rate threshold, To calculate the window length, The number of abnormal inflection points. This represents the total number of time periods. To identify damage factors in images, For model parameters, This is a nonlinear function used to normalize the image damage results.

5. The method for tunnel structure deformation identification and risk classification based on a multimodal large model according to claim 1, characterized in that, The identification of the abnormal inflection point is based on the improved residual change rate criterion, wherein the residual is defined as: ; When satisfied When that time point is determined to be an abnormal inflection point, then... These are measured values. For predicted values, For time step, For historical residual weights, The rate of change threshold, This is the threshold value for the residual amplitude.

6. The method for tunnel structure deformation identification and risk classification based on a multimodal large model according to any one of claims 1 to 5, characterized in that, The method further includes an intelligent early warning and response generation step, specifically: trend prediction based on the output of the multimodal large model. Compared with measured values Calculate the weighted historical residuals. ； in Historical residual weights; Calculate the comprehensive risk index based on the fused multimodal features ； in This represents the current deformation. The deformation threshold, For the rate threshold, For window length, The number of abnormal inflection points. This represents the total number of time periods. Image damage factor, For model parameters, It is a nonlinear normalization function; Construct an adaptive joint discrimination threshold, where the magnitude threshold and the rate of change threshold are respectively: ； in The prediction uncertainty estimate given for the large model, For near absolute deviation of the midpoint, For threshold parameters; When satisfied When ; it is automatically determined to be the first The system assesses risk level and generates a comprehensive early warning report, which includes at least the risk level, future forecast trend, and timeframe. The system collects abnormal inflection points and recommends appropriate measures, which are then pushed to the tunnel management terminal or remote monitoring platform via a human-machine interface or network communication module. This enables automatic graded early warning of tunnel structural safety risks based on multi-source data fusion, trend prediction, and automatic identification of abnormal inflection points.

7. A tunnel structure deformation identification and risk classification system based on a multimodal large model, characterized in that, include: The multimodal data acquisition module is used to acquire multi-source monitoring data such as strain, displacement, temperature and images of the tunnel structure, and to perform preprocessing and synchronization. The large model analysis module has a built-in cross-modal large model, which is used to receive the multimodal data and perform feature modeling and fusion, and output deformation trend prediction results and abnormal inflection point identification results. The risk assessment and early warning module is used to assess the tunnel risk level based on the analysis results of the large model and output alarm or visual early warning information.

8. The tunnel structure deformation identification and risk classification system based on a multimodal large model according to claim 7, characterized in that, The system can be deployed on edge computing devices at the tunnel site for real-time processing, or deployed on a cloud platform to centrally receive and analyze sensor data, thereby supporting flexible deployment in different application scenarios.

9. The tunnel structure deformation identification and risk classification system based on a multimodal large model according to claim 7, characterized in that, The large model analysis module further includes a multimodal feature fusion unit and a temporal prediction unit, wherein: The multimodal feature fusion unit is used to extract and uniformly represent cross-modal features of strain, displacement, temperature and image data collected by different sensors, so as to avoid the problem of insufficient prediction accuracy of a single sensor. The time-series prediction unit is used to automatically model the deformation development trend based on fusion features and dynamically identify abnormal inflection points in combination with actual monitoring values, thereby achieving automatic risk level classification without the need for manual experience rules.

10. A computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the method as claimed in any one of claims 1 to 6.

Images