Foundation pit deformation intelligent early warning system and method based on multi-modal fusion

Abstract

The invention relates to the technical field of engineering safety monitoring, in particular to a foundation pit deformation intelligent early warning system based on multi-modal fusion and a method thereof.According to the system, quality evaluation and weighting processing are conducted on multi-modal sensor data through a self-adaptive weight dynamic distribution module, and the data credibility is ensured; the multi-modal feature cross extraction module extracts and interacts features by using a specific sub-network and a multi-head attention mechanism, integrates information through a space-time diagram convolutional network, and generates accurate fusion feature representation; the multi-granularity abnormal mode identification module is combined with a mixed density network and time sequence analysis to accurately identify deformation anomalies; the causal reasoning and weight feedback module analyzes deformation reasons through a causal graph model and provides feedback for sensor weight adjustment; according to the system, the precision and reliability of deformation detection are remarkably improved, the detection precision is improved to the millimeter level, the accuracy is improved by 40%, and powerful technical support is provided for engineering safety monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of engineering safety monitoring, and particularly to an intelligent early warning system and method for foundation pit deformation based on multi-modal fusion, which is applicable to deformation monitoring and early warning during the construction process of foundation pits in the fields of construction engineering, municipal engineering, traffic engineering, etc. Background Technique

[0002] With the in-depth development of urban construction, the number of deep foundation pit projects is increasing continuously, the scale is expanding continuously, and the construction conditions are becoming increasingly complex. The safety of foundation pit projects is related to the safety of surrounding buildings, underground pipelines and construction personnel. Deformation monitoring and early warning have become a key link to ensure the safety of foundation pit construction.

[0003] The prior art such as Chinese Patent CN116953696A discloses a foundation pit deformation detection method and system. This technology monitors the surrounding environment of the foundation pit by means of electromagnetic wave scanning, establishes echo points based on the scanned echoes, screens the areas that need secondary scanning according to the distances between the echo points, constructs a feature model based on the echo point group, and issues an early warning signal by comparing the changes in the feature models in the time series.

[0004] However, the above technology has the following deficiencies: First, relying only on a single electromagnetic wave scanning modality and lacking multi-source data verification, it is easily affected by environmental interference and leads to misjudgment; Second, using fixed parameter thresholds and early warning ranges, it cannot adapt to the characteristic changes of different geological conditions and construction stages; Third, the feature extraction method is simple, relying only on basic distance calculation and plane construction, lacking in-depth feature mining; In addition, it cannot distinguish between deformations caused by normal construction and abnormal deformations, and the early warning mechanism is simple, only issuing an early warning based on the change trend exceeding the range, and unable to provide analysis of the deformation causes and treatment suggestions; Finally, each monitoring terminal analyzes independently, lacking global correlation analysis, the system cannot learn and improve from historical data, and lacks intelligence and adaptability. Summary of the Invention

[0005] The purpose of the present invention is to provide an intelligent early warning system and method for foundation pit deformation based on multi-modal fusion, aiming to solve the problems of single-modal perception limitation, fixed parameter thresholds, simple feature extraction, lack of construction context recognition, single-point data analysis, simple early warning mechanism and lack of learning ability in the prior art.

[0006] The present invention proposes an intelligent early warning system for foundation pit deformation based on multi-modal fusion, including:

[0007] An adaptive weight dynamic allocation module, which is used to evaluate the quality of data from multi-modal sensors, generate a weight matrix representing the credibility of each sensor under the current environmental conditions, and output the weighted multi-modal data;

[0008] The multimodal feature cross-extraction module, connected to the adaptive weight dynamic allocation module, is used to receive the weighted multimodal data, extract initial features through modality-specific sub-networks, implement cross-modal feature interaction using the multi-head attention mechanism, and integrate spatial correlations through the spatio-temporal graph convolutional network to generate a fused feature representation;

[0009] The multi-granularity anomaly pattern recognition module, connected to the multimodal feature cross-extraction module, is used to receive the fused feature representation, apply a mixture density network to predict the probability distribution of deformation metrics, and identify anomaly patterns by combining time series decomposition and contrast learning;

[0010] The causal inference and weight feedback module, connected to the multi-granularity anomaly pattern recognition module, is used to receive the anomaly pattern, infer the cause of deformation based on the causal graph model, calculate the sensor contribution, and feedback weight adjustment suggestions to the adaptive weight dynamic allocation module;

[0011] The distributed federated learning module, connected to the multimodal feature cross-extraction module, the multi-granularity anomaly pattern recognition module, and the causal inference and weight feedback module, is used to coordinate knowledge sharing and model optimization among multiple projects while protecting the privacy of the original data.

[0012] Preferably, the adaptive weight dynamic allocation module includes:

[0013] The sensor quality assessment unit, used to calculate a scoring matrix representing the data quality of each sensor;

[0014] The environmental impact assessment unit, used to generate an environmental correction factor representing the impact degree of the current environmental parameters on each sensor;

[0015] The weight calculation unit, used to generate the weight matrix at the current moment based on the scoring matrix, historical weights, and the environmental correction factor;

[0016] Among them, the weight matrix W(t) is calculated by the following formula:

[0017] W(t) = Softmax[α·Q(s,t) + β·W(t - 1) + γ·C(t)],

[0018] In the formula, Q(s,t) represents the data quality score of sensor s at time t, W(t - 1) represents the weight at the previous moment, C(t) represents the current environmental context vector, and α, β, γ are adjustable hyperparameters.

[0019] Preferably, the multimodal feature cross-extraction module includes:

[0020] The modal-specific feature extraction unit includes a vibration feature extraction sub-network, an inclination feature extraction sub-network, a pressure feature extraction sub-network, and a water level feature extraction sub-network, which are respectively used to extract the initial features of the corresponding modality;

[0021] The cross-modal attention unit is used to calculate the interaction relationship between different modality features and generate cross-feature representations;

[0022] The spatio-temporal graph convolution unit is used to construct a graph structure based on the physical layout of the sensors and integrate spatial correlation and temporal dependence through graph convolution operations.

[0023] Preferably, the cross-modal attention unit realizes feature interaction in the following way:

[0024] F ij = MultiHeadAttention(Q i , K j , V j ),

[0025]

[0026] where i and j represent different modalities, is a learnable projection matrix.

[0027] Preferably, the multi-granularity anomaly pattern recognition module includes:

[0028] The deformation index prediction unit is used to map the fused features into basic displacement indices;

[0029] The mixture density network unit is used to model the multi-modal probability distribution for each deformation index;

[0030] The time series decomposition unit is used to decompose the deformation sequence into a trend term, a seasonal term, and a residual term;

[0031] The contrastive learning detection unit is used to evaluate the differences between the current state and known normal and abnormal patterns.

[0032] Preferably, the mixture density network unit models the conditional probability distribution of the deformation index y through the following formula:

[0033]

[0034] where, π k is the weight of the k-th mixture component, N represents the Gaussian distribution, μ k and σ k are the mean and standard deviation of the k-th component respectively.

[0035] Preferably, the causal reasoning and weight feedback module includes:

[0036] A causal graph construction unit for establishing a directed graph representing the causal relationship between observed variables and latent variables;

[0037] A causal intervention analysis unit for calculating the effects under different intervention strategies;

[0038] A counterfactual reasoning unit for evaluating the deformation development paths in different scenarios;

[0039] A sensor influence degree evaluation unit for calculating the contribution degree of each sensor to anomaly detection.

[0040] Preferably, the sensor influence degree evaluation unit calculates the influence degree of sensor s through the following formula:

[0042] Influence(s) = E[|P(A|D) - P(A|D\{s\})|],

[0043] where A represents an abnormal event, D represents all sensor data, D\{s\} represents the data set after removing sensor s, and E represents the expected value.

[0044] Preferably, the distributed federated learning module includes:

[0045] A local training unit for training a local model on each engineering project;

[0046] A secure aggregation unit for aggregating the model updates of each project while protecting privacy;

[0047] A domain adaptation unit for handling the distribution shift between different projects;

[0048] A knowledge distillation unit for transferring complex model knowledge to a lightweight model for edge deployment.

[0049] An intelligent early warning method for foundation pit deformation based on multimodal fusion, the method includes:

[0050] Collect multimodal sensor data, evaluate the quality of each sensor data, generate the current weight matrix based on environmental factors and historical weights, and obtain weighted multimodal data;

[0051] Perform modality-specific feature extraction on the weighted multimodal data, achieve cross-modal feature interaction through a multi-head attention mechanism, and integrate the spatial relationships of sensors by combining a spatio-temporal graph convolutional network to generate a fused feature representation;

[0052] Based on the fused feature representation, apply a mixture density network to predict the probability distribution of deformation indicators, and identify abnormal patterns through time series decomposition and contrast learning;

[0053] Use the causal graph model to infer the possible causes of the abnormal pattern, calculate the contribution degree of each sensor to anomaly detection, and generate weight adjustment suggestions;

[0054] Coordinate knowledge sharing among multiple engineering projects through distributed federated learning, use domain adaptation technology to handle distribution shift, optimize the model and apply it to subsequent monitoring.

[0055] The present invention has the following beneficial effects compared with the prior art:

[0056] 1. The accuracy and reliability of deformation detection are improved through multi-modal data fusion, the detection accuracy is improved from centimeter level to millimeter level, and the accuracy rate is increased by 40%;

[0057] 2. The innovative adaptive weight allocation mechanism can dynamically adjust the contribution degree of each sensor according to environmental conditions and data quality, and greatly improve the adaptability of the system to complex environments;

[0058] 3. The multi-modal feature cross-extraction network effectively captures the complementary relationship between different types of sensing data, and enhances the expression ability of the system for complex deformation patterns;

[0059] 4. The multi-granularity abnormal pattern recognition technology combines causal reasoning ability, which can not only detect anomalies, but also explain the reasons for deformation, providing a scientific basis for engineering decisions;

[0060] 5. The distributed knowledge transfer mechanism based on federated learning enables the system to learn from historical engineering experience, and the adaptation time of new projects is shortened from 7 days to 1 day, greatly improving the deployment efficiency. Brief Description of the Drawings

[0061] Figure 1 is a schematic diagram of the overall system architecture of the present invention;

[0062] Figure 2 is a schematic diagram of the structure of the adaptive weight dynamic allocation module of the present invention;

[0063] Figure 3 is a schematic diagram of the structure of the multi-modal feature cross-extraction module of the present invention;

[0064] Figure 4 is a schematic diagram of the structure of the multi-granularity abnormal pattern recognition module of the present invention;

[0065] Figure 5 is a schematic diagram of the structure of the causal reasoning and weight feedback module of the present invention;

[0066] Figure 6 is a schematic diagram of the structure of the distributed federated learning module of the present invention;

[0067] Figure 7It is the flow chart of the intelligent early warning method for foundation pit deformation of the present invention;

[0068] Figure 8 It is the deployment schematic diagram of the present invention in actual engineering applications;

[0069] Figure 9 It is the comparison chart of the system early warning effect of the present invention;

[0070] Figure 10 It is the schematic diagram of the system data flow of the present invention. Detailed implementation manners

[0071] Please refer to the appendix Figure 1-10 , and the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0072] Embodiment 1: Overall system architecture

[0073] As Figure 1 shown, the intelligent early warning system for foundation pit deformation based on multi-modal fusion provided by the present invention includes an adaptive weight dynamic allocation module 1, a multi-modal feature cross extraction module 2, a multi-granularity abnormal pattern recognition module 3, a causal reasoning and weight feedback module 4, and a distributed federated learning module 5.

[0074] The adaptive weight dynamic allocation module 1 is used to evaluate the quality of data from multi-modal sensors, generate a weight matrix representing the credibility of each sensor under the current environmental conditions, and output the weighted multi-modal data. The multi-modal sensors include but are not limited to vibration sensors, tilt sensors, pressure sensors, and groundwater level sensors, etc.

[0075] The multi-modal feature cross extraction module 2 is connected to the adaptive weight dynamic allocation module 1, and is used to receive the weighted multi-modal data, extract initial features through modality-specific sub-networks, use the multi-head attention mechanism to achieve cross-modal feature interaction, and integrate spatial correlation through the spatio-temporal graph convolutional network to generate a fused feature representation.

[0076] The multi-granularity abnormal pattern recognition module 3 is connected to the multi-modal feature cross extraction module 2, and is used to receive the fused feature representation, apply a mixture density network to predict the probability distribution of the deformation index, and identify abnormal patterns by combining time series decomposition and contrast learning.

[0077] The causal reasoning and weight feedback module 4 is connected to the multi-granularity abnormal pattern recognition module 3, and is used to receive the abnormal pattern, infer the deformation cause based on the causal graph model, calculate the sensor contribution degree, and feedback weight adjustment suggestions to the adaptive weight dynamic allocation module 1.

[0078] The distributed federated learning module 5 is connected to the multi-modal feature cross-extraction module 2, the multi-granularity anomaly pattern recognition module 3, and the causal inference and weight feedback module 4, and is used to coordinate knowledge sharing and model optimization among multiple projects while protecting the privacy of the original data.

[0079] The data flow between the modules is as follows: First, the adaptive weight dynamic allocation module 1 receives the original data collected by the multi-modal sensors. After preprocessing and weight allocation, the weighted data is transmitted to the multi-modal feature cross-extraction module 2; then, the multi-modal feature cross-extraction module 2 extracts the fusion features and transmits them to the multi-granularity anomaly pattern recognition module 3; then, the multi-granularity anomaly pattern recognition module 3 outputs the anomaly patterns and transmits them to the causal inference and weight feedback module 4; then, the causal inference and weight feedback module 4 generates weight adjustment suggestions and feeds them back to the adaptive weight dynamic allocation module 1, and at the same time outputs the deformation explanation and processing suggestions; finally, the distributed federated learning module 5 regularly collects the model updates of each module, performs model optimization, and distributes the updated model parameters back to each functional module to form a complete closed-loop data flow.

[0080] Embodiment 2: Adaptive Weight Dynamic Allocation Module

[0081] As Figure 2 shown, the adaptive weight dynamic allocation module 1 includes a sensor quality assessment unit 11, an environmental impact assessment unit 12, and a weight calculation unit 13.

[0082] The sensor quality assessment unit 11 is used to calculate a score matrix representing the data quality of each sensor. Specifically, the sensor quality assessment unit 11 generates a quality score Q(s,t) by analyzing the signal-to-noise ratio, integrity, and historical consistency of the sensor data, where s represents the sensor identifier and t represents the current time. The quality score calculation considers the following factors: signal stability, data integrity rate, historical accuracy performance, and consistency with adjacent sensor data.

[0083] The environmental impact assessment unit 12 is used to generate an environmental correction factor representing the degree of influence of the current environmental parameters on each sensor. The environmental parameters E include external factors such as temperature, humidity, and vibration intensity that may affect the performance of the sensors. For each type of sensor, according to its sensitivity to different environmental factors, an environmental correction factor is calculated to adjust the weight calculation process.

[0084] The weight calculation unit 13 is used to generate a weight matrix at the current time based on the score matrix, historical weights, and environmental correction factors. The weight matrix W(t) is calculated by the following formula:

[0085] W(t) = Softmax[α·Q(s,t) + β·W(t - 1) + γ·C(t)],

[0086] Among them, Q(s,t) represents the data quality score of sensor s at time t, W(t - 1) represents the weight at the previous time, C(t) represents the current environmental context vector, and α, β, and γ are adjustable hyperparameters that control the influence degree of each factor on weight update. Preferably, the value range of α is 0.4 - 0.6, the value range of β is 0.2 - 0.4, and the value range of γ is 0.1 - 0.3. The Softmax function ensures that the sum of all weights is 1, making the weights of each sensor relative values.

[0087] In practical applications, this module dynamically adjusts the weight allocation according to time and environmental changes, enabling the system to adaptively handle situations such as sensor failures and environmental interferences. For example, when a certain sensor is strongly interfered with and the data quality deteriorates, the system will automatically reduce its weight and increase the weights of other normally working sensors to ensure that the overall performance of the system is not affected by single-point failures.

[0088] Embodiment 3: Multimodal Feature Cross-Extraction Module

[0089] As Figure 3 shown, the multimodal feature cross-extraction module 2 includes a modality-specific feature extraction unit 21, a cross-modal attention unit 22, and a spatio-temporal graph convolutional unit 23.

[0090] The modality-specific feature extraction unit 21 includes a vibration feature extraction sub-network 211, an inclination feature extraction sub-network 212, a pressure feature extraction sub-network 213, and a water level feature extraction sub-network 214, which are respectively used to extract the initial features of the corresponding modalities.

[0091] The vibration feature extraction sub-network 211 uses wavelet transform and a multi-scale convolutional network to extract frequency-domain features and time-domain features from vibration signals, and can effectively identify different types of geological vibration patterns. The inclination feature extraction sub-network 212 mainly processes data from inclination sensors and captures the inclination change rate and acceleration features through a gradient feature extractor. The pressure feature extraction sub-network 213 adopts a residual network structure to process data such as soil pressure and water pressure and extract pressure distribution and change trend features. The water level feature extraction sub-network 214 uses a long short-term memory network to extract long-term trends and short-term fluctuation features from groundwater level data.

[0092] The cross-modal attention unit 22 is used to calculate the interaction relationship between different modality features and generate cross-feature representations. This unit realizes information interaction between modalities and is calculated in the following way:

[0093] F ij = MultiHeadAttention(Qi , K j , V j ),

[0094]

[0095] where i and j represent different modalities, is a learnable projection matrix. The multi-head attention mechanism allows the model to focus on multiple interrelationships between different modality features, enhancing the expressive power of feature fusion.

[0096] The spatio-temporal graph convolutional unit 23 is used to construct a graph structure based on the physical layout of the sensors, and integrate spatial correlation and temporal dependence through graph convolutional operations. In this unit, each sensor is represented as a node v in the graph G, and the edge connection weight e(v, u) is calculated based on physical distance and correlation, and graph convolutional operations are applied to extract spatially correlated features.

[0097] The main advantage of this module is its ability to capture the complementarity and correlation between different sensor data, enhancing the system's expressive power for complex deformation patterns through cross-modal feature interaction. At the same time, the spatio-temporal graph convolutional structure effectively integrates the spatial distribution information of the sensors, enabling the system to understand the spatial propagation characteristics of deformations.

[0098] Example 4: Cross-modal attention unit of the multi-modal feature cross-extraction module

[0099] Furthermore, as Figure 3 shown, the cross-modal attention unit 22 realizes the interaction between different modality features through the multi-head attention mechanism, enhancing the system's representation ability for complex deformation patterns.

[0100] The multi-head attention mechanism allows the model to learn information from different representation subspaces, enhancing the diversity of feature interaction. In the cross-modal attention unit, first, the features of each modality are linearly projected into the query (Query), key (Key), and value (Value) spaces:

[0101]

[0102] where Feature i and Feature j represent the feature vectors of modality i and modality j respectively, are the corresponding projection matrices.

[0103] Then, calculate the attention weights and obtain the weighted features:

[0104] F ij = MultiHeadAttention(Q i , Kj , V j ),

[0105] The multi - head attention mechanism is defined as:

[0106] MultiHeadAttention(Q, K, V) = Concat(head1, head2,..., head h )W O ,

[0107] where each head is calculated as follows:

[0108]

[0109] In practical applications, the preferred number of heads h is 4 or 8, and the dimension d k is determined according to the input feature dimension. The attention mechanism forms weighted connections between different modalities, allowing the system to learn the mutual influence relationships between modalities such as vibration - tilt, pressure - water level, etc., and greatly improving the system's understanding ability of complex deformation situations.

[0110] In this way, the system can automatically learn the important associations between different modalities, such as the relationship between groundwater level changes and soil pressure changes, or the temporal association between vibration events and subsequent tilt changes, thus providing a more comprehensive perception of the deformation situation.

[0111] Example 5: Multi - granularity anomaly pattern recognition module

[0112] As Figure 4 shown, the multi - granularity anomaly pattern recognition module 3 includes a deformation index prediction unit 31, a mixture density network unit 32, a time - series decomposition unit 33, and a contrastive learning detection unit 34.

[0113] The deformation index prediction unit 31 is used to map the fused features to basic displacement indices. This unit receives the fused feature representation output by the multi - modal feature cross - extraction module and maps it to a series of basic deformation indices through a fully - connected layer, including key monitoring parameters such as settlement amount, horizontal displacement, tilt angle, deformation rate, etc.

[0114] The mixture density network unit 32 is used to model the multi - modal probability distribution for each deformation index. Traditional methods usually only predict a single value, while this unit can estimate the complete probability distribution to more accurately express the uncertainty of the prediction. The mixture density network models the conditional probability distribution of the deformation index y through the following formula:

[0115]

[0116] where π k is the weight of the k - th mixture component, N represents the Gaussian distribution, μk and σ k are the mean and standard deviation of the k-th component respectively. In this way, the system can represent the multimodal distribution of the deformation index, taking into account multiple possibilities under different deformation modes.

[0117] The time series decomposition unit 33 is used to decompose the deformation sequence into a trend term, a seasonal term, and a residual term. This unit adopts time series decomposition technology to decompose the deformation time series X(t) into:

[0118] Trend term T(t): representing the long-term change trend;

[0119] Seasonal term S(t): representing the periodic change pattern;

[0120] Residual term R(t): representing random fluctuations and potential anomalies;

[0121] By separating these components, the system can distinguish the deformation trend caused by normal construction from abnormal fluctuations, improving the warning accuracy.

[0122] The contrastive learning detection unit 34 is used to evaluate the differences between the current state and known normal and abnormal patterns. This unit compares the current observed data with normal and abnormal patterns in the historical database, calculates the similarity score, and identifies potential anomalies based on the similarity. Contrastive learning enhances the system's generalization ability to new types of anomalies by maximizing the similarity between normal pattern representations while minimizing the similarity to abnormal patterns.

[0123] The innovation of this module lies in combining probability modeling and time series analysis techniques to achieve anomaly detection from single-threshold judgment to multi-dimensional and multi-granularity. It can not only identify obvious sudden anomalies but also discover potential risks with slow evolution, significantly improving the time lead and accuracy of early warning.

[0124] Example 6: Mixture Density Network Unit

[0125] Furthermore, as Figure 4 shown, the mixture density network unit 32 provides a complete uncertainty representation of deformation prediction by modeling the conditional probability distribution of the deformation index.

[0126] Traditional methods usually only output a single predicted value and cannot express the uncertainty and multimodality of the prediction. While the mixture density network can represent complex conditional probability distributions, including multimodal distributions, skewed distributions, etc., by combining multiple Gaussian distributions. This network models the conditional probability distribution of the deformation index y through the following formula:

[0127]

[0128] where π kπ(x) is the weight of the k-th mixture component, and π k (x) ≥ 0; N represents the Gaussian distribution μ k (x) and σ k (x) are the mean and standard deviation of the k-th component respectively; K is the number of mixture components, usually taking values from 3 to 5. All parameters π k (x), μ k (x) and σ k (x) are all generated by the neural network according to the input feature x.

[0129] The network training uses the negative log-likelihood loss function:

[0130]

[0131] In practical applications, this unit can generate a complete predictive distribution, not only outputting the most likely deformation value, but also providing a prediction interval and the probabilities of different scenarios. For example, for the prediction of foundation pit settlement, the system may give a prediction result with a 70% probability of settlement of 1 - 2 mm, a 20% probability of settlement of 2 - 3 mm, and a 10% probability of settlement of 3 - 5 mm, providing more comprehensive risk assessment information than a single predicted value of settlement of 2 mm.

[0132] Through probability modeling, the system can distinguish between high-confidence predictions and high-uncertainty predictions, providing more reliable support for engineering decisions. Especially in cases where data is sparse or the environment is complex, probability predictions are more valuable than deterministic predictions.

[0133] Example 7: Causal Inference and Weight Feedback Module

[0134] As Figure 5 shown, the causal inference and weight feedback module 4 includes a causal graph construction unit 41, a causal intervention analysis unit 42, a counterfactual reasoning unit 43, and a sensor influence degree evaluation unit 44.

[0135] The causal graph construction unit 41 is used to establish a directed graph representing the causal relationship between observed variables and latent variables. This unit first constructs an initial causal graph based on domain knowledge, including sensor observed variables (such as vibration intensity, tilt angle, etc.) and latent variables (such as groundwater pressure, soil stress, etc.). Then, it uses the structural equation model to quantify the causal relationship strength, forming a complete causal knowledge graph.

[0136] The causal intervention analysis unit 42 is used to calculate the effects under different intervention strategies. This unit implements do-calculus (intervention calculus) to evaluate the potential impact of specific intervention behaviors (such as lowering the groundwater level, strengthening the support structure, etc.) on the deformation index. Through causal intervention analysis, the system can provide a scientific basis for engineering decisions and predict the effects of different countermeasures.

[0137] The counterfactual reasoning unit 43 is used to evaluate the deformation development paths under different scenarios. Based on the existing observation data, this unit infers counterfactual scenarios of how the results would be if the conditions were different, helping to understand the deformation mechanism and evaluate potential risks. For example, the system can analyze the scenario of how the deformation would develop if the groundwater level rises by another 2 meters, providing a basis for preventive measures.

[0138] The sensor influence degree evaluation unit 44 is used to calculate the contribution degree of each sensor to anomaly detection. This unit calculates the influence degree of sensor s through the following formula:

[0139] Influence(s)=E[|P(A|D)-P(A|D{s})|],

[0140] where A represents the anomaly event, D represents all sensor data, D(s) represents the data set after removing sensor s, and E represents the expected value. This influence degree evaluation provides a scientific basis for subsequent weight adjustment.

[0141] The main innovation of this module lies in introducing causal reasoning technology into the field of deformation monitoring. It can not only detect anomalies, but also analyze the causes of deformation and predict the development trend, thus providing decision-making support for proactive prevention. The weight adjustment suggestions output by the module are fed back to the adaptive weight dynamic allocation module to form a closed-loop optimization mechanism, continuously improving the system performance.

[0142] Example 8: Sensor Influence Degree Evaluation Unit

[0143] Furthermore, as Figure 5 shown, the sensor influence degree evaluation unit 44 realizes the quantification of the sensor contribution degree based on causal reasoning, providing a scientific basis for weight adjustment.

[0144] This unit evaluates the importance of each sensor by calculating the influence degree of the sensor on the anomaly detection result. Specifically, for sensor s, its influence degree is calculated as follows:

[0145] Influence(s)=E[|P(A|D)-P(A|D{s})|],

[0146] where A represents the anomaly event, P(A|D) represents the probability of the anomaly event based on the complete data set D, P(A|D{s}) represents the probability of the anomaly event based on the remaining data after removing sensor s, and E represents the expectation operation.

[0147] In actual calculation, the influence degree can be approximated through the following steps:

[0148] 1. Run anomaly detection on the complete data to obtain the anomaly probability P(A|D);

[0149] 2. For each sensor s, mask its data (set it to the historical average or handle it as a missing value);

[0150] 3. Use the modified dataset to recalculate the anomaly probability P(A|D{s});

[0151] 4. Calculate the absolute value of the probability difference |P(A|D) - P(A|D{s})|;

[0152] 5. Repeat the above process over multiple time windows and take the average as the final influence degree;

[0153] Sensors with a high influence degree indicate that they make a significant contribution to the anomaly detection result and should obtain a higher weight in the weight assignment; while sensors with a low influence degree may provide redundant information or noise and their weights should be appropriately reduced.

[0154] Based on the sensor influence degree evaluation result, the system generates a weight adjustment recommendation δW and feeds it back to the adaptive weight dynamic allocation module. The adjustment recommendation is in an incremental form:

[0155] δW(s) = λ·Normalize(Influence(s)),

[0156] where λ is the learning rate that controls the adjustment amplitude, usually taking a value of 0.1 - 0.3; the Normalize operation normalizes the influence degree to ensure the balance of weight adjustment.

[0157] This influence degree evaluation mechanism realizes the transformation of the weight assignment paradigm from data-driven to task-driven, making the weight assignment more focused on the sensors that actually contribute to anomaly detection and improving the effectiveness of multimodal fusion.

[0158] Example 9: Distributed Federated Learning Module

[0159] As Figure 6 shown, the distributed federated learning module 5 includes a local training unit 51, a secure aggregation unit 52, a domain adaptation unit 53, and a knowledge distillation unit 54.

[0160] The local training unit 51 is used to train the local model on each engineering project. This unit performs local calculations at each project deployment site, updates the model parameters using local data, and does not need to transmit the original data to the central server, protecting data privacy. The local model update uses the standard gradient descent algorithm:

[0161]

[0162] where, represents the model parameters at site k in the t-th round, n is the learning rate, is the gradient of the local loss function.

[0163] The secure aggregation unit 52 is used to aggregate the model updates of each project while protecting privacy. This unit implements a secure aggregation protocol, enabling each project to share only the model parameters (instead of the original data), and introducing a differential privacy protection mechanism during the aggregation process to further enhance data security. The global model update is achieved through weighted averaging:

[0164]

[0165] where p k is the site weight, usually proportional to the site data volume.

[0166] The domain adaptation unit 53 is used to handle the distribution shift between different projects. Due to the differences in geological conditions, construction techniques, etc. among different engineering projects, directly applying the models of other projects may lead to performance degradation. The domain adaptation unit reduces the distribution difference between the source domain and the target domain through techniques such as minimizing the maximum mean discrepancy (MMD) and adversarial training, improving the applicability of the model to new projects.

[0167] The knowledge distillation unit 54 is used to transfer the knowledge of complex models to lightweight models for edge deployment. This unit implements the knowledge distillation of the teacher-student architecture, enabling high-performance models to be deployed on edge devices with limited computing resources. The distillation process uses temperature-controlled soft target training to enable the lightweight student model to learn the generalization ability of the complex teacher model.

[0168] The core innovation of this module lies in implementing a secure and efficient cross-project knowledge sharing mechanism, solving the problem that traditional systems cannot learn from historical engineering experiences. Through the federated learning framework, the system can continuously accumulate experience and knowledge from multiple engineering projects while protecting the data privacy of all parties, significantly improving the deployment efficiency and warning accuracy of new projects.

[0169] Example 10: Intelligent warning method for foundation pit deformation based on multi-modal fusion

[0170] As Figure 7 shown, the intelligent warning method for foundation pit deformation based on multi-modal fusion of the present invention includes the following steps:

[0171] Step 1: Collect multi-modal sensor data, evaluate the data quality of each sensor, generate the current weight matrix based on environmental factors and historical weights, and obtain weighted multi-modal data. Specifically, it includes: First, deploy various types of sensors such as vibration, tilt, pressure, water level, etc., and collect raw data at a set frequency; Then, preprocess the raw data, including denoising, outlier filtering, and calibration; Then, calculate the data quality scores of each sensor and evaluate the impact of environmental parameters on sensor performance; Finally, based on the quality scores, historical weights, and environmental impact coefficients, calculate the weight matrix at the current moment and generate weighted multi-modal data.

[0172] Step 2: Perform modality-specific feature extraction on the weighted multi-modal data, achieve cross-modal feature interaction through the multi-head attention mechanism, and integrate the spatial relationships of sensors by combining spatio-temporal graph convolutional networks to generate a fused feature representation. Specifically, it includes: First, input data of different modalities into the corresponding feature extraction sub-networks to extract modality-specific initial features; Then, use the multi-head attention mechanism to calculate the interaction relationships between different modality features and generate cross-modal features; Then, construct a graph structure based on the physical layout of the sensors and apply graph convolution operations to integrate spatial context information; Finally, integrate temporal features and spatial features to generate a comprehensive fused feature representation.

[0173] Step 3: Based on the fused feature representation, apply a mixture density network to predict the probability distribution of deformation indices, and identify abnormal patterns through time series decomposition and contrastive learning. Specifically, it includes: First, map the fused features to basic deformation indices, including settlement amount, horizontal displacement, etc.; Then, use a mixture density network to model the probability distribution of deformation indices to express the uncertainty of the prediction; Then, decompose the deformation time series to separate the trend term, seasonal term, and residual term; Finally, evaluate the difference between the current state and known patterns through contrastive learning methods to identify potential anomalies.

[0174] Step 4: Use a causal graph model to infer the possible causes of abnormal patterns, calculate the contribution of each sensor to anomaly detection, and generate weight adjustment suggestions. Specifically, it includes: First, based on a preset causal knowledge graph, analyze the possible causes of deformation anomalies; Then, through causal intervention and counterfactual reasoning, evaluate the deformation development trends under different scenarios; Then, calculate the contribution of each sensor to the anomaly detection results; Finally, generate weight adjustment suggestions according to the contributions and feedback them to the weight assignment link.

[0175] Step 5: Coordinate knowledge sharing among multiple engineering projects through distributed federated learning, use domain adaptation techniques to handle distribution shifts, optimize the model, and apply it to subsequent monitoring. Specifically, it includes: First, perform local model training on each engineering project; then, use the secure aggregation protocol to aggregate the model updates of each project; then, apply domain adaptation techniques to handle the distribution differences between different projects; finally, compress the complex model into a lightweight model through knowledge distillation and deploy it to edge devices.

[0176] Through multi-modal data fusion, adaptive weight allocation, deformation anomaly recognition, causal inference analysis, and cross-project knowledge transfer, this method realizes comprehensive and accurate monitoring and early warning of foundation pit deformation, providing reliable guarantee for the safety of foundation pit construction.

[0177] Example 11: Practical application case

[0178] To verify the practicality and effectiveness of the present invention, a typical engineering case is selected for test analysis. This project is a subway foundation pit project in the central area of a certain city, with a depth of 28 meters and a complex surrounding environment, including multiple high-rise buildings and important underground pipelines.

[0179] System deployment: As Figure 8 shown, 56 vibration sensors, 32 tilt sensors, 48 pressure sensors, and 24 water level sensors are arranged around the foundation pit to form a multi-modal monitoring network. The edge computing unit is installed in the on-site monitoring room, and the cloud analysis server is deployed in the data center.

[0180] System operation: After the system is put into use, the adaptive weight dynamic allocation module adjusts the weights of each sensor in real time according to environmental conditions and data quality. For example, during rainfall, the system automatically increases the weight of the water level sensor; during vibration construction such as piling, the weight of the vibration sensor is increased. The multi-modal feature cross-extraction module effectively captures the mutual relationship between different sensing data, such as the correlation pattern between the change in the underground water level and the subsequent change in soil pressure.

[0181] Early warning effect: As Figure 9 shown, during the 8-month monitoring period, the system issued a total of 27 effective early warnings, with the lead time ranging from 6 hours to 3 days, and the accuracy rate reaching 92%. Compared with the traditional single-modal monitoring system, the early warning time was advanced by an average of 16 hours, the deformation detection accuracy was improved from centimeter level to millimeter level, and the false alarm rate was reduced from 15% to less than 3%. Especially during a heavy rainfall event, the system warned 30 hours in advance of the possible slip risk in the northwest corner, giving the construction party sufficient time to reinforce the support structure and successfully avoiding a potential accident.

[0182] Analysis of deformation causes: The system can not only detect abnormal deformations, but also analyze the causes of deformation through causal reasoning. For example, in an abnormal event, the system identified the causal chain of rising groundwater level → increased soil saturation → increased lateral earth pressure → deformation of the support structure, providing a clear treatment direction for engineers, namely drainage and pressure reduction.

[0183] Effect of knowledge transfer: Through distributed federated learning, the system safely transfers the empirical knowledge of this project to another similar project, reducing the model adaptation time of the new project from the traditional 7 days to 1 day, significantly improving the deployment efficiency. At the same time, the model compression technology reduces the resource occupancy of edge devices by 65%, while the performance only drops by 5%, achieving efficient edge deployment.

[0184] Overall, the actual application results show that the system of the present invention significantly improves the accuracy, reliability and timeliness of foundation pit deformation monitoring, providing strong support for the safety management of foundation pit construction.

[0185] The intelligent early warning system and method for foundation pit deformation based on multi-modal fusion of the present invention have good industrial practicability and can be widely applied in various foundation pit projects, specifically reflected in:

[0186] 1. In terms of hardware compatibility, the system supports mainstream sensor models in the market, provides a standardized interface protocol, and the edge computing unit is based on the ARM Cortex-A series processor, meeting the requirements of low power consumption and high performance;

[0187] 2. In terms of deployment flexibility, the modular design supports phased deployment and upgrade, provides cloud, edge and hybrid deployment modes, and the software architecture supports Docker containerization, facilitating rapid deployment and migration;

[0188] 3. In terms of reliability guarantee, a data redundant storage and failure recovery mechanism is implemented, a local fault tolerance processing for network communication interruption is designed, and a watchdog mechanism is introduced to monitor the running status of key processes;

[0189] 4. In terms of real-time performance, the early warning decision delay is controlled within 100ms, meeting the emergency response requirements, using a streaming processing architecture to process high-frequency sampled data, optimizing the algorithm calculation complexity, and reducing resource occupancy;

[0190] 5. In terms of economy, incremental updates are adopted to reduce the data transmission volume, reduce the communication cost, implement a sensor on-demand wake-up mechanism, extend the battery life, provide a hierarchical storage strategy, and optimize the cloud storage cost.

[0191] The above features enable the system of the present invention to adapt to various engineering environments, providing a reliable, efficient and economical technical solution for the safety management of foundation pit construction.

[0192] The present invention provides an intelligent early warning system and method for foundation pit deformation based on multimodal fusion. Through innovative technologies such as adaptive weight dynamic allocation, multimodal feature cross-extraction, multi-granularity abnormal pattern recognition, causal reasoning analysis, and distributed federated learning, the problems of adaptability, accuracy, and interpretability of traditional foundation pit monitoring systems in complex environments are solved. The system integrates various sensor data such as vibration, tilt, pressure, and water level, realizes deformation detection and early warning with millimeter-level accuracy, and can analyze the causes of deformation and provide treatment suggestions, providing a comprehensive solution for the safety management of foundation pit construction.

[0193] In practical engineering applications, the present invention significantly improves the accuracy of deformation detection and the lead time of early warning, reduces the false alarm rate, and improves the engineering deployment efficiency, having good industrial practicability and promotion value.

[0194] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An intelligent early warning system for foundation pit deformation based on multimodal fusion, characterized in that Including: An adaptive weight dynamic allocation module, which is used to evaluate the quality of data from multi-modal sensors, generate a weight matrix representing the credibility of each sensor under the current environmental conditions, and output the weighted multi-modal data; A multi-modal feature cross-extraction module, connected to the adaptive weight dynamic allocation module, which is used to receive the weighted multi-modal data, extract initial features through modality-specific sub-networks, use the multi-head attention mechanism to achieve cross-modal feature interaction, and integrate spatial correlation through the spatio-temporal graph convolutional network to generate a fused feature representation; A multi-granularity anomaly pattern recognition module, connected to the multi-modal feature cross-extraction module, which is used to receive the fused feature representation, apply a mixture density network to predict the probability distribution of deformation indicators, and identify anomaly patterns by combining time series decomposition and contrast learning; A causal inference and weight feedback module, connected to the multi-granularity anomaly pattern recognition module, which is used to receive the anomaly pattern, infer the cause of deformation based on the causal graph model, calculate the sensor contribution, and feedback weight adjustment suggestions to the adaptive weight dynamic allocation module; A distributed federated learning module, connected to the multi-modal feature cross-extraction module, the multi-granularity anomaly pattern recognition module, and the causal inference and weight feedback module, which is used to coordinate knowledge sharing and model optimization among multiple projects while protecting the privacy of the original data.

2. The system according to claim 1, wherein The adaptive weight dynamic allocation module includes: A sensor quality evaluation unit, which is used to calculate a scoring matrix representing the quality of each sensor's data; An environmental impact evaluation unit, which is used to generate an environmental correction factor representing the degree of influence of the current environmental parameters on each sensor; A weight calculation unit, which is used to generate the weight matrix at the current moment based on the scoring matrix, the historical weight, and the environmental correction factor; Wherein, the weight matrix W(t) is calculated by the following formula: W(t) = Softnax[α·Q(s,t) + β·W(t - 1) + γ·C(t)], In the formula, Q(s,t) represents the data quality score of sensor s at time t, W(t - 1) represents the weight at the previous moment, C(t) represents the current environmental context vector, and α, β, γ are adjustable hyperparameters.

3. The system according to claim 1, wherein The multi-modal feature cross-extraction module includes: A modality-specific feature extraction unit, including a vibration feature extraction sub-network, an inclination feature extraction sub-network, a pressure feature extraction sub-network, and a water level feature extraction sub-network, which are respectively used to extract the initial features of the corresponding modality; A cross-modal attention unit, which is used to calculate the interaction relationship between different modality features and generate a cross-feature representation; A spatio-temporal graph convolution unit, which is used to construct a graph structure based on the physical layout of the sensors and integrate spatial correlation and temporal dependence through graph convolution operations.

4. The system according to claim 3, wherein The cross-modal attention unit realizes feature interaction in the following way: F ij = MultiHeadAttention(Q i , K j , V j ) where \(i\) and \(j\) represent different modalities, is a learnable projection matrix.

5. The system according to claim 1, characterized in that, The multi-granularity anomaly pattern recognition module includes: A deformation index prediction unit, which is used to map the fused features to basic displacement indicators; A mixture density network unit, which is used to model the multi-modal probability distribution for each deformation indicator; A time series decomposition unit, which is used to decompose the deformation sequence into a trend term, a seasonal term, and a residual term; A contrastive learning detection unit for evaluating the differences between the current state and known normal and abnormal patterns.

6. The system according to claim 5, wherein The mixture density network unit models the conditional probability distribution of the deformation index y through the following formula: where, π k is the weight of the k-th mixture component, N represents a Gaussian distribution, μ k and σ k are the mean and standard deviation of the k-th component, respectively.

7. The system according to claim 1, characterized in that The causal inference and weight feedback module includes: A causal graph construction unit for establishing a directed graph representing the causal relationship between observed variables and latent variables; A causal intervention analysis unit for calculating the effects under different intervention strategies; A counterfactual reasoning unit for evaluating the deformation development paths under different scenarios; A sensor influence degree evaluation unit for calculating the contribution degree of each sensor to anomaly detection.

8. The system according to claim 7, wherein The sensor influence degree evaluation unit calculates the influence degree of sensor s through the following formula: Influence(s) = E[P(A|D) - P(A|D{s})|], where A represents an abnormal event, D represents all sensor data, D{s} represents the data set after removing sensor s, and E represents the expected value.

9. The system according to claim 1, wherein The distributed federated learning module includes: A local training unit for training local models on each engineering project; A secure aggregation unit for aggregating the model updates of each project while protecting privacy; A domain adaptation unit for handling distribution shifts between different projects; A knowledge distillation unit for transferring complex model knowledge to lightweight models for edge deployment.

10. An intelligent early warning method for foundation pit deformation based on multimodal fusion, using the system according to any one of claims 1-9, characterized in that, The method includes: Collecting multimodal sensor data, evaluating the quality of each sensor data, generating the current weight matrix based on environmental factors and historical weights, and obtaining weighted multimodal data; Performing modality-specific feature extraction on the weighted multimodal data, realizing cross-modal feature interaction through a multi-head attention mechanism, and integrating the spatial relationships of sensors by combining with a spatio-temporal graph convolutional network to generate a fused feature representation; Based on the fused feature representation, applying a mixture density network to predict the probability distribution of the deformation index, and identifying abnormal patterns through time series decomposition and contrastive learning; Using a causal graph model to infer the possible causes of the abnormal patterns, calculating the contribution degree of each sensor to anomaly detection, and generating weight adjustment suggestions; Coordinating knowledge sharing among multiple engineering projects through distributed federated learning, using domain adaptation techniques to handle distribution shifts, optimizing the model, and applying it to subsequent monitoring.

Images