Intelligent deformation early warning system and method for foundation pit based on multi-modal fusion

Abstract

The present application relates to the technical field of engineering safety monitoring, in particular to a foundation pit deformation intelligent early warning system and method based on multi-modal fusion, the system carries out quality evaluation and weighted processing on multi-modal sensor data through a self-adaptive weight dynamic distribution module to ensure data reliability; a multi-modal feature cross extraction module extracts and interacts features using a specific sub-network and a multi-head attention mechanism, integrates information through a space-time graph convolution network, and generates accurate fusion feature representation; a multi-granularity abnormal pattern recognition module combines a hybrid density network and time series analysis to accurately identify deformation abnormalities; a causal reasoning and weight feedback module analyzes deformation causes through a causal graph model to provide feedback for sensor weight adjustment; the system significantly improves the accuracy and reliability of deformation detection, improves detection accuracy to the millimeter level, and increases accuracy by 40%, providing strong technical support for engineering safety monitoring.

Description

Technical Field

[0001] The present invention relates to the field of engineering safety monitoring technology, and specifically to an intelligent early warning system for foundation pit deformation based on multimodal fusion and a method thereof, which is suitable for deformation monitoring and early warning during foundation pit construction in the fields of construction engineering, municipal engineering, transportation engineering, etc. Background Art

[0002] With the in-depth development of urban construction, the number and scale of deep foundation pit projects are increasing, and the construction conditions are becoming increasingly complex. The safety of foundation pit projects is closely related to the safety of surrounding buildings, underground pipelines, and construction workers. Deformation monitoring and early warning have become a key link in ensuring the safety of foundation pit construction.

[0003] Existing technologies, such as Chinese patent CN116953696A, disclose a foundation pit deformation detection method and system. This technology monitors the environment around the foundation pit through electromagnetic wave scanning, establishes echo points based on the scanning echoes, determines the area requiring a second scan by screening the distance between the echo points, constructs a feature model based on the echo point group, and issues a warning signal by comparing the changes in the feature model over a time series.

[0004] However, the above technology has the following shortcomings: First, it only relies on a single electromagnetic wave scanning mode, lacks multi-source data verification, and is easily affected by environmental interference and misjudgment; second, it uses fixed parameter thresholds and warning ranges, which cannot adapt to the characteristic changes of different geological conditions and construction stages; third, the feature extraction method is simple, relying only on basic distance calculations and plane construction, and lacks deep feature mining; in addition, it cannot distinguish between deformation caused by normal construction and abnormal deformation, and the warning mechanism is simple, only issuing warnings based on the change trend out of range, and cannot provide deformation cause analysis and treatment suggestions; finally, each monitoring terminal performs independent analysis, lacks global correlation analysis, and the system cannot learn and improve from historical data, lacking intelligence and adaptive capabilities. Summary of the Invention

[0005] The purpose of the present invention is to provide an intelligent early warning system for foundation pit deformation based on multimodal fusion and its method, aiming to solve the problems existing in the prior art, such as single modal perception limitation, fixed parameter threshold, simple feature extraction, lack of construction context recognition, single-point data analysis, simple early warning mechanism and no learning ability.

[0006] The present invention proposes an intelligent early warning system for foundation pit deformation based on multimodal fusion, comprising:

[0007] An adaptive weight dynamic allocation module is used to evaluate the quality of data from multimodal sensors, generate a weight matrix representing the credibility of each sensor under current environmental conditions, and output the weighted multimodal data;

[0008] A multimodal feature cross-extraction module, connected to the adaptive weight dynamic allocation module, is used to receive the weighted multimodal data, extract initial features through a modality-specific subnetwork, implement cross-modal feature interaction using a multi-head attention mechanism, and integrate spatial correlations through a spatiotemporal graph convolutional network to generate a fused feature representation;

[0009] A multi-granularity abnormal pattern recognition module, connected to the multimodal feature cross extraction module, is used to receive the fused feature representation, apply a mixed density network to predict the probability distribution of deformation indicators, and combine time series decomposition and contrastive learning to identify abnormal patterns;

[0010] a causal reasoning and weight feedback module, connected to the multi-granularity abnormal pattern recognition module, configured to receive the abnormal pattern, infer the deformation cause based on the causal graph model, calculate the sensor contribution, and feed back weight adjustment suggestions to the adaptive weight dynamic allocation module;

[0011] A distributed federated learning module is connected to the multimodal feature cross-extraction module, the multi-granularity abnormal pattern recognition module, and the causal reasoning and weight feedback module, and is used to coordinate knowledge sharing and model optimization among multiple projects while protecting the privacy of original data.

[0012] Preferably, the adaptive weight dynamic allocation module includes:

[0013] A sensor quality assessment unit, used to calculate a scoring matrix representing the quality of each sensor data;

[0014] An environmental impact assessment unit, used to generate an environmental correction factor indicating the degree of influence of current environmental parameters on each sensor;

[0015] A weight calculation unit, configured to generate a weight matrix at a current moment based on the scoring matrix, the historical weights, and the environmental correction factors;

[0016] The weight matrix W(t) is calculated by the following formula:

[0017] W(t)=Softmax[α·Q(s,t)+β·W(t-1)+γ·C(t)],

[0018] Where Q(s,t) represents the data quality score of sensor s at time t, W(t-1) represents the weight at the previous moment, C(t) represents the current environment context vector, and α, β, and γ are adjustable hyperparameters.

[0019] Preferably, the multimodal feature cross extraction module includes:

[0020] The modal-specific feature extraction unit includes a vibration feature extraction subnetwork, a tilt feature extraction subnetwork, a pressure feature extraction subnetwork, and a water level feature extraction subnetwork, each of which is used to extract the initial features of the corresponding modal;

[0021] Cross-modal attention unit, used to calculate the interaction between different modal features and generate cross-feature representations;

[0022] The spatiotemporal graph convolution unit is used to build a graph structure based on the physical layout of sensors and integrate spatial correlation and temporal dependency through graph convolution operations.

[0023] Preferably, the cross-modal attention unit realizes feature interaction in the following manner:

[0024] F ij =MultiHeadAttention(Q i ,K j ,V j ),

[0025]

[0026] Where i and j represent different modes, is the learnable projection matrix.

[0027] Preferably, the multi-granularity abnormal pattern recognition module includes:

[0028] The deformation index prediction unit is used to map the fusion features into basic displacement indicators;

[0029] Mixture density network units, used to model multimodal probability distributions for each deformation metric;

[0030] Time series decomposition unit, used to decompose the deformation series into trend term, seasonal term and residual term;

[0031] Contrastive learning detection unit that evaluates how the current state differs from known normal and abnormal patterns.

[0032] Preferably, the mixed density network unit models the conditional probability distribution of the deformation index y by the following formula:

[0033]

[0034] Among them, π k is the weight of the kth mixture component, N represents the Gaussian distribution, μ k and σ k are the mean and standard deviation of the kth component respectively.

[0035] Preferably, the causal reasoning and weight feedback module includes:

[0036] A causal graph construction unit is used to build a directed graph representing the causal relationship between observed variables and latent variables;

[0037] Causal intervention analysis unit, used to calculate the effects of different intervention strategies;

[0038] Counterfactual reasoning unit, used to evaluate the deformation development path under different scenarios;

[0039] The sensor influence evaluation unit is used to calculate the contribution of each sensor to anomaly detection.

[0040] Preferably, the sensor influence evaluation unit calculates the influence of sensor s by the following formula:

[0041] Influence(s)=E[|P(A|D)-P(A|D{s})|],

[0042] Where A represents an abnormal event, D represents all sensor data, D{s} represents the data set after removing sensor s, and E represents the expected value.

[0043] Preferably, the distributed federated learning module includes:

[0044] Local training unit, used to train local models on various engineering projects;

[0045] A secure aggregation unit, used to aggregate model updates from various projects while protecting privacy;

[0046] Domain adaptation unit, used to deal with distribution shifts between different projects;

[0047] Knowledge distillation unit, used to transfer complex model knowledge to lightweight models for easy edge deployment.

[0048] An intelligent early warning method for foundation pit deformation based on multimodal fusion, the method comprising:

[0049] Collect multimodal sensor data, evaluate the quality of each sensor data, generate the current weight matrix based on environmental factors and historical weights, and obtain weighted multimodal data;

[0050] Extract modality-specific features from the weighted multimodal data, implement cross-modal feature interaction through a multi-head attention mechanism, integrate sensor spatial relationships with a spatiotemporal graph convolutional network, and generate a fused feature representation;

[0051] Based on the fused feature representation, a mixture density network is applied to predict the probability distribution of deformation indicators, and abnormal patterns are identified through time series decomposition and contrastive learning;

[0052] Using a causal graph model to infer the possible causes of the abnormal pattern, calculate the contribution of each sensor to anomaly detection, and generate weight adjustment suggestions;

[0053] Distributed federated learning is used to coordinate knowledge sharing among multiple engineering projects, domain adaptation technology is used to deal with distribution shifts, and model optimization is performed and applied to subsequent monitoring.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. The accuracy and reliability of deformation detection are improved through multimodal data fusion, increasing detection precision from centimeters to millimeters, and increasing accuracy by 40%;

[0056] 2. An innovative adaptive weight allocation mechanism dynamically adjusts the contribution of each sensor based on environmental conditions and data quality, significantly improving the system's adaptability to complex environments.

[0057] 3. The multimodal feature cross-extraction network effectively captures the complementary relationships between different types of sensor data, enhancing the system's ability to represent complex deformation patterns.

[0058] 4. Multi-granularity anomaly pattern recognition technology combined with causal reasoning capabilities can not only detect anomalies but also explain the causes of deformation, providing a scientific basis for engineering decision-making;

[0059] 5. The distributed knowledge transfer mechanism based on federated learning enables the system to learn from historical engineering experience, shortening the adaptation time for new projects from 7 days to 1 day, significantly improving deployment efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a schematic diagram of the overall system architecture of the present invention;

[0061] Figure 2 It is a structural diagram of the adaptive weight dynamic allocation module of the present invention;

[0062] Figure 3 Schematic diagram of the structure of the multimodal feature cross extraction module of the present invention;

[0063] Figure 4 It is a structural diagram of the multi-granularity abnormal pattern recognition module of the present invention;

[0064] Figure 5 It is a schematic diagram of the structure of the causal reasoning and weight feedback module of the present invention;

[0065] Figure 6 Schematic diagram of the distributed federated learning module of the present invention;

[0066] Figure 7This is a flow chart of the foundation pit deformation intelligent early warning method of the present invention;

[0067] Figure 8 It is a schematic diagram of the deployment of the present invention in actual engineering applications;

[0068] Figure 9 This is a comparison chart of the early warning effects of the system of the present invention;

[0069] Figure 10 It is a schematic diagram of data flow in the system of the present invention. DETAILED DESCRIPTION

[0070] Please refer to the attached Figure 1-10 The present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0071] Example 1: Overall system architecture

[0072] like Figure 1 As shown, the foundation pit deformation intelligent early warning system based on multimodal fusion provided by the present invention includes an adaptive weight dynamic allocation module 1, a multimodal feature cross-extraction module 2, a multi-granularity abnormal pattern recognition module 3, a causal reasoning and weight feedback module 4 and a distributed federated learning module 5.

[0073] The adaptive weight dynamic allocation module 1 is used to evaluate the quality of data from multimodal sensors, generate a weight matrix representing the credibility of each sensor under current environmental conditions, and output the weighted multimodal data. Multimodal sensors include but are not limited to vibration sensors, tilt sensors, pressure sensors, and groundwater level sensors.

[0074] The multimodal feature cross-extraction module 2 is connected to the adaptive weight dynamic allocation module 1 to receive weighted multimodal data, extract initial features through the modality-specific sub-network, realize cross-modal feature interaction using the multi-head attention mechanism, and integrate spatial correlation through the spatiotemporal graph convolutional network to generate a fused feature representation.

[0075] The multi-granularity abnormal pattern recognition module 3 is connected to the multimodal feature cross-extraction module 2, and is used to receive the fused feature representation, apply the mixed density network to predict the probability distribution of deformation indicators, and combine time series decomposition and contrastive learning to identify abnormal patterns.

[0076] The causal reasoning and weight feedback module 4 is connected to the multi-granularity abnormal pattern recognition module 3, and is used to receive abnormal patterns, infer the deformation cause based on the causal graph model, calculate the sensor contribution, and feedback weight adjustment suggestions to the adaptive weight dynamic allocation module 1.

[0077] The distributed federated learning module 5 is connected with the multimodal feature cross-extraction module 2, the multi-granularity abnormal pattern recognition module 3 and the causal reasoning and weight feedback module 4 to coordinate knowledge sharing and model optimization among multiple projects while protecting the privacy of the original data.

[0078] The data flow between each module is as follows: First, the adaptive weight dynamic allocation module 1 receives the raw data collected by the multimodal sensor, and after preprocessing and weight allocation, transmits the weighted data to the multimodal feature cross-extraction module 2; then, the multimodal feature cross-extraction module 2 extracts the fusion features and transmits them to the multi-granularity abnormal pattern recognition module 3; then, the multi-granularity abnormal pattern recognition module 3 outputs the abnormal pattern and transmits it to the causal reasoning and weight feedback module 4; then, the causal reasoning and weight feedback module 4 generates weight adjustment suggestions, feeds them back to the adaptive weight dynamic allocation module 1, and outputs deformation explanations and processing suggestions at the same time; finally, the distributed federated learning module 5 regularly collects model updates from each module, performs model optimization, and distributes the updated model parameters back to each functional module to form a complete closed-loop data flow.

[0079] Example 2: Adaptive Weight Dynamic Allocation Module

[0080] like Figure 2 As shown, the adaptive weight dynamic allocation module 1 includes a sensor quality assessment unit 11 , an environmental impact assessment unit 12 and a weight calculation unit 13 .

[0081] The sensor quality assessment unit 11 is used to calculate a scoring matrix representing the quality of each sensor data. Specifically, the unit 11 analyzes the signal-to-noise ratio, integrity, and historical consistency of the sensor data to generate a quality score Q(s,t), where s represents the sensor identifier and t represents the current time. The quality score calculation considers the following factors: signal stability, data integrity, historical accuracy, and consistency with adjacent sensor data.

[0082] The environmental impact assessment unit 12 is used to generate an environmental correction factor that represents the degree to which the current environmental parameters affect each sensor. Environmental parameters E include external factors such as temperature, humidity, and vibration intensity that may affect sensor performance. For each sensor type, an environmental correction factor is calculated based on its sensitivity to different environmental factors and used to adjust the weight calculation process.

[0083] The weight calculation unit 13 is used to generate the weight matrix at the current moment based on the scoring matrix, historical weights and environmental correction factors. The weight matrix W(t) is calculated by the following formula:

[0084] W(t)=Softmax[α·Q(s,t)+β·W(t-1)+γ·C(t)],

[0085] Where Q(s,t) represents the data quality score of sensor s at time t, W(t-1) represents the weight at the previous time, C(t) represents the current environmental context vector, and α, β, and γ are adjustable hyperparameters that control the degree to which each factor influences the weight update. Preferably, α ranges from 0.4 to 0.6, β ranges from 0.2 to 0.4, and γ ranges from 0.1 to 0.3. The softmax function ensures that the sum of all weights is 1, making the weights of each sensor relative.

[0086] In practice, this module dynamically adjusts weight distribution based on time and environmental changes, enabling the system to adaptively handle situations such as sensor failure and environmental interference. For example, if a sensor experiences severe interference, causing data quality to degrade, the system automatically reduces its weight and increases the weight of other normally functioning sensors, ensuring that overall system performance is not affected by single point failures.

[0087] Example 3: Multimodal feature cross-extraction module

[0088] like Figure 3 As shown, the multimodal feature cross-extraction module 2 includes a modality-specific feature extraction unit 21, a cross-modal attention unit 22 and a spatiotemporal graph convolution unit 23.

[0089] The modal-specific feature extraction unit 21 includes a vibration feature extraction subnetwork 211 , a tilt feature extraction subnetwork 212 , a pressure feature extraction subnetwork 213 and a water level feature extraction subnetwork 214 , which are respectively used to extract initial features of corresponding modes.

[0090] The vibration feature extraction subnetwork 211 uses wavelet transforms and multi-scale convolutional networks to extract frequency and time domain features from vibration signals, effectively identifying different types of geological vibration patterns. The tilt feature extraction subnetwork 212 primarily processes data from the tilt sensor, capturing tilt change rate and acceleration features through a gradient feature extractor. The pressure feature extraction subnetwork 213 uses a residual network structure to process data such as soil pressure and water pressure, extracting pressure distribution and change trend characteristics. The water level feature extraction subnetwork 214 uses a long short-term memory network to extract long-term trends and short-term fluctuation characteristics from groundwater level data.

[0091] The cross-modal attention unit 22 is used to calculate the interaction between features of different modalities and generate cross-feature representations. This unit realizes the information interaction between modalities and is calculated in the following way:

[0092] F ij =MultiHeadAttention(Qi ,K j ,V j ),

[0093]

[0094] Where i and j represent different modes, is a learnable projection matrix. The multi-head attention mechanism allows the model to focus on the various relationships between features of different modalities, enhancing the expressive power of feature fusion.

[0095] The spatiotemporal graph convolution unit 23 is used to construct a graph structure based on the physical layout of sensors, integrating spatial correlation and temporal dependencies through graph convolution operations. In this unit, each sensor is represented as a node v in the graph G. The edge connection weight e(v,u) is calculated based on physical distance and correlation. Graph convolution operations are applied to extract spatially relevant features.

[0096] The main advantage of this module is its ability to capture the complementarity and correlation between different sensor data, enhancing the system's ability to represent complex deformation patterns through cross-modal feature interaction. Furthermore, the spatiotemporal graph convolutional architecture effectively integrates the spatial distribution information of sensors, enabling the system to understand the spatial propagation characteristics of deformation.

[0097] Example 4: Cross-modal Attention Unit of Multimodal Feature Cross-Extraction Module

[0098] Furthermore, if Figure 3 As shown, the cross-modal attention unit 22 realizes the interaction between different modal features through a multi-head attention mechanism, thereby improving the system's ability to represent complex deformation patterns.

[0099] The multi-head attention mechanism allows the model to learn information from different representation subspaces, enhancing the diversity of feature interactions. In the cross-modal attention unit, the features of each modality are first transformed into the query, key, and value spaces through linear projection:

[0100]

[0101] Among them, Feature i and Feature j denote the eigenvectors of mode i and mode j respectively, is the corresponding projection matrix.

[0102] Then, calculate the attention weights and obtain the weighted features:

[0103] F ij =MultiHeadAttention(Q i ,Kj ,V j ),

[0104] The multi-head attention mechanism is defined as:

[0105] MultiHeadAttention(Q,K,V)=Concat(head1,head2,...,head h )W O ,

[0106] Each head is calculated as follows:

[0107]

[0108] In practical applications, the preferred number of heads h is 4 or 8, and the dimension d k Determined based on the input feature dimensions. The attention mechanism forms weighted connections between different modalities, allowing the system to learn the interplay between modalities like vibration-tilt and pressure-water level, significantly improving the system's ability to understand complex deformations.

[0109] In this way, the system can automatically learn important relationships between different modes, such as the relationship between groundwater level changes and soil pressure changes, or the temporal relationship between vibration events and subsequent tilt changes, thereby providing more comprehensive deformation situation awareness.

[0110] Example 5: Multi-granularity abnormal pattern recognition module

[0111] like Figure 4 As shown, the multi-granularity abnormal pattern recognition module 3 includes a deformation index prediction unit 31, a mixed density network unit 32, a time series decomposition unit 33 and a contrastive learning detection unit 34.

[0112] The deformation index prediction unit 31 is used to map the fused features into basic displacement indicators. This unit receives the fused feature representation output by the multimodal feature cross-extraction module and maps it into a series of basic deformation indicators through a fully connected layer, including key monitoring parameters such as settlement, horizontal displacement, tilt angle, and deformation rate.

[0113] The mixture density network unit 32 is used to model a multimodal probability distribution for each deformation indicator. Traditional methods usually only predict a single value, but this unit can estimate the complete probability distribution, more accurately expressing the uncertainty of the prediction. The mixture density network models the conditional probability distribution of the deformation indicator y using the following formula:

[0114]

[0115] Among them, π k is the weight of the kth mixture component, N represents the Gaussian distribution, μk and σ k are the mean and standard deviation of the kth component, respectively. In this way, the system can represent the multimodal distribution of deformation indicators, taking into account the various possibilities under different deformation modes.

[0116] The time series decomposition unit 33 is used to decompose the deformation sequence into trend terms, seasonal terms and residual terms. This unit uses time series decomposition technology to decompose the deformation time series X(t) into:

[0117] Trend term T(t): represents the long-term trend of change;

[0118] Seasonal term S(t): represents the periodic change pattern;

[0119] Residual term R(t): represents random fluctuations and potential anomalies;

[0120] By separating these components, the system can distinguish deformation trends caused by normal construction from abnormal fluctuations, improving the accuracy of early warnings.

[0121] The contrastive learning detection unit 34 evaluates the difference between the current state and known normal and abnormal patterns. This unit compares the current observation data with the normal and abnormal patterns in the historical database, calculates a similarity score, and identifies potential anomalies based on the similarity. Contrastive learning enhances the system's ability to generalize to new anomalies by maximizing the similarity between normal pattern representations while minimizing the similarity with abnormal patterns.

[0122] The innovation of this module lies in the combination of probability modeling and time series analysis technology, which realizes the transition from single threshold judgment to multi-dimensional and multi-granular anomaly detection. It can not only identify obvious sudden anomalies, but also discover slowly evolving potential risks, greatly improving the lead time and accuracy of early warning.

[0123] Example 6: Mixed Density Network Unit

[0124] Furthermore, if Figure 4 As shown, the mixture density network unit 32 provides a complete uncertainty representation of deformation prediction by modeling the conditional probability distribution of deformation indicators.

[0125] Traditional methods typically only output a single prediction value and fail to express the uncertainty and multimodality of the prediction. However, the mixture density network, by combining multiple Gaussian distributions, can represent complex conditional probability distributions, including multimodal and skewed distributions. The network models the conditional probability distribution of the deformation index y using the following formula:

[0126]

[0127] Among them, π k(x) is the weight of the k-th mixture component, And π k (x)≥0; N represents Gaussian distribution μ k (x) and σ k (x) are the mean and standard deviation of the kth component respectively; K is the number of mixture components, usually 3-5. All parameters π k (x), μ k (x) and σ k (x) are all generated by the neural network based on the input feature x.

[0128] The network training uses the negative log-likelihood loss function:

[0129]

[0130] In practice, the unit generates a complete prediction distribution, outputting not only the most likely deformation value but also the prediction range and the probability of different scenarios. For example, for foundation pit settlement prediction, the system might give a 70% probability of a 1-2mm settlement, a 20% probability of a 2-3mm settlement, and a 10% probability of a 3-5mm settlement. This provides a more comprehensive risk assessment than a single prediction of 2mm.

[0131] Through probabilistic modeling, the system can distinguish between high-confidence predictions and high-uncertainty predictions, providing more reliable support for engineering decisions. Especially in cases where data is sparse or the environment is complex, probabilistic predictions are more valuable than deterministic predictions.

[0132] Example 7: Causal Reasoning and Weight Feedback Module

[0133] like Figure 5 As shown, the causal reasoning and weight feedback module 4 includes a causal graph construction unit 41, a causal intervention analysis unit 42, a counterfactual reasoning unit 43 and a sensor influence evaluation unit 44.

[0134] The causal graph construction unit 41 is used to build a directed graph representing the causal relationships between observed variables and latent variables. This unit first constructs an initial causal graph based on domain knowledge, including sensor observed variables (such as vibration intensity and tilt angle) and latent variables (such as groundwater pressure and soil stress). Then, a structural equation model is used to quantify the strength of the causal relationships, forming a complete causal knowledge graph.

[0135] The causal intervention analysis unit 42 calculates the effects of different intervention strategies. This unit implements a do-cal culus to assess the potential impact of specific interventions (such as lowering the groundwater level or strengthening the support structure) on deformation indicators. Through causal intervention analysis, the system can provide a scientific basis for engineering decision-making and predict the effectiveness of different response measures.

[0136] The counterfactual reasoning unit 43 is used to assess deformation development paths under different scenarios. Based on existing observational data, this unit infers counterfactual scenarios of what would have happened under different conditions, helping to understand deformation mechanisms and assess potential risks. For example, the system can analyze how deformation would develop if the groundwater level rose another 2 meters, providing a basis for preventive measures.

[0137] The sensor influence evaluation unit 44 is used to calculate the contribution of each sensor to anomaly detection. This unit calculates the influence of sensor s using the following formula:

[0138] Influence(s)=E[|P(A|D)-P(A|D{s})|],

[0139] Where A represents an abnormal event, D represents all sensor data, D(s) represents the dataset after sensor s is removed, and E represents the expected value. This impact assessment provides a scientific basis for subsequent weight adjustments.

[0140] This module's key innovation lies in its introduction of causal reasoning technology into deformation monitoring. This technology not only detects anomalies but also analyzes deformation causes and predicts future trends, providing proactive preventive decision support. The module's output weight adjustment recommendations are fed back to the adaptive weight dynamic allocation module, forming a closed-loop optimization mechanism that continuously improves system performance.

[0141] Example 8: Sensor Impact Assessment Unit

[0142] Furthermore, if Figure 5 As shown, the sensor influence evaluation unit 44 implements the quantification of sensor contribution based on causal reasoning, providing a scientific basis for weight adjustment.

[0143] This unit evaluates the importance of each sensor by calculating its influence on the anomaly detection results. Specifically, for sensor s, its influence is calculated as follows:

[0144] Influence(s)=E[|P(A|D)-P(A|D{s})|],

[0145] Where A represents an abnormal event, P(A|D) represents the probability of an abnormal event based on the complete dataset D, P(A|D{s}) represents the probability of an abnormal event based on the remaining data after removing sensor s, and E represents the desired action.

[0146] In actual calculation, the influence can be approximated by the following steps:

[0147] 1. Run anomaly detection on the complete data to obtain the anomaly probability P(A|D);

[0148] 2. For each sensor s, mask its data (set it to the historical average or handle missing values);

[0149] 3. Recalculate the anomaly probability P(A|D{s}) using the modified dataset;

[0150] 4. Calculate the absolute value of the probability difference |P(A|D)-P(A|D{s})|;

[0151] 5. Repeat the above process over multiple time windows and take the average value as the final influence;

[0152] Sensors with high influence indicate that they contribute significantly to the anomaly detection results and should be given higher weights in the weight distribution; while sensors with low influence may provide redundant information or noise and their weights should be appropriately reduced.

[0153] Based on the sensor impact assessment results, the system generates weight adjustment suggestions δW and feeds them back to the adaptive weight dynamic allocation module. The adjustment suggestions are in incremental form:

[0154] δW(s)=λ·Normalize(Influence(s)),

[0155] Where λ is the learning rate, which controls the adjustment amplitude and is usually set to 0.1 to 0.3. The Normalize operation normalizes the influence to ensure the balance of weight adjustment.

[0156] This impact assessment mechanism realizes a paradigm shift in weight allocation from data-driven to task-driven, making the weight allocation more focused on sensors that make actual contributions to anomaly detection and improving the effectiveness of multimodal fusion.

[0157] Example 9: Distributed Federated Learning Module

[0158] like Figure 6 As shown, the distributed federated learning module 5 includes a local training unit 51, a security aggregation unit 52, a domain adaptation unit 53 and a knowledge distillation unit 54.

[0159] The local training unit 51 is used to train local models on each project. This unit performs local calculations at each project deployment site and uses local data to update model parameters. There is no need to transmit the original data to the central server, thus protecting data privacy. The local model update uses the standard gradient descent algorithm:

[0160]

[0161] in, represents the model parameters of site k in round t, n is the learning rate, is the gradient of the local loss function.

[0162] The secure aggregation unit 52 is used to aggregate model updates for each project while protecting privacy. This unit implements a secure aggregation protocol, allowing each project to only share model parameters (rather than raw data), and introduces a differential privacy protection mechanism during the aggregation process to further enhance data security. Global model updates are achieved through weighted averaging:

[0163]

[0164] where p k is the site weight, which is usually proportional to the amount of site data.

[0165] The domain adaptation unit 53 addresses distribution shifts between projects. Due to differences in geological conditions, construction techniques, and other factors, directly applying models from other projects can lead to performance degradation. The domain adaptation unit uses techniques such as minimizing the maximum mean difference (MMD) and adversarial training to reduce the distribution differences between the source and target domains, improving the model's applicability to new projects.

[0166] The knowledge distillation unit 54 is used to transfer complex model knowledge to a lightweight model for easy edge deployment. This unit implements knowledge distillation in the teacher-student architecture, enabling the deployment of high-performance models even on edge devices with limited computing resources. The distillation process uses temperature-controlled soft target training to enable the lightweight student model to learn the generalization capabilities of the complex teacher model.

[0167] The module's core innovation lies in its secure and efficient cross-project knowledge sharing mechanism, addressing the inability of traditional systems to learn from historical engineering experience. Through a federated learning framework, the system can continuously accumulate experience and knowledge from multiple engineering projects while protecting the data privacy of all parties, significantly improving the efficiency of new project deployment and the accuracy of early warnings.

[0168] Example 10: Intelligent early warning method for foundation pit deformation based on multimodal fusion

[0169] like Figure 7 As shown, the intelligent early warning method for foundation pit deformation based on multimodal fusion of the present invention includes the following steps:

[0170] Step 1: Collect multimodal sensor data, evaluate the quality of each sensor's data, and generate a current weight matrix based on environmental factors and historical weights to obtain weighted multimodal data. Specifically, this involves first deploying various types of sensors, such as vibration, tilt, pressure, and water level, to collect raw data at a set frequency. Next, preprocess the raw data, including denoising, outlier filtering, and calibration. Then, calculate the data quality score for each sensor and assess the impact of environmental parameters on sensor performance. Finally, based on the quality score, historical weights, and environmental impact coefficients, calculate the current weight matrix to generate weighted multimodal data.

[0171] Step 2: Extract modality-specific features from weighted multimodal data, implement cross-modal feature interaction through a multi-head attention mechanism, and integrate sensor spatial relationships using a spatiotemporal graph convolutional network to generate a fused feature representation. Specifically, the different modal data are fed into the corresponding feature extraction subnetwork to extract modality-specific initial features. Next, the multi-head attention mechanism is used to calculate the interaction between the different modal features to generate cross-modal features. A graph structure is then constructed based on the physical layout of the sensors, and graph convolution operations are applied to integrate spatial context information. Finally, temporal and spatial features are integrated to generate a comprehensive fused feature representation.

[0172] Step 3: Based on the fused feature representation, a mixture density network is applied to predict the probability distribution of deformation indicators. Anomalous patterns are identified through time series decomposition and contrastive learning. Specifically, the fused features are first mapped into basic deformation indicators, including settlement and horizontal displacement. Next, a mixture density network is used to model the probability distribution of deformation indicators to express the uncertainty of the prediction. The deformation time series is then decomposed to separate the trend term, seasonal term, and residual term. Finally, contrastive learning is used to evaluate the difference between the current state and the known pattern to identify potential anomalies.

[0173] Step 4: Use the causal graph model to infer the possible causes of abnormal patterns, calculate the contribution of each sensor to anomaly detection, and generate weight adjustment recommendations. Specifically, the process involves: first, analyzing the possible causes of deformation anomalies based on a pre-set causal knowledge graph; then, evaluating deformation trends under different scenarios through causal intervention and counterfactual reasoning; then, calculating the contribution of each sensor to the anomaly detection results; and finally, generating weight adjustment recommendations based on the contribution and feeding them back to the weight allocation process.

[0174] Step 5: Distributed federated learning is used to coordinate knowledge sharing across multiple engineering projects. Domain adaptation techniques are used to address distribution shifts, optimize models, and apply them to subsequent monitoring. Specifically, this involves: first, local model training is performed on each engineering project; then, model updates from each project are aggregated using a secure aggregation protocol; then, domain adaptation techniques are applied to address distribution differences between different projects; and finally, knowledge distillation is used to compress complex models into lightweight models for deployment on edge devices.

[0175] This method achieves comprehensive and accurate monitoring and early warning of foundation pit deformation through multimodal data fusion, adaptive weight allocation, deformation anomaly identification, causal reasoning analysis, and cross-project knowledge transfer, providing reliable protection for foundation pit construction safety.

[0176] Example 11: Practical Application Case

[0177] To verify the practicality and effectiveness of the present invention, a typical engineering case study was selected for testing and analysis. The project involved a subway station foundation pit in a central city area, with a depth of 28 meters and a complex surrounding environment, including multiple high-rise buildings and important underground pipelines.

[0178] System deployment: Figure 8 As shown, 56 vibration sensors, 32 tilt sensors, 48 ​​pressure sensors, and 24 water level sensors are deployed around the foundation pit to form a multimodal monitoring network. Edge computing units are installed in the on-site monitoring room, and cloud-based analysis servers are deployed in the data center.

[0179] System Operation: After the system is operational, the adaptive weight dynamic allocation module adjusts the weights of each sensor in real time based on environmental conditions and data quality. For example, during rainfall, the system automatically increases the weight of the water level sensor; during vibrating construction activities such as piling, the weight of the vibration sensor is increased. The multimodal feature cross-extraction module effectively captures the interrelationships between different sensor data, such as the correlation pattern between groundwater level changes and subsequent soil pressure changes.

[0180] Warning effect: Figure 9 As shown, during the eight-month monitoring period, the system issued 27 effective warnings, with lead times ranging from six hours to three days and an accuracy rate of 92%. Compared with traditional single-modality monitoring systems, warning times were shortened by an average of 16 hours, deformation detection accuracy was improved from centimeters to millimeters, and the false alarm rate was reduced from 15% to below 3%. In particular, during a heavy rainfall event, the system issued a 30-hour advance warning of a potential slip risk in the northwest corner, allowing the construction team ample time to reinforce the support structure and successfully avert a potential accident.

[0181] Deformation Cause Analysis: The system not only detects deformation anomalies but also analyzes their causes through causal reasoning. For example, in one abnormal event, the system identified the causal chain: rising groundwater levels, increased soil saturation, increased lateral earth pressure, and finally deformation of the support structure. This provided engineers with a clear solution: drainage and pressure reduction.

[0182] Knowledge transfer: Through distributed federated learning, the system securely transfers empirical knowledge from one project to another similar project, reducing model adaptation time for new projects from the traditional seven days to one day, significantly improving deployment efficiency. Furthermore, model compression technology reduces resource usage on edge devices by 65%, while only decreasing performance by 5%, enabling efficient edge deployment.

[0183] In general, the practical application results show that the system of the present invention significantly improves the accuracy, reliability and timeliness of foundation pit deformation monitoring, and provides strong support for the safety management of foundation pit construction.

[0184] The multimodal fusion-based intelligent early warning system for foundation pit deformation and its method have good industrial applicability and can be widely used in various foundation pit projects, as embodied in the following aspects:

[0185] 1. In terms of hardware compatibility, the system supports mainstream sensor models on the market and provides standardized interface protocols. The edge computing unit is based on the ARM Cortex-A series processor, meeting the requirements of low power consumption and high performance.

[0186] 2. In terms of deployment flexibility, the modular design supports phased deployment and upgrades, providing cloud, edge, and hybrid deployment modes. The software architecture supports Docker containerization, facilitating rapid deployment and migration.

[0187] 3. In terms of reliability, we implement data redundancy storage and failure recovery mechanisms, design local fault-tolerant processing for network communication interruptions, and introduce a watchdog mechanism to monitor the running status of key processes.

[0188] 4. In terms of real-time performance, the early warning decision delay is controlled within 100ms, meeting emergency response requirements. The streaming processing architecture is used to process high-frequency sampling data, optimize algorithm computational complexity, and reduce resource usage.

[0189] 5. In terms of economy, incremental updates are used to reduce data transmission volume, lower communication costs, implement an on-demand sensor wake-up mechanism, extend battery life, provide a hierarchical storage strategy, and optimize cloud storage costs.

[0190] The above characteristics enable the system of the present invention to adapt to various engineering environments and provide a reliable, efficient and economical technical solution for foundation pit construction safety management.

[0191] This invention provides an intelligent early warning system for foundation pit deformation based on multimodal fusion and its method. Through innovative technologies such as adaptive dynamic weight allocation, multimodal feature cross-extraction, multi-granularity anomaly pattern recognition, causal reasoning analysis, and distributed federated learning, it addresses the adaptability, accuracy, and interpretability issues of traditional foundation pit monitoring systems in complex environments. The system integrates data from multiple sensors, including vibration, tilt, pressure, and water level, to achieve millimeter-level deformation detection and early warning. It also analyzes the causes of deformation and provides treatment recommendations, providing a comprehensive solution for foundation pit construction safety management.

[0192] In actual engineering applications, the present invention significantly improves the deformation detection accuracy and early warning lead time, reduces the false alarm rate, and improves engineering deployment efficiency, and has good industrial applicability and promotion value.

[0193] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. The intelligent early warning system for foundation pit deformation based on multimodal fusion is characterized by: include: An adaptive weight dynamic allocation module is used to evaluate the quality of data from multimodal sensors, generate a weight matrix representing the credibility of each sensor under current environmental conditions, and output the weighted multimodal data; A multimodal feature cross-extraction module, connected to the adaptive weight dynamic allocation module, is used to receive the weighted multimodal data, extract initial features through a modality-specific subnetwork, implement cross-modal feature interaction using a multi-head attention mechanism, and integrate spatial correlations through a spatiotemporal graph convolutional network to generate a fused feature representation; A multi-granularity abnormal pattern recognition module, connected to the multimodal feature cross extraction module, is used to receive the fused feature representation, apply a mixed density network to predict the probability distribution of deformation indicators, and combine time series decomposition and contrastive learning to identify abnormal patterns; a causal reasoning and weight feedback module, connected to the multi-granularity abnormal pattern recognition module, configured to receive the abnormal pattern, infer the deformation cause based on the causal graph model, calculate the sensor influence, and feed back weight adjustment suggestions to the adaptive weight dynamic allocation module; A distributed federated learning module, connected to the multimodal feature cross-extraction module, the multi-granularity anomaly pattern recognition module, and the causal reasoning and weight feedback module, is used to coordinate knowledge sharing and model optimization among multiple projects while protecting the privacy of original data; The causal reasoning and weight feedback module includes: A causal graph construction unit is used to build a directed graph representing the causal relationship between observed variables and latent variables; Causal intervention analysis unit, used to calculate the effects of different intervention strategies; Counterfactual reasoning unit, used to evaluate the deformation development path under different scenarios; A sensor impact evaluation unit is used to calculate the impact of each sensor on anomaly detection; The sensor influence evaluation unit calculates the influence of sensor s by the following formula: ， in Indicates an abnormal event, Represents all sensor data, represents the dataset after removing sensor s, Indicates expected value.

2. The system according to claim 1, wherein: The adaptive weight dynamic allocation module includes: A sensor quality assessment unit, used to calculate a scoring matrix representing the quality of each sensor data; An environmental impact assessment unit, used to generate an environmental correction factor indicating the degree of influence of current environmental parameters on each sensor; A weight calculation unit, configured to generate a weight matrix at a current moment based on the scoring matrix, the historical weights, and the environmental correction factors; Among them, the weight matrix Calculated by the following formula: , Where, represents the data quality score of sensor s at time t, represents the weight at the previous moment, Represents the current environment context vector, 、 、 is a tunable hyperparameter.

3. The system according to claim 1, wherein: The multimodal feature cross extraction module includes: The modal-specific feature extraction unit includes a vibration feature extraction subnetwork, a tilt feature extraction subnetwork, a pressure feature extraction subnetwork, and a water level feature extraction subnetwork, each of which is used to extract the initial features of the corresponding modal; Cross-modal attention unit, used to calculate the interaction between different modal features and generate cross-feature representations; The spatiotemporal graph convolution unit is used to build a graph structure based on the physical layout of sensors and integrate spatial correlation and temporal dependency through graph convolution operations.

4. The system according to claim 3, characterized in that The cross-modal attention unit achieves feature interaction in the following ways: ， ， ， ， in 、 Indicates different modes, 、 、 is the learnable projection matrix.

5. The system according to claim 1, wherein: The multi-granularity abnormal pattern recognition module includes: The deformation index prediction unit is used to map the fusion features into basic displacement indicators; Mixture density network units, used to model multimodal probability distributions for each deformation metric; Time series decomposition unit, used to decompose the deformation series into trend term, seasonal term and residual term; Contrastive learning detection unit that evaluates how the current state differs from known normal and abnormal patterns.

6. The system according to claim 5, characterized in that The mixed density network unit models the conditional probability distribution of the deformation index y through the following formula: , in, It is The weights of the mixture components, represents a Gaussian distribution, and Respectively The mean and standard deviation of the components.

7. The system according to claim 1, wherein: The distributed federated learning module includes: Local training unit, used to train local models on various engineering projects; A secure aggregation unit, used to aggregate model updates from various projects while protecting privacy; Domain adaptation unit, used to deal with distribution shifts between different projects; Knowledge distillation unit, used to transfer complex model knowledge to lightweight models for easy edge deployment.

8. An intelligent early warning method for foundation pit deformation based on multimodal fusion, using the system according to any one of claims 1 to 7, characterized in that: The method comprises: Collect multimodal sensor data, evaluate the quality of each sensor data, generate the current weight matrix based on environmental factors and historical weights, and obtain weighted multimodal data; Extract modality-specific features from the weighted multimodal data, implement cross-modal feature interaction through a multi-head attention mechanism, integrate sensor spatial relationships with a spatiotemporal graph convolutional network, and generate a fused feature representation; Based on the fused feature representation, a mixture density network is applied to predict the probability distribution of deformation indicators, and abnormal patterns are identified through time series decomposition and contrastive learning; Using a causal graph model to infer the possible causes of the abnormal pattern, calculate the impact of each sensor on anomaly detection, and generate weight adjustment suggestions; Distributed federated learning is used to coordinate knowledge sharing among multiple engineering projects, domain adaptation technology is used to deal with distribution shifts, and model optimization is performed and applied to subsequent monitoring.

Images