Rock-soil monitoring signal fidelity noise reduction and adaptive repair method and system for strong interference working condition

Abstract

The application discloses a rock-soil monitoring signal fidelity noise reduction and self-adaptive repair method and system for strong interference working conditions, relates to the technical field of rock-soil engineering safety monitoring and signal processing, and comprises the following steps: obtaining a rock-soil monitoring signal to be processed, performing variational mode decomposition on the rock-soil monitoring signal, and reconstructing to generate a noise reduction basis sequence; constructing a multi-dimensional feature matrix, performing anomaly detection on the noise reduction basis sequence by using an extended isolated forest model, and outputting an anomaly mark sequence; identifying continuous anomaly sections in the noise reduction basis sequence, performing point-by-point or section-by-section prediction filling on data in the anomaly sections, and outputting a repaired complete sequence; and performing variational mode decomposition and noise reduction on the complete sequence again, and quantitatively evaluating the repair quality of the original filled section by using a physical fidelity index. The application solves the problem that a small structure anomaly is smoothed and submerged by an algorithm under strong interference, introduces a real engineering strong interference event waveform as a calibration pulse source, and enhances the engineering fitting property of the constraint condition.

Description

Technical Field

[0001] This invention relates to the field of geotechnical engineering safety monitoring and signal processing technology, and in particular to a method and system for fidelity-preserving noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions. Background Technology

[0002] Geotechnical engineering safety monitoring is a core technical means to ensure the safe operation of major engineering structures such as landslides, deep foundation pits, tunnels, and high slopes. Multi-source sensor systems, represented by inclinometers, earth pressure cells, vibrating wire sensors, and GNSS receivers, continuously collect key parameters such as displacement and stress during construction and operation. Their measured data directly support stability criteria and early warning threshold triggering. However, these sensors inevitably encounter strong interference conditions during long-term field service, such as vibrations from construction machinery, blasting impacts, heavy rainfall, and lightning electromagnetic fields. The original monitoring signals are often superimposed with high-energy non-stationary noise, leading to complex contamination such as pulse-type disturbances, local jumps, and even continuous data loss. Such strong interference signals severely degrade the reliability of monitoring data, resulting in increased false alarm rates and missed alarm risks in automated safety assessment systems. In some cases, minute precursor anomalies may be masked by noise, delaying the optimal early warning window for structural instability.

[0003] To address noise reduction and anomaly repair of monitoring data, existing technologies have developed processing methods such as Variational Mode Decomposition (VMD), Extended Isolation Forest (EIF), and Long Short-Term Memory (LSTM) networks. However, most current conventional solutions simply execute these algorithms sequentially, lacking deep adaptation and coordination for strong interference conditions. Specifically, there are at least three common problems: First, the noise reduction stage is usually optimized based on subjective experience or the final detection statistics, lacking independent constraints directly linked to the physical fidelity of the signal. This makes it easy for weak precursor pulses representing early structural degradation to be lost due to over-smoothing. Second, the anomaly detection stage often uses globally fixed thresholds or static criteria for anomaly marking, failing to perceive and respond to real-time changes in the residual noise level after noise reduction. This results in numerous false alarms during periods of strong interference and insensitivity to small-amplitude gradual anomalies during stable periods. Third, the compensation and repair stage generally uses a single pre-trained model or geometric interpolation for missing data filling, neglecting the significant non-stationarity of soil and rock deformation as it evolves with construction procedures and environmental conditions, leading to insufficient repair accuracy and long-term reliability.

[0004] In summary, how to achieve the organic integration of high-fidelity noise reduction, dynamic detection, and local high-precision compensation under strong interference conditions has become a pressing technical challenge in the field of geotechnical monitoring data management. Summary of the Invention

[0005] Therefore, it is necessary to provide a method and system for high-fidelity noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions to address the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a method for fidelity denoising and adaptive repair of geotechnical monitoring signals under strong interference conditions, including:

[0007] S1. Acquire the soil and rock monitoring signal to be processed, calibrate and lock the variational mode decomposition parameters with physical fidelity index as constraint, perform variational mode decomposition on the soil and rock monitoring signal, remove the noise-dominant mode, and reconstruct and generate a noise-reduced base sequence.

[0008] S2. Based on the denoised base sequence and the noise-dominant mode, a multi-dimensional feature matrix is ​​constructed, and an extended isolated forest model is used to detect anomalies in the denoised base sequence and output anomaly labeling sequences.

[0009] S3. Based on the anomaly marker sequence, identify continuous abnormal segments in the denoised base sequence, train the time series prediction model in situ using the normal data adjacent to the abnormal segments, and perform point-by-point or segment-by-segment prediction and filling of the data within the abnormal segments, outputting the repaired complete sequence.

[0010] S4. Re-execute variational mode decomposition and noise reduction on the complete sequence, and quantitatively evaluate the repair quality of the original filled segment using physical fidelity index. Based on the comparison between the evaluation results and the preset qualified conditions, determine whether to accept the current repair result or trigger the upgrade repair strategy, and output the final repair sequence.

[0011] Secondly, a geotechnical monitoring signal fidelity reduction and adaptive repair system for highly interference-prone working conditions is provided. This system includes:

[0012] The physical constraint noise reduction module is used to acquire the soil and rock monitoring signal to be processed, calibrate and lock the variational mode decomposition parameters with physical fidelity index as constraint, and perform variational mode decomposition on the soil and rock monitoring signal to remove the noise-dominant mode and reconstruct and generate a noise-reduced base sequence.

[0013] The anomaly detection and classification module is used to construct a multi-dimensional feature matrix based on the denoised base sequence and the noise-dominant mode, and to use an extended isolated forest model to detect anomalies in the denoised base sequence and output anomaly label sequences.

[0014] The ready-to-use and trainable repair module is used to identify continuous abnormal segments in a noise-reduced base sequence based on anomaly-marked sequences, train a time series prediction model in situ using normal data adjacent to the abnormal segments, and perform point-by-point or segment-by-segment prediction and filling of the data within the abnormal segments, outputting the repaired complete sequence.

[0015] The closed-loop verification feedback module is used to re-perform variational mode decomposition and noise reduction on the complete sequence, quantitatively evaluate the repair quality of the original filled segment using physical fidelity indicators, and determine whether to accept the current repair result or trigger an upgraded repair strategy based on the comparison of the evaluation results with the preset qualified conditions, and output the final repair sequence.

[0016] The beneficial effects of this invention are as follows:

[0017] 1. By establishing physical independence constraints, the noise reduction process is rigidly capped at the pulse peak retention rate, locking it within a reliable range that retains physical precursor characteristics. This fundamentally solves the problem of small structural anomalies being smoothed out by the algorithm under strong interference. Furthermore, real engineering strong interference event waveforms are introduced as calibration pulse sources, enhancing the engineering fit of the constraints.

[0018] 2. By establishing a dynamic threshold mapping mechanism based on noise residual feedback, an adaptive dynamic balance between noise reduction residuals and detection sensitivity is achieved. This closed-loop mechanism enables the model to automatically suppress false alarms caused by noise disturbances in real soil and rock environments with drastic fluctuations in noise intensity, while restoring sensitivity to weak anomalies when noise levels decrease. Further, a frequency band separation feedforward adjustment is introduced. The coefficients enable the response intensity of the dynamic threshold to be automatically adapted according to the VMD decomposition quality, enhancing the algorithm's adaptability under different signal characteristics.

[0019] 3. By establishing a local, ready-to-use, and trainable LSTM repair strategy, combined with a safety isolation zone and a bidirectional fusion mechanism, the unique time-varying deformation patterns of the vicinity of anomaly points can be captured and repaired in a targeted manner. Compared to the global model, this strategy naturally adapts to the non-stationary dynamic characteristics of soil and rock deformation patterns due to changes in construction conditions and environmental conditions, achieving higher accuracy in missing value compensation. Furthermore, the introduced EIF fractional feedback correction mechanism verifies the quality of the filled values ​​in real time and automatically fine-tunes them during the iterative filling process, significantly suppressing the cumulative error drift in filling continuous missing segments. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0021] Figure 1 This is a flowchart of a method for fidelity denoising and adaptive repair of geotechnical monitoring signals under strong interference conditions according to an embodiment of the present invention;

[0022] Figure 2 This is a flowchart of step S1 in the method for fidelity denoising and adaptive repair of geotechnical monitoring signals under strong interference conditions according to an embodiment of the present invention;

[0023] Figure 3 This is a flowchart of step S2 in the method for fidelity denoising and adaptive repair of geotechnical monitoring signals under strong interference conditions according to an embodiment of the present invention;

[0024] Figure 4 This is a flowchart of step S3 in the method for fidelity denoising and adaptive repair of geotechnical monitoring signals under strong interference conditions according to an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of the principle of a geotechnical monitoring signal fidelity reduction and adaptive repair system for strong interference conditions according to an embodiment of the present invention.

[0026] Figure 6 This is a system architecture diagram (edge ​​computing gateway and cloud monitoring platform) according to an embodiment of the present invention.

[0027] Figure 7 These are screenshots of the system software interface according to an embodiment of the present invention.

[0028] The reference numerals are: 1. Physical constraint noise reduction module; 2. Anomaly detection and classification module; 3. Ready-to-use training and repair module; 4. Closed-loop verification and feedback module. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0030] Please see Figure 1 A method for fidelity denoising and adaptive repair of geotechnical monitoring signals under strong interference conditions, including:

[0031] S1. Acquire the soil and rock monitoring signal to be processed, calibrate and lock the variational mode decomposition parameters with physical fidelity index as constraint, perform variational mode decomposition on the soil and rock monitoring signal, remove the noise-dominant mode, and reconstruct and generate a noise-reduced base sequence.

[0032] Specifically, based on historical data during periods of stable operation without anomalies, VMD (Variational Mode Decomposition) parameters are calibrated and locked using the dual constraints of pulse signal peak energy retention rate and signal-to-noise ratio improvement, ensuring that the noise reduction process is independent of subsequent anomaly detection results. VMD decomposition is performed on the signal to be processed, and the 1 to 2 highest frequency noise modes (i.e., noise-dominant modes) are fixedly removed to reconstruct and generate a noise reduction base sequence.

[0033] In the description of this invention, as Figure 2As shown, the soil and rock monitoring signal to be processed is acquired. Using physical fidelity indices as constraints, variational mode decomposition parameters are calibrated and locked. Variational mode decomposition is then performed on the soil and rock monitoring signal to remove noise-dominant modes and reconstruct a denoised base sequence, including:

[0034] S11. Collect historical monitoring data of the engineering site under stable operating conditions as a calibration dataset.

[0035] Specifically, historical monitoring data of the engineering site under stable operating conditions such as no construction and no rainfall (e.g., 30 consecutive days without abnormalities) are collected, along with historical monitoring data of the engineering site under stable operating conditions and monitoring data corresponding to known typical disturbance events, which together serve as the calibration dataset. Among them, the stable operating condition data is used as the basic threshold for training the extended isolated forest model, and the known typical disturbance event data is used as the reference benchmark for calibrating the pulse peak energy retention rate.

[0036] S12. Set a calibration pulse sequence in the calibration dataset; the calibration pulse sequence is the implanted simulated pulse sequence and the actual pulse sequence extracted from the waveforms of known strong interference events at the engineering site.

[0037] Specifically, to simulate the precursory micro-pulses that may exist under strong interference, simulated pulse sequences with a width of 3 to 5 sampling points and an amplitude of 5 to 8 times the current local standard deviation are randomly implanted into the original calibration data.

[0038] S13. Set the search range for each variational mode decomposition parameter to form multiple parameter combinations, and perform variational mode decomposition on each parameter combination to obtain the corresponding number of intrinsic mode components; the variational mode decomposition parameters include the number of modes and the penalty factor.

[0039] Specifically, the search space for key VMD parameters is defined, including the number of modes. The search range is 2 to 8, with a penalty factor. The search range is 1000 to 5000. For each group... Parameter combination, perform VMD decomposition, and obtain Each intrinsic mode component.

[0040] S14. Sort the intrinsic modal components from high to low center frequency, and according to the value of the number of modes, remove a preset number of noise-dominant modes with the highest center frequency. Then, superimpose and reconstruct the remaining intrinsic modal components to obtain the noise reduction sequence.

[0041] Specifically, the modes are sorted from highest to lowest center frequency, and the one with the highest center frequency is always removed (when...). When the number of decomposition layers is small, the bandwidth of each mode is relatively large, and the highest frequency mode has concentrated most of the high-frequency noise energy, removing one mode can effectively suppress noise) or two modes (when At this stage, the decomposition is more refined, and high-frequency noise energy may be distributed in the two highest modes (so two need to be removed to ensure sufficiency) to determine the noise-dominant mode. The remaining modes are then directly superimposed to reconstruct the denoised sequence.

[0042] S15. For each set of parameters, calculate the physical fidelity index of the denoised sequence. The physical fidelity index includes the signal-to-noise ratio improvement and the pulse peak energy retention rate.

[0043] Specifically, for each set of parameter combinations, two evaluation metrics are calculated: signal-to-noise ratio improvement. and pulse peak energy retention rate The expressions are as follows:

[0044] ;

[0045] ;

[0046] In the formula, It is a noise-free real signal (which is the original data during the stationary period in the calibration stage). This represents the square of the Euclidean norm of the signal. This represents the total number of simulated pulses implanted. Let i be the time-domain segment of the i-th pulse in the original calibration signal. This is the corresponding segment of the pulse after VMD noise reduction and reconstruction. This indicates taking the maximum absolute value within the given segment.

[0047] Furthermore, the calculation of the peak energy retention rate can be achieved by using sensor response signals corresponding to known strong interference events such as construction blasting or heavy machinery impacts at the engineering site as the calibration pulse source, instead of manually injected simulated pulses. Specifically, several segments of sensor output waveforms corresponding to blasting vibrations or impacts confirmed by on-site construction logs are extracted from historical data during the stable period. The maximum absolute peak value of each event waveform is extracted as a reference value. After VMD noise reduction and reconstruction, the maximum absolute peak value of the noise-reduced waveform is extracted from the corresponding time position, and the ratio of the two is the peak energy retention rate. The advantage of this scheme is that it utilizes typical strong interference waveforms that have occurred in real engineering projects for calibration. Their spectral characteristics and amplitude characteristics naturally match the actual noise environment of the monitoring site, overcoming the potential mismatch between artificially simulated pulses and real strong interference signals in terms of frequency band distribution and energy level. This further enhances the engineering applicability and calibration reliability of the physical independence constraint in step one.

[0048] S16. Using the signal-to-noise ratio improvement being greater than or equal to the first threshold and the pulse peak energy retention rate being greater than or equal to the second threshold as dual constraints, filter the parameter combinations that satisfy the dual constraints, and select the optimal locking parameters, including the number of locked modes and the penalty factor.

[0049] Specifically, with and As a double rigid constraint, the selection process is based on meeting the conditions. Combinations. If multiple combinations satisfy the condition, select the appropriate combination. The largest value is used as the locking parameter; if no combination satisfies both conditions simultaneously, the condition is relaxed. After reaching 85%, re-filter within the original search space and select... The combination with the highest value is used as the locking parameter. Once determined, this locking parameter remains unchanged throughout the entire engineering monitoring period and is not adjusted in any way based on subsequent downstream indicators such as the F1 score of EIF testing or the root mean square error of LSTM compensation.

[0050] In the description of this invention, a dual constraint condition is used, where the signal-to-noise ratio improvement is greater than or equal to a first threshold and the pulse peak energy retention rate is greater than or equal to a second threshold. Parameter combinations that satisfy the dual constraint condition are screened, and the optimal locking parameters are selected, including:

[0051] S161. If multiple sets of parameter combinations simultaneously satisfy the dual constraint conditions, the one with the largest increase in signal-to-noise ratio shall be selected as the locking parameter.

[0052] S162. If there is no parameter combination that satisfies all conditions, the second threshold of the pulse peak energy retention rate is lowered and the parameters are re-selected, and the combination with the highest pulse peak energy retention rate is selected as the locking parameter. If there is still no parameter combination that satisfies the conditions after lowering the second threshold, the combination with the highest pulse peak energy retention rate among all parameter combinations is selected as the locking parameter.

[0053] S17. Variational mode decomposition is performed on the geotechnical monitoring signal to be processed using the locked mode number and penalty factor, and reconstructed according to the same noise-dominant mode elimination rule to obtain the noise-reduced base sequence.

[0054] Specifically, for the strong interference monitoring sequence to be processed, the locked sequence is directly used. and Perform VMD decomposition and reconstruct the data according to the same high-frequency noise removal rules to obtain the denoised basis sequence. This serves as the basic input for subsequent steps.

[0055] S2. Based on the denoised base sequence and the noise-dominant mode, a multi-dimensional feature matrix is ​​constructed, and an extended isolated forest model is used to detect anomalies in the denoised base sequence and output anomaly labeling sequences.

[0056] Specifically, this step constructs a dynamic closed-loop feedback path between noise reduction residue and detection sensitivity, while introducing interference source classification and anomaly degree grading mechanisms to achieve differentiated intelligent responses.

[0057] In the description of this invention, as Figure 3 As shown, a multi-dimensional feature matrix is ​​constructed based on the denoised base sequence and the noise-dominant mode. An extended isolated forest model is then used to detect anomalies in the denoised base sequence, and the output anomaly marker sequence includes:

[0058] S21. Extract the noise-dominant modes that were removed in the variational mode decomposition, calculate the root mean square value of the sliding window to form the original noise intensity sequence, and normalize the original noise intensity sequence to obtain the noise intensity curve.

[0059] Specifically, after VMD decomposition, the first [item] that was removed... The mode (noise-dominant mode) is denoted as .right Calculate the root mean square value of its sliding window to form a noise intensity sequence. To eliminate the influence of the original signal amplitude dimensions, the original noise intensity sequence is normalized and mapped to the interval [0, 1], resulting in the noise intensity curve. .

[0060] S22. Apply a sliding window to the noise reduction base sequence, extract the temporal statistical features and differential features under each window, and incorporate the noise intensity curve as a one-dimensional additional feature into the temporal statistical features and differential features to form a complete multi-dimensional feature matrix.

[0061] Specifically, for the noise reduction base sequence Apply a sliding window (the window width matches the data sampling frequency and the physical period; for example, 48 points corresponding to 48 hours can be used for hourly data) to extract time-series statistical features and difference features under each window. Feature dimensions should include at least: window mean. Window standard deviation Window poor The maximum absolute value of a first-order difference sequence Second-order difference mean and the deviation of the current value from the window mean. Then normalize the result. As a one-dimensional additional feature, it is incorporated to form a complete multi-dimensional feature matrix X, where the eigenvector in the t-th row is denoted as... .

[0062] S23. Using the calibration dataset, after selecting the locking parameters, execute steps S21 and S22 to obtain the corresponding calibration feature matrix, train the extended isolated forest model, and calculate the specified percentile of the abnormal scores of all normal samples as the base threshold.

[0063] Specifically, using the calibration data from the stationary period obtained in step S1, after locking the VMD parameters, steps S21 and S22 are performed to obtain the corresponding feature matrix, and an EIF model (Extended Isolation Forest model) is trained. The key parameter of EIF is set as follows: number of trees. Subsampling size Scalability (d represents the feature dimension, achieving full expansion). Calculate the 95th or 99th percentile of the abnormal scores for all normal samples, denoted as the baseline threshold. .

[0064] S24. Based on the local mean of the noise intensity curves within the local window before and after the current time, establish a dynamic threshold mapping function; when the local noise intensity increases, the dynamic threshold automatically increases, and when the local noise intensity decreases, the dynamic threshold automatically converges to the basic threshold.

[0065] Specifically, when detecting or processing non-stationary data online, a dynamic threshold is defined. for:

[0066] ;

[0067] In the formula, This is the adjustment coefficient, typically taken as 0.1 to 0.3; Within a short time window before and after the current moment (e.g., 10 points before and 10 points after). The local mean of the function. The physical meaning of this function is: when the local noise intensity increases (e.g., due to the introduction of vibrations from construction blasting), As the noise level rises, the dynamic threshold automatically increases, suppressing false high-score anomalies caused by residual noise; when the noise intensity decreases to a stable level... Approaching a stable low value, Automatic convergence to Maintain a keen ability to detect subtle real anomalies.

[0068] Adjustment coefficient in dynamic threshold mapping function Its value is not fixed, but is automatically calculated based on the center frequency distribution of each mode obtained from the VMD decomposition in step S1. Define the frequency band separation. for:

[0069] ;

[0070] In the formula, The center frequency of the noise mode eliminated in step one. The center frequency of the adjacent highest-order non-noise mode is retained. Based on this, the adjustment coefficient... The value is determined according to the following adaptive formula:

[0071] ;

[0072] In the formula, The base adjustment factor (default 0.2) This is the scaling factor (default 5.0). The physical meaning of this adaptive mechanism is: when VMD can clearly separate the noise mode from the effective signal mode in the frequency domain ( A relatively large noise level indicates that the noise reduction effect is sufficient and the residual noise level is stable. Automatic reduction, dynamic threshold The response to noise fluctuations tends to be conservative and stable, avoiding unnecessary frequent threshold adjustments; when the frequency band separation is poor ( A smaller value indicates that noise and signal have frequency domain aliasing, making noise reduction difficult and leaving high uncertainty in residual noise. The dynamic threshold is automatically increased, making it more sensitive to changes in noise intensity to suppress additional false alarms caused by frequency band aliasing, thereby achieving feedforward adaptive adjustment of the frequency band separation degree to the dynamic threshold response intensity.

[0073] S25. For the feature vector at the time to be detected, calculate the original anomaly score using the trained extended isolated forest model; compare the original anomaly score with the dynamic threshold, calculate the anomaly deviation ratio, and mark the current time as an anomaly based on the range of the anomaly deviation ratio.

[0074] Specifically, for the feature vector at the time to be detected The original anomaly score is obtained using the EIF trained in step S23. Compare it with the dynamic threshold calculated in step S24 Comparison, defining abnormal deviation ratios .in accordance with The value categorizes exceptions into three levels: (If it is level 0), then it is judged as normal and marked. No processing is triggered. (Level 1) indicates a mild abnormality and is marked as such. Only logs are recorded; if more than three consecutive Level 1 anomalies occur, the case is upgraded to Level 2. (Level 2) is then judged as abnormal and marked. This triggers the anomaly handling process and enters the interference source type determination and differential repair phase, mandating closed-loop verification. Among these, the anomaly score... The calculation formula is:

[0075] ;

[0076] In the formula, For the sample Average path length across all extended isolated trees For a given subsample size The average path length normalization constant under the following conditions is the harmonic number, and the constant 0.5772 is Euler's constant.

[0077] S26. For time periods with abnormal markers, the time-frequency characteristics of each modality and the fluctuation pattern of the noise intensity curve are used to perform similarity matching with various interference features in the pre-constructed interference source feature library to determine the type of interference source, and to assign differentiated repair processing strategies according to the type of interference source.

[0078] Specifically, based on the anomaly detection, for each time period marked as an anomaly, the time-frequency characteristics and noise intensity curves of each mode obtained from the VMD decomposition in step one are used. The system automatically classifies interference sources based on their fluctuation patterns. Beforehand, it extracts spectral, temporal, and energy features from sensor signals corresponding to typical interference events at the engineering site, such as blasting operations, heavy machinery operations, and heavy rainfall, to construct an interference source feature library. During online classification, it extracts the corresponding features of the current anomaly segment, calculates the similarity with each type in the feature library, and labels the type with the highest similarity exceeding a threshold as the interference source. Differentiated processing strategies are assigned based on different types.

[0079] Explosive impact type: When the length of the abnormal segment is ≤3 points, it is directly removed and linear interpolation is used without triggering LSTM repair; when the length is >3 points, LSTM repair is triggered, and the data on the right side is used first.

[0080] Mechanical vibration type: The LSTM input step size L is adjusted to an integer multiple of the vibration period;

[0081] Electromagnetic interference type: Directly eliminate and replace with local median filtering, without triggering LSTM repair;

[0082] Rainfall noise type: Expand the training data time window to include the entire rainfall process;

[0083] Composite interference type: Standard LSTM repair is used, and the closed-loop verification threshold is verified. Increased to 90%;

[0084] Unknown interference type: adopt the standard process, and if the closed-loop verification fails, it will directly trigger manual review.

[0085] S3. Based on the anomaly marker sequence, identify continuous abnormal segments in the denoised base sequence, train the time series prediction model in situ using the normal data adjacent to the abnormal segments, and perform point-by-point or segment-by-segment prediction and filling of the data within the abnormal segments, outputting the repaired complete sequence.

[0086] Specifically, this invention abandons the commonly used global pre-trained single prediction model in the prior art, and instead uses the normal data of the clean area next to it to train a lightweight long short-term memory network model on the spot to fill in each abnormal segment marked in step S2, which is ready to be trained and discarded after use.

[0087] In the description of this invention, as Figure 4 As shown, based on the anomaly marker sequence, continuous anomalous segments are identified in the denoised base sequence. A time-series prediction model is trained in situ using normal data adjacent to the anomalous segments. The data within the anomalous segments is then predicted and filled point-by-point or segment-by-segment. The output, repaired complete sequence includes:

[0088] S31. Merge consecutive abnormal points into one or more abnormal segments according to the abnormal marker sequence, and set a preset number of sampling points before and after each abnormal segment as a physical safety isolation zone.

[0089] Specifically, consecutive outliers are merged into several outlier segments based on the outlier label sequence. For each outlier segment, M sampling points before and after it (default M=5) are set as a physical safety isolation zone. Data points within these isolation zones do not participate in the construction of the training set to prevent distortion values ​​from the outlier edges from contaminating the LSTM training samples.

[0090] S32. Set the normal data region that can be used for training and compare it with the preset minimum length threshold required for training. Select the prediction method based on the comparison result. The prediction method includes forward long short-term memory network, reverse long short-term memory network or bidirectional weighted fusion.

[0091] Specifically, a clean area (normal data area) is defined for training. The left side consists of continuous normal sequences before the start of the abnormal segment and after removing the isolation band, while the right side consists of continuous normal sequences after the end of the abnormal segment and after removing the isolation band. The minimum length threshold required for training is set to 3L (L is the LSTM input stride, default 48, ensuring at least 3 times the input length of the training samples). Prioritize training the forward LSTM using sequences with sufficient length on the left side; if the left side is insufficient but the right side is sufficient, and the processing mode is non-real-time offline batch mode, switch to the inverse LSTM using the right side data; if both sides are sufficient, the forward and inverse models can be trained in parallel. During the prediction phase, weighted fusion is performed based on the distance of the abnormal point from the left and right clean areas. The weighted fusion formula is:

[0092] ;

[0093] In the formula, and Forward LSTM and backward LSTM for outliers, respectively. The predicted value, and These are the outliers. The weighting method, which considers the distance to the left and right clean zone boundaries, makes the prediction results more reliable for directions that are closer to the boundary.

[0094] The online mode receives monitoring data in real-time, either point-by-point or segment-by-segment, and the system must complete processing before the next data point arrives, placing strict requirements on latency. The criteria for this mode are: the system runs on an edge computing gateway, the data source is a real-time data stream, and the system parameter `mode` is set to `online`. This mode only supports forward LSTM prediction. The offline batch mode receives and stores complete monitoring data. The system reads entire data segments in batches for post-processing, with no strict latency requirements, and can utilize the complete data context on both sides of anomaly points. The criteria for this mode are: the system runs on a cloud server, the data source is batch data already written to disk, and the system parameter `mode` is set to `offline`. This mode supports forward LSTM, reverse LSTM, and bidirectional weighted fusion.

[0095] In the description of this invention, a trained time-series prediction model is used to perform single-step prediction and fill in each anomaly point within an anomaly segment. For continuous anomaly segments, an iterative single-step prediction method is adopted, incorporating the current predicted value into the subsequent input window to sequentially scroll and fill the entire segment, including:

[0096] S321. Take the continuous normal sequence after removing the isolation band before the start of the abnormal segment as the left data source, and take the continuous normal sequence after removing the isolation band after the end of the abnormal segment as the right data source.

[0097] S322. Based on the input step size of the time series prediction model, set the minimum length threshold required for training.

[0098] S323. Calculate the lengths of the left and right data sources respectively, and compare each length with the minimum length threshold.

[0099] S324. Select the prediction method based on the comparison results. If only the length of the left data source meets the requirements, select the forward long short-term memory network. If only the length of the right data source meets the requirements and the current mode is offline batch processing, select the reverse long short-term memory network. If the lengths of both the left and right data sources meet the requirements, select the bidirectional weighted fusion method.

[0100] S33. Based on the selected prediction method, train the time series prediction model using normal data; the time series prediction model includes an input layer, a long short-term memory hidden layer, and a fully connected output layer.

[0101] Specifically, a JIT ultra-lightweight LSTM model is trained to obtain a time-series prediction model. The architecture consists of an input layer (input dimension L×1), an LSTM hidden layer (32 neurons), followed by a fully connected output layer (output dimension 1, linear activation). Mean squared error is used as the loss function, and Adam is used as the optimizer. An early stopping strategy is employed during training: the first 90% of the local training data is used as the training set, and the last 10% as the validation set. Training is terminated and the optimal weights are restored when the validation loss does not decrease for three consecutive epochs. The maximum number of training epochs does not exceed 50.

[0102] S34. Using the trained time series prediction model, perform single-step prediction and fill in each abnormal point in the abnormal segment. For continuous abnormal segments, adopt the iterative single-step prediction method, and include the current prediction value into the subsequent input window to sequentially scroll and fill the entire segment.

[0103] Specifically, for single-point anomalies, L clean points (with isolation bands removed) to the left of the current point are taken as the input sequence. After the time series prediction model outputs the predicted values, these are filled in and marked as LSTM compensation. For continuous anomaly segments, an iterative single-step prediction strategy is adopted: first, the first anomaly point is predicted and filled, and this predicted value is used as a known value in the input window of the next anomaly point, and so on, filling the entire segment sequentially. When the length of the anomaly segment exceeds a preset threshold (e.g., 12), a confidence reduction warning is issued in online mode, and in offline mode, it automatically switches to bidirectional LSTM weighted fusion filling.

[0104] When performing iterative single-step prediction and filling of continuous outlier segments, a filling confidence weighted correction mechanism based on EIF outlier scores is introduced. The specific operation is as follows: In each iteration, the filling value of the k-th outlier point is predicted. Then, the filled value is temporarily put back into the sequence, and the multidimensional feature vector corresponding to that point is calculated. and EIF abnormal score If abnormal scores Below the current dynamic threshold If the confidence level is high, the filled value is retained and the prediction of the next outlier continues; if the outlier score is low... This indicates that the filling value is inconsistent with the dynamic pattern of the neighboring sequence, and the prediction may be biased. At this time, a local fine-tuning is automatically triggered to... Centered on, with radius (where This is the scaling factor, with a default value of 0.5. Generate a perturbation candidate set (based on the local standard deviation of the adjacent clean area) { , , , , The corresponding feature vectors are constructed one by one, and the EIF anomaly score is calculated. The candidate value with the lowest anomaly score is selected as the final filling value. This correction mechanism re-couples the anomaly detection module in step two with the compensation and repair module in step three in a closed loop. In the deep prediction stage, where the risk of error accumulation in filling continuous missing segments is high, the EIF model is used as a "filling quality controller," which can significantly suppress the inherent drift trend of iterative prediction and improve the consistency between the filling sequence of long continuous anomaly segments and the global normal pattern.

[0105] In the description of this invention, a trained time-series prediction model is used to perform single-step prediction and fill in each anomaly point within an anomaly segment. For continuous anomaly segments, an iterative single-step prediction method is adopted, incorporating the current predicted value into the subsequent input window to sequentially scroll and fill the entire segment, including:

[0106] S341. For a single anomaly, take the number of consecutive clean points corresponding to the input step size to the left of the current anomaly point as the input sequence, input them into the time series prediction model to obtain the predicted value, and then fill it into the current anomaly point.

[0107] S342. For continuous outlier segments, starting from the first point of the outlier segment, if a forward long short-term memory network or bidirectional weighted fusion is used, the number of data points corresponding to the input step size to the left of the current outlier point is taken as the input sequence; if a reverse long short-term memory network is used, the number of data points corresponding to the input step size to the right of the current outlier point is taken and the input sequence is constructed in reverse order of time. After the predicted value is output by the time series prediction model, it is filled into the current outlier point; the predicted value is included in the input window to predict the next outlier point, and this process is iterated until the entire outlier segment is completely filled.

[0108] S343. After predicting the fill value of the current outlier in each iteration, the fill value is temporarily put back into the sequence. The extended isolated forest outlier score at the corresponding time of the current outlier is calculated. If the outlier score is lower than the adaptive detection threshold at the current time, the fill value is retained and the prediction of the next outlier continues. If the outlier score is greater than or equal to the adaptive detection threshold at the current time, the fill value is locally fine-tuned and then filled in.

[0109] S35. Once the outlier segment is filled, immediately release the temporarily trained time-series prediction model and record the model information during the processing as a structured log. The model information includes the original value, the value after noise reduction, the outlier label, the outlier score, the filled value, and the model confidence.

[0110] Specifically, after the anomalous segment is filled, the temporarily trained LSTM model is immediately released to reduce memory overhead. At the same time, the original value, the denoised value, the anomalous label, the anomalous score, the filled value, the boundaries of the training data window used, the model confidence, and the computation time are recorded as a structured log for upstream platforms to retrieve and trace.

[0111] S4. Re-execute variational mode decomposition and noise reduction on the complete sequence, and quantitatively evaluate the repair quality of the original filled segment using physical fidelity index. Based on the comparison between the evaluation results and the preset qualified conditions, determine whether to accept the current repair result or trigger the upgrade repair strategy, and output the final repair sequence.

[0112] Specifically, after the LSTM repair is completed in step S3, the repaired complete sequence is re-input into the VMD noise reduction module in step S1 to perform secondary noise reduction and physical fidelity verification, forming a global closed loop from noise reduction, detection, repair, verification to feedback.

[0113] In the description of this invention, variational mode decomposition and noise reduction are re-executed on the complete sequence. The repair quality of the original filled segment is quantitatively evaluated using physical fidelity indicators. Based on the comparison of the evaluation results with preset acceptance criteria, it is determined whether to accept the current repair result or trigger an upgraded repair strategy. The final repair sequence output includes:

[0114] S41. Using the complete sequence as input, perform variational mode decomposition using the same locked set of mode numbers and penalty factors, and reconstruct it according to the same elimination rules to obtain the secondary denoised sequence.

[0115] Specifically, the complete repair sequence output in step S3 As input, use the same group locked in step one. The parameters are subjected to VMD decomposition and reconstructed according to the same high-frequency elimination rules to obtain a secondary noise reduction sequence. .

[0116] S42. For the complete sequence, identify the time domain interval corresponding to the abnormal segment of the time series prediction model, take the local peak of the signal in the time domain interval as the evaluation object, and calculate the pulse peak energy retention rate of the secondary noise reduction sequence as the restoration fidelity index.

[0117] Specifically, for the repaired complete sequence, the time-domain interval corresponding to the original LSTM aberration segment is identified. The local peak value of the signal within this interval is used as the evaluation object to calculate the pulse peak energy retention rate after secondary noise reduction. The calculation method is the same as in step S15. The formula.

[0118] In this step, Take the signal segment corresponding to the time domain interval from the complete sequence. Extract the signal segment corresponding to the time domain interval from the secondary noise reduction sequence.

[0119] It should be noted that during the closed-loop verification stage in step S42, the pulse peak energy retention rate... The objectivity stems from the following facts: the locking parameters in step S1 The real calibration data that has passed the stable period is independently optimized and fixed. It characterizes the global noise-signal statistical distribution model of the current monitoring point under the condition of no abnormality in the frequency domain, which will not be changed by the repair behavior of any subsequent abnormal segment.

[0120] Based on this In this step, the relative structural consistency coefficient between the repair sequence and the calibrated denoising model is defined. Its calculation still follows the formula in step S15, only with the formula modified accordingly. Replace with complete sequence The signal segment corresponding to the original abnormal segment will be Replace with a secondary noise reduction sequence The signal segment corresponding to the original abnormal segment.

[0121] The physical meaning of this coefficient is: the repaired complete sequence After re-entering the locked variational mode decomposition filter, the degree to which the peak energy of the signal within the original anomalous segment is preserved. This is determined by the overall frequency domain statistical characteristics (i.e., ...) of a local waveform in the repair sequence compared to the normal sequence. If the corresponding frequency band division criteria are incompatible, the local waveform will be identified as a noise-dominant mode and largely eliminated during second-order variational mode decomposition, resulting in... A significant decrease; conversely, if the repair sequence is compatible with the time-frequency statistical characteristics of the adjacent normal segment, then It can be maintained above the acceptable threshold. Therefore, It provides an objective statistical benchmark for testing the quality of repair, the validity of which is based on the parameter locking that has been independently completed in step S15.

[0122] S43. When the repair fidelity index is greater than or equal to the preset qualified threshold, the repair quality is deemed qualified and the current filling result is accepted; when the repair fidelity index is lower than the preset qualified threshold, different levels of response strategies are triggered according to the degree of deviation of the repair fidelity index.

[0123] Specifically, set a fidelity qualification threshold. .like If so, the repair quality is considered acceptable, and the filling result is accepted. This indicates that the peak energy of the repaired filler segment decayed excessively after secondary noise reduction, triggering a graded response.

[0124] Level 1 response (slight deviation) ): Automatically revert to step three, increase the number of neurons in the LSTM hidden layer to 64, extend the maximum number of training rounds to 100, retrain and refill, and re-execute the closed-loop verification, for a maximum of 2 loops; if the loop is completed 2 times... If the result is still below 85%, it will automatically downgrade to a Level 2 response strategy. If the Level 2 response is still unsatisfactory, it will be directly upgraded to a Level 3 response, and the automatic result filling will be abandoned.

[0125] Level 2 response (moderate deviation) ): Switch to bidirectional LSTM weighted fusion mode or relax the security isolation band from M=5 to M=3, then repair and verify again;

[0126] Level 3 response (serious deviation) The abnormal segment is automatically marked as having insufficient confidence for repair, the automatic filling result is abandoned, a manual review reminder log is generated, and it is pushed to the monitoring platform for engineers to intervene and make a judgment.

[0127] S44. Record the repair fidelity index value, response level, number of loops, and final processing result of each closed-loop verification as a structured log.

[0128] Specifically, each closed-loop verification will be performed. Values, response levels, number of loops, and final processing results are recorded as structured logs for subsequent statistical analysis.

[0129] Please see Figure 5 A system for high-fidelity noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions is provided. This system includes:

[0130] The physical constraint noise reduction module 1 is used to acquire the soil and rock monitoring signal to be processed, calibrate and lock the variational mode decomposition parameters with physical fidelity index as constraint, perform variational mode decomposition on the soil and rock monitoring signal, remove the noise-dominant mode, and reconstruct and generate a noise-reduced base sequence.

[0131] Anomaly detection and classification module 2 is used to construct a multi-dimensional feature matrix based on the denoised base sequence and the noise-dominant mode, and to use an extended isolated forest model to perform anomaly detection on the denoised base sequence, outputting anomaly label sequences.

[0132] The ready-to-use and trainable repair module 3 is used to identify continuous abnormal segments in the denoised base sequence based on the abnormal marker sequence, train the time series prediction model in situ using the normal data adjacent to the abnormal segment, and perform point-by-point or segment-by-segment prediction and filling of the data within the abnormal segment, and output the repaired complete sequence.

[0133] The closed-loop verification feedback module 4 is used to re-perform variational mode decomposition and noise reduction on the complete sequence, quantitatively evaluate the repair quality of the original filling segment using physical fidelity indicators, and determine whether to accept the current repair result or trigger an upgraded repair strategy based on the comparison between the evaluation result and the preset qualified conditions, and output the final repair sequence.

[0134] like Figure 6As shown, this system is divided into three main parts: the perception layer, the edge computing gateway, and the cloud monitoring platform. The perception layer consists of a multi-source sensor composed of an inclinometer, earth pressure cell, vibrating wire sensor, and GNSS receiver, used to collect raw geotechnical monitoring signals from the geotechnical engineering site and send the collected monitoring signals to the edge computing gateway. The edge computing gateway is equipped with an ARM Cortex architecture and a GPU. Internally, the gateway sequentially performs VMD noise reduction, EIF detection, and JIT-LSTM repair. The VMD noise reduction adopts physical independence constraints, with exemplary parameters K=6 and α=2000. The EIF detection is configured with a dynamic threshold. And achieve Adaptive adjustment and JIT-LSTM repair are employed to perform local modeling, use-and-discard, and combine EIF confidence correction to complete signal repair. The edge computing gateway is equipped with a circular buffer that can store 90 days of data, storing the processed data and structured JSON logs in a local database. After processing, the data is uploaded to the cloud monitoring platform. The cloud monitoring platform is configured with a web visualization interface to display an overlay view of the original, denoised, and repaired signals, as well as highlighting anomalies in red. It also has functions such as one-click manual confirmation / cancellation, inputting and outputting data to the early warning model, and retrieving structured logs.

[0135] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments described herein are merely illustrative of the invention and are not intended to limit the scope of protection of the invention.

[0136] Example 1

[0137] This example uses a deep displacement monitoring project of a large landslide as its engineering background. The landslide is equipped with an automated fixed inclinometer, with a monitoring depth of 10m and a sampling interval of 1 hour, and has been operating continuously for more than 2 years. During the monitoring period, it experienced several heavy rainfall events and blasting operations for nearby road construction. The original monitoring sequence contained high-amplitude pulse noise, periodic temperature drift interference, and missing zero-value segments due to data transmission packet loss.

[0138] According to the method of the present invention, the above monitoring data is processed in accordance with the following steps.

[0139] Step 1: VMD noise reduction under physical independence constraints

[0140] First, 30 consecutive days of historical monitoring data for the landslide during the non-flood season and without any blasting operations were selected as the calibration dataset for the stable period. No early warning events occurred during this period, and manual verification confirmed that there were no abnormalities.

[0141] To simulate the subtle precursor pulse response that may occur under strong interference, 20 rectangular pulse sequences with a width of 4 sampling points (corresponding to 4 hours) and an amplitude of 6 times the standard deviation of the current sliding window (48 points wide) were randomly inserted into the calibration data at random time intervals. The width and amplitude of these simulated pulses were designed with reference to the typical precursor signal characteristics recorded in the previous accelerated creep stages of this landslide.

[0142] Setting the number of VMD modes The search range is {3, 4, 5, 6, 7, 8}, with a penalty factor of [missing information]. The search range is {1000, 2000, 3000, 4000, 5000}, totaling 30 parameter combinations. For each parameter combination, a complete VMD decomposition and reconstruction process is performed on the calibration data:

[0143] (1) Call the VMD algorithm to decompose the signal into one modal component { The modes are arranged from low to high according to their center frequencies;

[0144] (2) The one with the highest center frequency The modal was identified as a noise mode and discarded.

[0145] (3) Put the remaining The noise-reconstructed signal is obtained by superimposing the modes. ;

[0146] (4) Calculate the two evaluation indicators.

[0147] The pulse peak energy retention rate Calculate using the following formula:

[0148] ;

[0149] In this embodiment, N p =20, for each implanted pulse, extract its time-domain segment from the original calibration signal and the noise-reconstructed signal respectively. (Within a range of 50 sampling points before and after the pulse center time), extract the maximum absolute value within the segment, calculate the ratio, and then take the average of 20 pulses.

[0150] The signal-to-noise ratio (SNR) improvement ΔSNR is calculated using the following formula:

[0151] ;

[0152] In the formula, This is calibration data for the stationary period (this data can be approximated as a noise-free real signal during the stationary period). It is the square of the second norm of the entire sequence.

[0153] Iterate through 30 sets of parameter combinations and record the corresponding values ​​for each set. and After comparison and selection, the parameter combinations were... , Simultaneously satisfy and Since it meets the dual constraints, it has been locked as the VMD working parameter for this project. Once locked, this parameter combination will not be modified regardless of the subsequent EIF test results.

[0154] The original monitoring sequence during the period of strong interference to be processed is input into the VMD module, and decomposition is performed with K=6 and α=2000. The 6th mode with the highest center frequency is removed, and the first 5 modes are superimposed to obtain the noise-reduced base sequence. The sequence length is the same as the original sequence.

[0155] Step 2: EIF Dynamic Anti-Anomaly Detection Based on Noise Residual Feedback

[0156] Extract the 6th mode that was removed in step one. The noise intensity curve is calculated. In this embodiment, the root mean square value of a sliding window is used as the intensity measure: the window width is taken as... Points (corresponding to 24 hours) are used to calculate the local root mean square value at each time point, forming... Subsequently, Linearly normalize to the [0, 1] interval to obtain the normalized noise intensity curve. :

[0157] ;

[0158] For the denoised sequence Construct multidimensional features. Set the sliding window width to 48 points and extract the following feature dimensions: window mean. Window standard deviation Window poor The maximum absolute value of a first-order difference sequence Second-order difference mean and the deviation of the current value from the window mean. and This ultimately forms the feature matrix X, where each row corresponds to a 7-dimensional feature vector at a given time step. .

[0159] Using the same stationary period calibration data used in step one, after locking the VMD parameters, steps two (1) and two (2) are re-executed to obtain the feature matrix, and the EIF model is trained. The EIF parameters are set as follows: number of trees. Subsampling size The expansion degree is l=6 (feature dimension is 7, l=d−1 achieves full expansion). After training, the 95th percentile of the abnormal scores of all normal samples is calculated and denoted as the base threshold. In this embodiment .

[0160] Determine the adjustment coefficient First, calculate the frequency band separation. After decomposition in step one, the center frequency of the eliminated noise mode (6th mode) =0.198Hz, the center frequency of the highest-order non-noise mode (5th mode) retained. ,but:

[0161] ;

[0162] Take the base adjustment coefficient =0.2, scaling factor Calculate adaptive :

[0163] ;

[0164] For each time t to be detected, calculate its dynamic threshold:

[0165] ;

[0166] In the formula, Take 10 points before and after the current time (21 points in total). Local mean.

[0167] Feature vector at the time of detection Anomaly scores are calculated using a trained EIF model. The abnormal score is calculated using the following formula:

[0168] ;

[0169] In the formula, =256, H(k)=ln(k)+0.5772. E[h(xt)] is the sample... The average path length from the root node to a leaf node in all 100 expanded isolated trees. =, is considered an anomaly, and is marked. Otherwise, mark .

[0170] According to statistics, during the three-month period of strong interference including blasting construction and heavy rainfall, a total of 67 abnormal points were detected in this embodiment. After comparison with the on-site construction log and manual verification, 62 of them corresponded to blasting events, sensor transmission packet loss or known equipment disturbances, 3 corresponded to actual slope accelerated creep events, and only 2 were suspected false alarms, with a false alarm rate of about 3.0%.

[0171] Step 3: Local JIT-LSTM Repair Based on Deformation Memory Effect

[0172] Based on the abnormal label sequence The continuous anomalies are merged into anomaly segments. In this embodiment, a total of 67 anomaly locations were obtained, which were merged into 42 anomaly segments, of which 39 segments were single-point anomalies and 3 segments were continuous anomalies (with lengths of 3 points, 7 points and 15 points respectively).

[0173] For each anomalous segment, a physical safety isolation zone with M=5 sampling points is defined. The isolation zone is drawn by taking 5 points forward and 5 points backward from the start and end boundaries of the anomalous segment. Data within the isolation zone is not used for training.

[0174] Identify available training data sources. Set the LSTM input stride to L=48, and the minimum training length threshold to 3L=144 points.

[0175] Taking a continuous anomalous segment of length 15 points (corresponding to continuous packet loss during a heavy rainfall event) as an example: after removing the isolation band on the left side of this segment, there is a continuous normal sequence of length 310 points, which meets the 144-point threshold requirement; the normal sequence on the right side has a length of 186 points, which is also sufficient. Therefore, a bidirectional LSTM weighted fusion strategy is adopted.

[0176] 144 points from the left clean zone closest to the anomalous segment were taken as training data for the forward LSTM, and 144 points from the right clean zone closest to the anomalous segment were taken as training data for the backward LSTM. Weighted fusion was implemented during the prediction phase based on the different distances from the boundary of the anomalous segment.

[0177] The LSTM model adopts the following lightweight structure: a single-layer LSTM with 32 hidden units; an input dimension of 1 (univariate sequence) and an input step size of L=48; a fully connected output layer with 1 neuron, linear activation, and outputting a single-step prediction value.

[0178] The loss function of the LSTM model is the mean squared error. The Adam optimizer was used with a learning rate of 0.001. An early stopping strategy was implemented: the first 90% of the training data was used as the training set and the last 10% as the validation set in chronological order. Training was stopped if the validation loss did not improve for three consecutive rounds.

[0179] After the forward and backward models are trained independently, predictions are made for each of the 15 outliers within the continuous outlier segment. For the k-th outlier... The weighted fusion formula is:

[0180] ;

[0181] In the formula, and These are the outliers. Number of sampling points to the left and right clean zone boundaries (excluding isolation zones).

[0182] When filling the continuous outlier segment of length 15, a fill confidence weighted correction mechanism based on EIF outlier scores is enabled. The fill value is predicted in each iteration. Then, it is put back into the sequence to construct a 7-dimensional feature vector for that point. Calculate the EIF anomaly score With the current dynamic threshold Compare:

[0183] like If the confidence level of the filled value is satisfactory, it is retained and the prediction of the next outlier continues. In the 15-point filling process of this embodiment, the first 11 points and the last 2 points all passed the verification on the first attempt. The initial predicted value of the 12th filled point. =0.571, verification failed, triggering local fine-tuning.

[0184] The fine-tuning process is as follows: Calculate the local standard deviation σ of the adjacent clean area. local =1.24 (calculated from 48 points behind the left clean area), take the scaling factor λ=0.5, and generate the fine-tuning radius:

[0185] δ=λ⋅ ⋅σ local =0.5×0.589×1.24≈0.365;

[0186] Generate a candidate set centered on the initial predicted value. , , , , Substitute each candidate value into the sequence to construct a feature vector and calculate the EIF anomaly score. The results are shown in Table 1.

[0187] Table 1 Abnormal Score Table

[0188] The candidate value -0.183 corresponds to the lowest anomaly score (0.552 < 0.571), and it is selected as the final fill value. This mechanism effectively suppresses the drift trend of iterative prediction in the depth of continuous anomaly segments, ensuring that the fill sequence is consistent with the global normal pattern.

[0189] After the continuous abnormal segment is filled, immediately release the temporarily trained forward and backward LSTM models and free up memory.

[0190] After all 42 abnormal segments have been compensated, the original values, denoised values, abnormal labels, abnormal scores, filled values, training data window boundaries (including isolation zone positions), model confidence, and computation time are generated into structured JSON logs, written to the local database of the on-site edge computing gateway, and synchronized to the cloud monitoring platform according to a preset cycle.

[0191] Example 2

[0192] This embodiment is a system implementation method corresponding to the method in Embodiment 1.

[0193] An industrial-grade gateway device with edge computing capabilities is deployed within the slope monitoring station. This gateway integrates an ARM Cortex-A78AE processor and an embedded GPU, and carries the software module corresponding to the method of this invention. The fixed inclinometer at the monitoring site transmits the real-time acquired tilt angle data to this gateway at 10-minute intervals via an RS-485 bus.

[0194] like Figure 7 The diagram shown is a system interaction interface diagram of the software module corresponding to this embodiment of the invention. The interface title is "Rock and Soil Monitoring Signal Fidelity Denoising and Adaptive Repair System". The interface has tabs for data generation, full-process processing, analysis and visualization, and system information. Currently, the user is on the full-process processing tab. The full-process processing page is divided into three parameter configuration areas according to a three-level linkage mechanism. Step one is the VMD denoising parameter configuration area, where the minimum value of K, the maximum value of K, the minimum value of α, and the maximum value of α can be configured. Step two is the EIF anomaly detection parameter configuration area, where the number of trees and the subsampling size can be set. And the threshold percentile; Step 3 is the JIT-LSTM repair parameter configuration area, where you can configure the input step size L, safety isolation band M, and maximum number of training rounds; At the bottom of the interface, there is a button to run the full process, which is used to trigger the execution of the complete algorithm process of VMD noise reduction, EIF anomaly detection, and JIT-LSTM repair.

[0195] The software within the gateway processes the data stream online according to the complete process of steps S1 to S3 described in Example 1:

[0196] (1) Upon receiving a new data point, append it to the end of a circular buffer with a length of 90 days (approximately 12,960 sampling points);

[0197] (2) Perform VMD noise reduction on the latest incremental segment in the buffer. The VMD parameters K=6 and α=2000 have been locked and fixed during deployment according to the calibration process.

[0198] (3) Construct feature vectors for the denoised incremental data, call the trained EIF model to calculate the anomaly score, and base the result on the real-time updated noise intensity curve. Dynamically adjust threshold Determine whether the current point is abnormal;

[0199] (4) If it is determined to be normal, the noise reduction value is directly output; if it is determined to be abnormal, JIT-LSTM local modeling and compensation filling is triggered, the repair value is output, and the abnormal event is written to the alarm log.

[0200] (5) The processed data stream is uploaded to the cloud monitoring platform via 4G / 5G network. The platform's web interface displays a real-time overlay view of the original / noise reduction / repair curves. Abnormal points are highlighted in red, and manual one-click confirmation or undoing of repair is supported.

[0201] During a 48-hour period of heavy rainfall, the deep displacement monitoring sequence of this slope experienced continuous data packet loss, resulting in a missing segment of 18 points. The system of this invention automatically detected the anomaly and triggered bidirectional LSTM weighted fusion filling. Combined with the EIF confidence correction mechanism, it completed the compensation calculation for the entire missing segment and output the repaired sequence within approximately 2.3 seconds. During this process, after receiving an alarm message on the cloud platform, the on-duty engineer remotely reviewed the confidence comparison between the original anomaly score curve and the candidate filling values, confirming that the repair result was reasonable and required no manual intervention. The entire event handling process was fully recorded in a structured log, providing continuous and reliable data support for the subsequent landslide early warning model.

[0202] In summary, by utilizing the technical solution described above, the denoising process is locked within a reliable range that retains physical precursor characteristics, with a rigid baseline of pulse peak retention rate, thanks to the establishment of physical independence constraints. This fundamentally solves the problem of subtle structural anomalies being obscured by algorithmic smoothing under strong interference. Furthermore, the introduction of real engineering strong interference event waveforms as calibration pulse sources enhances the engineering fit of the constraints. Through the established dynamic threshold mapping mechanism based on noise residual feedback, an adaptive dynamic balance between denoising residuals and detection sensitivity is achieved. This closed-loop mechanism enables the model to automatically suppress false alarms caused by noise disturbances in real soil and rock environments with drastic noise intensity fluctuations, while restoring sensitivity to weak anomalies when noise decreases. The further introduced frequency band separation feedforward adjustment... The coefficients allow the dynamic threshold response intensity to automatically adapt to the VMD decomposition quality, enhancing the algorithm's adaptability under different signal characteristics. Through a locally-trainable LSTM repair strategy, combined with a safety isolation zone and a bidirectional fusion mechanism, it can specifically capture and repair the unique time-varying deformation patterns in the vicinity of anomaly points. Compared to the global model, this strategy naturally adapts to the non-stationary dynamic characteristics of soil deformation patterns due to changes in construction conditions and environmental factors, achieving higher precision in missing value compensation. Furthermore, the introduced EIF fractional feedback correction mechanism verifies the quality of the filled values ​​in real time and automatically fine-tunes them during the iterative filling process, significantly suppressing the cumulative error drift in filling continuous missing segments.

[0203] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

Claims

1. A method for fidelity-preserving noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions, characterized in that, include: S1. Acquire the soil and rock monitoring signal to be processed, calibrate and lock the variational mode decomposition parameters with physical fidelity index as constraint, perform variational mode decomposition on the soil and rock monitoring signal, remove the noise-dominant mode, and reconstruct and generate a noise-reduced base sequence. S2. Based on the denoised base sequence and the noise-dominant mode, a multi-dimensional feature matrix is ​​constructed, and an extended isolated forest model is used to detect anomalies in the denoised base sequence and output anomaly labeling sequences. S3. Based on the anomaly marker sequence, identify continuous abnormal segments in the denoised base sequence, train the time series prediction model in situ using the normal data adjacent to the abnormal segments, and perform point-by-point or segment-by-segment prediction and filling of the data within the abnormal segments, outputting the repaired complete sequence. S4. Re-execute variational mode decomposition and noise reduction on the complete sequence, and quantitatively evaluate the repair quality of the original filled segment using physical fidelity index. Based on the comparison between the evaluation results and the preset qualified conditions, determine whether to accept the current repair result or trigger the upgrade repair strategy, and output the final repair sequence.

2. The method for fidelity-preserving noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions as described in claim 1, characterized in that, The process of acquiring the soil and rock monitoring signal to be processed, calibrating and locking the variational mode decomposition parameters under the constraint of physical fidelity index, performing variational mode decomposition on the soil and rock monitoring signal, removing noise-dominant modes, and reconstructing and generating a noise-reduced base sequence includes: S11. Collect historical monitoring data of the engineering site under stable working conditions as a calibration dataset; S12. Set a calibration pulse sequence in the calibration dataset; the calibration pulse sequence is an implanted simulated pulse sequence and an actual pulse sequence extracted from the waveforms of known strong interference events at the engineering site; S13. Set the search range for each variational mode decomposition parameter to form multiple parameter combinations, and perform variational mode decomposition on each parameter combination to obtain the corresponding number of intrinsic mode components; the variational mode decomposition parameters include the number of modes and the penalty factor; S14. Sort the intrinsic mode components from high to low center frequency, and according to the value of the number of modes, remove a fixed number of noise-dominant modes with the highest center frequency. Then, superimpose and reconstruct the remaining intrinsic mode components to obtain the noise reduction sequence. S15. For each set of parameters, calculate the physical fidelity index of the denoised sequence; the physical fidelity index includes the signal-to-noise ratio improvement and the pulse peak energy retention rate. S16. Using the signal-to-noise ratio improvement greater than or equal to the first threshold and the pulse peak energy retention rate greater than or equal to the second threshold as dual constraints, filter the parameter combinations that satisfy the dual constraints, and select the optimal locking parameters, including the number of locked modes and the penalty factor. S17. Variational mode decomposition is performed on the geotechnical monitoring signal to be processed using the locked mode number and penalty factor, and reconstructed according to the same noise-dominant mode elimination rule to obtain the noise-reduced base sequence.

3. The method for fidelity-preserving noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions as described in claim 2, characterized in that, The method uses a dual constraint condition, where the signal-to-noise ratio improvement is greater than or equal to a first threshold and the pulse peak energy retention rate is greater than or equal to a second threshold, to screen parameter combinations that satisfy the dual constraint condition and select the optimal locking parameters, including: S161. If multiple sets of parameter combinations simultaneously satisfy the dual constraint conditions, the one with the largest increase in signal-to-noise ratio shall be selected as the locked parameter. S162. If there is no parameter combination that satisfies all conditions, the second threshold of the pulse peak energy retention rate is lowered and the parameters are re-selected, and the combination with the highest pulse peak energy retention rate is selected as the locking parameter. If there is still no parameter combination that satisfies the conditions after lowering the second threshold, the combination with the highest pulse peak energy retention rate among all parameter combinations is selected as the locking parameter.

4. The method for fidelity-preserving noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions as described in claim 1, characterized in that, The method involves constructing a multidimensional feature matrix based on the denoised base sequence and the noise-dominant mode, and then using an extended isolated forest model to detect anomalies in the denoised base sequence. The output anomaly marker sequence includes: S21. Extract the noise-dominant modes that were removed in the variational mode decomposition, calculate the root mean square value of the sliding window, form the original noise intensity sequence, and normalize the original noise intensity sequence to obtain the noise intensity curve. S22. Apply a sliding window to the noise reduction base sequence, extract the temporal statistical features and differential features under each window, and incorporate the noise intensity curve as a one-dimensional additional feature into the temporal statistical features and differential features to form a complete multi-dimensional feature matrix. S23. Using the calibration dataset, after selecting the locking parameters, execute steps S21 and S22 to obtain the corresponding calibration feature matrix, train the extended isolated forest model, and calculate the specified percentile of the abnormal scores of all normal samples as the base threshold. S24. Based on the local mean of the noise intensity curves within the local window before and after the current time, establish a dynamic threshold mapping function; when the local noise intensity increases, the dynamic threshold automatically increases, and when the local noise intensity decreases, the dynamic threshold automatically converges to the base threshold. S25. For the feature vector of the time to be detected, the original anomaly score is calculated using the trained extended isolated forest model; the original anomaly score is compared with the dynamic threshold to calculate the anomaly deviation ratio, and the current time is marked as an anomaly according to the value range of the anomaly deviation ratio. S26. For time periods with abnormal markers, the time-frequency characteristics of each modality and the fluctuation pattern of the noise intensity curve are used to perform similarity matching with various interference features in the pre-constructed interference source feature library to determine the type of interference source, and to assign differentiated repair processing strategies according to the type of interference source.

5. The method for fidelity-preserving noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions as described in claim 4, characterized in that, The interference source feature library includes blasting impact type, mechanical vibration type, electromagnetic interference type, rain noise type, composite interference type and unknown interference type.

6. The method for fidelity-preserving noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions as described in claim 1, characterized in that, The method involves identifying consecutive anomalous segments in the denoised base sequence based on anomaly marker sequences, training a time-series prediction model in situ using normal data adjacent to the anomalous segments, and performing point-by-point or segment-by-segment prediction and filling of the data within the anomalous segments. The output of the repaired complete sequence includes: S31. Merge consecutive abnormal points into one or more abnormal segments according to the abnormal marker sequence, and set a preset number of sampling points before and after each abnormal segment as a physical safety isolation zone. S32. Set a normal data region that can be used for training and compare it with a preset minimum length threshold required for training. Select a prediction method based on the comparison result. The prediction method includes forward long short-term memory network, reverse long short-term memory network, or bidirectional weighted fusion. S33. Based on the selected prediction method, train a time series prediction model using normal data; the time series prediction model includes an input layer, a long short-term memory hidden layer, and a fully connected output layer. S34. Using the trained time series prediction model, perform single-step prediction and fill in each abnormal point in the abnormal segment. For continuous abnormal segments, adopt the iterative single-step prediction method and include the current prediction value into the subsequent input window to sequentially scroll and fill the entire segment. S35. Once the abnormal segment is filled, immediately release the temporarily trained time-series prediction model and record the model information during the processing as a structured log; the model information includes the original value, the value after noise reduction, the abnormal label, the abnormal score, the filled value, and the model confidence.

7. The method for fidelity-preserving noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions as described in claim 6, characterized in that, The process involves using a trained time-series prediction model to perform single-step prediction and fill in each anomaly point within an anomaly segment. For continuous anomaly segments, an iterative single-step prediction method is used, incorporating the current predicted value into subsequent input windows to sequentially fill the entire segment. S321. Take the continuous normal sequence after removing the isolation band before the start of the abnormal segment as the left data source, and take the continuous normal sequence after removing the isolation band after the end of the abnormal segment as the right data source. S322. Based on the input step size of the time series prediction model, set the minimum length threshold required for training; S323. Calculate the lengths of the left and right data sources respectively, and compare each length with the minimum length threshold. S324. Select the prediction method based on the comparison results. If only the length of the left data source meets the requirements, select the forward long short-term memory network. If only the length of the right data source meets the requirements and the current mode is offline batch processing, select the reverse long short-term memory network. If the lengths of both the left and right data sources meet the requirements, select the bidirectional weighted fusion method.

8. The method for fidelity-preserving noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions as described in claim 7, characterized in that, The process involves using a trained time-series prediction model to perform single-step prediction and fill in each anomaly point within an anomaly segment. For continuous anomaly segments, an iterative single-step prediction method is used, incorporating the current predicted value into subsequent input windows to sequentially fill the entire segment. S341. For a single point anomaly, take the number of consecutive clean points corresponding to the input step size to the left of the current anomaly point as the input sequence, input them into the time series prediction model to obtain the predicted value, and then fill them into the current anomaly point. S342. For continuous outlier segments, starting from the first point of the outlier segment, if a forward long short-term memory network or bidirectional weighted fusion is used, the number of data points corresponding to the input step size to the left of the current outlier point is taken as the input sequence; if a reverse long short-term memory network is used, the number of data points corresponding to the input step size to the right of the current outlier point is taken and the input sequence is constructed in reverse order of time. After the time series prediction model outputs the predicted value, it is filled into the current outlier point; the predicted value is included in the input window to predict the next outlier point, and this process is iterated until the entire outlier segment is completely filled. S343. After predicting the fill value of the current outlier in each iteration, the fill value is temporarily put back into the sequence. The extended isolated forest outlier score at the corresponding time of the current outlier is calculated. If the outlier score is lower than the adaptive detection threshold at the current time, the fill value is retained and the prediction of the next outlier continues. If the outlier score is greater than or equal to the adaptive detection threshold at the current time, the fill value is locally fine-tuned and then filled in.

9. The method for fidelity-preserving noise reduction and adaptive repair of geotechnical monitoring signals under strong interference conditions according to claim 1, characterized in that, The process involves re-performing variational mode decomposition and noise reduction on the complete sequence, using physical fidelity metrics to quantitatively evaluate the repair quality of the original filled segments, and comparing the evaluation results with preset acceptable conditions to determine whether to accept the current repair result or trigger an upgraded repair strategy. The final repair sequence output includes: S41. Using the complete sequence as input, perform variational mode decomposition using the same locked set of mode numbers and penalty factors, and reconstruct it according to the same elimination rules to obtain the secondary denoised sequence. S42. For the complete sequence, identify the time domain interval corresponding to the abnormal segment of the time series prediction model, take the local peak of the signal in the time domain interval as the evaluation object, and calculate the pulse peak energy retention rate of the secondary noise reduction sequence as the restoration fidelity index. S43. When the repair fidelity index is greater than or equal to the preset qualified threshold, the repair quality is deemed qualified and the current filling result is accepted; when the repair fidelity index is lower than the preset qualified threshold, different levels of response strategies are triggered according to the degree of deviation of the repair fidelity index. S44. Record the repair fidelity index value, response level, number of loops, and final processing result of each closed-loop verification as a structured log.

10. A system for fidelity denoising and adaptive repair of geotechnical monitoring signals under strong interference conditions, used to implement the method for fidelity denoising and adaptive repair of geotechnical monitoring signals under strong interference conditions as described in any one of claims 1-9, characterized in that, The system includes: The physical constraint noise reduction module is used to acquire the soil and rock monitoring signal to be processed, calibrate and lock the variational mode decomposition parameters with physical fidelity index as constraint, and perform variational mode decomposition on the soil and rock monitoring signal to remove the noise-dominant mode and reconstruct and generate a noise-reduced base sequence. The anomaly detection and classification module is used to construct a multi-dimensional feature matrix based on the denoised base sequence and the noise-dominant mode, and to use an extended isolated forest model to detect anomalies in the denoised base sequence and output anomaly label sequences. The ready-to-use and trainable repair module is used to identify continuous abnormal segments in a noise-reduced base sequence based on anomaly-marked sequences, train a time series prediction model in situ using normal data adjacent to the abnormal segments, and perform point-by-point or segment-by-segment prediction and filling of the data within the abnormal segments, outputting the repaired complete sequence. The closed-loop verification feedback module is used to re-perform variational mode decomposition and noise reduction on the complete sequence, quantitatively evaluate the repair quality of the original filled segment using physical fidelity indicators, and determine whether to accept the current repair result or trigger an upgraded repair strategy based on the comparison of the evaluation results with the preset qualified conditions, and output the final repair sequence.

Images