Slope monitoring and early warning transfer interference judgment method based on time sequence spike anomaly detection

Abstract

The application belongs to the technical field of slope monitoring, and relates to a slope monitoring early warning conversion interference judgment method based on time sequence spike anomaly detection, which comprises the following steps: obtaining sensing data sequence of a target slope, checking time sequence data continuity and completing time gap data, and then performing legality verification; performing standardization pretreatment on the data sequence that passes the verification, and then constructing time sequence samples through a sliding window; inputting the time sequence samples into a self-encoder network model, reconstructing sample data through model decoding, and calculating reconstruction error; obtaining an abnormal score of the sample data according to the reconstruction error, and determining spike anomaly; filtering the spike anomaly data through a preset time length, and judging whether to convert the slope monitoring early warning into an interference event according to the proportion of the filtered spike anomaly data quantity to the total monitoring data point quantity in the preset time length. The application can accurately capture data spike type anomaly and improve the accuracy of early warning conversion into an interference event.

Description

Technical Field

[0001] This invention belongs to the field of slope monitoring technology and relates to a method for judging the transformation of slope monitoring early warning into interference based on time-series spike anomaly detection. Background Technology

[0002] In slope monitoring, screening for anomalous data in time-series data is a crucial and cannot be overlooked. Spike anomalies, one of the most common types of anomalies in time-series data, manifest as data points suddenly deviating from the normal fluctuation range within a short period of time, followed by rapid recovery. Furthermore, this type of anomaly often corresponds to critical events such as equipment failure or abnormal equipment environment, making its accurate detection highly valuable for early warning and interference event analysis.

[0003] Existing methods for detecting time series anomalies mainly consist of statistical methods and traditional machine learning methods. Statistical methods, such as the 3σ principle and the quartile method, rely on assumptions about data distribution and are poorly adapted to non-stationary time series data. Traditional machine learning methods, such as Isolation Forest and KNN, struggle to capture the temporal dependence features of time series data. Summary of the Invention

[0004] To address the aforementioned technical problems in existing technologies, this invention proposes a method for judging the transition from early warning to interference in slope monitoring based on time-series spike anomaly detection. This method can accurately capture spike-type anomalies in data and improve the accuracy of early warning transitioning to interference events. The specific technical solution is as follows:

[0005] A method for judging the transition from slope monitoring and early warning to interference based on time-series spike anomaly detection includes:

[0006] Acquire the sensor data sequence obtained from the target slope monitoring, check the continuity of the time series data and fill in the time gap data, and then perform legality verification.

[0007] The data sequences that pass the verification are standardized and preprocessed, and then time series samples are constructed using a sliding window.

[0008] The time-series samples are input into the autoencoder network model, the sample data is reconstructed through model decoding, and the reconstruction error is calculated.

[0009] The anomaly score of the sample data is obtained based on the reconstruction error, and a dual-condition judgment of spike anomalies is made based on the scores of adjacent points.

[0010] The slope monitoring and early warning system filters out spike anomaly data by setting a preset time period, and determines whether to convert the slope monitoring and early warning into an interference event based on the proportion of the amount of spike anomaly data after filtering to the total number of monitoring data points within the preset time period.

[0011] Furthermore, the sensing data sequence includes a time series time_list and a numerical sequence baseline_list;

[0012] The legality verification includes:

[0013] Length check: len(time_list) = len(baseline_list);

[0014] Data volume verification: len(baseline_list)≥seq_len+1, where seq_len represents the length of the time series window;

[0015] If the above verification conditions are met, the verification passes; otherwise, an exception is thrown.

[0016] Furthermore, the data sequences that pass the verification are preprocessed using the StandardScaler.

[0017] Furthermore, the autoencoder network model includes an encoder and a decoder, wherein the encoder is used to map the input sample data into low-dimensional hidden layer features, and the decoder is used to reconstruct the original input from the hidden layer features.

[0018] Furthermore, during the training of the autoencoder network model, the mean squared error (MSE) is used as the loss function, and the Adam optimizer is used to iteratively train for epochs to update the model network parameters.

[0019] Furthermore, the step of obtaining the anomaly score of the sample data based on the reconstruction error specifically involves:

[0020] The squared error is calculated for each data point within each window, expressed as follows:

[0021] ;

[0022] Among them, This represents the j-th data point in the k-th window sample. For the corresponding reconstructed value;

[0023] The anomaly score for each data point is obtained by averaging the errors of all windows covering the same data point, expressed as: ;

[0024] Where step_error[s][is] is the squared error of s windows covering the i-th point; count[i] is the number of times the i-th data point is covered by the window, and count[i]≥1.

[0025] Furthermore, the spike anomaly is determined using a dual criteria based on the magnitude of the reconstruction error and the ratio of the errors of adjacent points, and must simultaneously meet the following two conditions:

[0026] raw_error[i]>threshold;

[0027] raw_error[i]>3*(raw_error[i-1]+raw_error[i+1]) / 2;

[0028] Here, threshold is the anomaly detection threshold, threshold=Quantile(raw_error,0.99), which means taking the 99th percentile of all raw_error values. This is used to identify spike-like patterns that are sudden changes and rapid recovery, distinguishing them from the gradual trend of a true landslide.

[0029] Furthermore, the filtering of spike abnormal data by preset duration is specifically as follows: the preset duration is Q hours, the time boundary is obtained by subtracting Q hours from the current time, and only abnormal data points with timestamps greater than or equal to the time boundary are retained.

[0030] Furthermore, when the filtered spikes abnormally meet the following two criteria, it is determined to be a transient sensor interference, and the slope warning is automatically converted into an interference event: Level 1: When the proportion of data volume to the total number of monitoring data points within the preset time period is lower than the preset threshold, the slope monitoring early warning will be converted into an interference event; Level 2: The anomaly shows no continuous increasing trend and no temporal clustering.

[0031] The beneficial effects of the present invention include at least the following:

[0032] (1) Unsupervised learning is adopted, eliminating the need for manual data labeling;

[0033] (2) Using window-weighted average error makes anomaly location more accurate;

[0034] (3) A dual-judgment rule is adopted to further reduce the false alarm rate;

[0035] (4) The model is lightweight and has a fast training and inference speed, and can support real-time detection. Attached Figure Description

[0036] Figure 1 This is a flowchart of a slope monitoring and early warning method for judging interference based on time-series spike anomaly detection, as described in this embodiment. Detailed Implementation

[0037] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0038] like Figure 1As shown in the figure, this embodiment of the invention provides a method for judging the transition from slope monitoring and early warning to interference based on time-series spike anomaly detection, including the following steps:

[0039] Step 1: Initialize the detector and configure the parameters.

[0040] Specifically, the detector is equipped with a standardizer (StandardScaler) and an autoencoder network model.

[0041] Initialize the detector and set the core hyperparameters: temporal window length seq_len, hidden layer dimension hidden_dim, latent space dimension latent_dim, training epochs, and learning rate lr.

[0042] The hyperparameter set Θ can be represented as:

[0043] Θ={seq_len, hidden_dim, latent_dim, epochs, lr}.

[0044] The autoencoder network model includes an encoder and a decoder. The encoder maps the input X to low-dimensional hidden layer features, and the decoder reconstructs the original input from the hidden layer features, as follows:

[0045] Encoder forward propagation:

[0046] ,

[0047] Decoder forward propagation:

[0048] ,

[0049] in, and is the output vector of the hidden layer of the model; z is the low-dimensional hidden layer feature vector output by the encoder; This is the reconstructed vector output by the decoder; This is the weight matrix for each layer of the model; These are the bias vectors for each layer of the model.

[0050] Step 2: Obtain the sensor data sequence monitored at key locations of the target slope, such as surface displacement, surface tilt, surface tilt acceleration, crack monitoring, surface water, etc., check the data continuity and fill in the time gaps, and then perform legality verification.

[0051] Specifically, after acquiring the sensor data sequence, the continuity of the time series data is automatically checked: empty data or data with time gaps are identified, and the LSTM model is used to predict missing or gapped data for data completion. The LSTM model is trained based on historical sensor data. The completed sensor data sequence includes a time series (time_list) and a numerical sequence (baseline_list). The time series (time_list) and the numerical sequence (baseline_list) are then validated.

[0052] Length check: len(time_list) = len(baseline_list);

[0053] Data volume check: len(baseline_list) ≥ seq_len + 1;

[0054] If the above verification conditions are met, the verification is successful and the next step can be performed; otherwise, an exception is thrown and the process is interrupted.

[0055] Step 3: Perform standardization preprocessing on the data that passes the verification.

[0056] Specifically, the valid time-series data is standardized using the StandardScaler to eliminate the influence of units, as shown in the following expression:

[0057] ,

[0058] in: Let x be the i-th data point in the data sequence. i The standardized value;

[0059] , representing the mean of the sequence;

[0060] , representing the standard deviation of the sequence;

[0061] N represents the total number of time series data points.

[0062] Step 4: Construct time series samples from standardized time series data using a sliding window.

[0063] Specifically, a window of length seq_len slides across the standardized time series data to construct time series samples. The expression for each time series sample is as follows:

[0064]

[0065] Where: x k This represents the value of the k-th data point in the standardized time series data;

[0066] Total sample size M: M = N - seq_len + 1;

[0067] The data vector of each window is flattened into a one-dimensional vector and used as the input to the autoencoder network model.

[0068] Step 5: Train the autoencoder network model.

[0069] Specifically, the mean squared error (MSE) is used as the loss function, and the Adam optimizer is used to train the autoencoder model.

[0070] The loss function is expressed as follows:

[0071] ,

[0072] in: This represents the j-th input data in the k-th window sample;

[0073] The Adam optimizer is used to iterate through training epochs and update the model network parameters.

[0074] Step 6: Obtain the error of window-level model decoding and reconstruction, calculate the weighted average of single-point reconstruction error, and use it as the final anomaly score for each data point.

[0075] Specifically, calculate the squared error for each point within each window:

[0076] ,

[0077] The final anomaly score for each data point is obtained by weighted averaging the errors of all windows covering the same data point. The expression for averaging the errors of all windows covering the same data point is:

[0078]

[0079] Where step_error[s][is] is the squared error of the s windows covering the i-th point; S i This represents the set of all windows containing point i.

[0080] count[i] is the number of times the i-th data point is covered by the window, and count[i]≥1.

[0081] Step 7: Adaptive anomaly threshold calculation.

[0082] Specifically, the 99th percentile of the scores for all individual anomalies is calculated and used as the anomaly detection threshold:

[0083] threshold=Quantile(raw_error,0.99) means taking the 99th percentile of all raw_error values.

[0084] Step 8: Set dual conditions to determine data point spike anomalies.

[0085] Specifically, a dual-condition judgment is made based on the anomaly score and the scores of adjacent points for each data point to mark spike anomalies.

[0086] The determination of the spike anomaly point requires that the following two conditions be met simultaneously:

[0087] raw_error[i]>threshold;

[0088] raw_error[i]>3*(raw_error[i-1]+raw_error[i+1]) / 2;

[0089] When both of the above conditions are met, mark is_spike[i]=True, indicating that the i-th point is judged as a spike anomaly.

[0090] Step 9: Filter out anomalies according to the preset time to obtain standardized detection results.

[0091] Specifically, the preset duration is Q hours. The time boundary is obtained by subtracting Q hours from the current time. Only abnormal data points with timestamps greater than or equal to this boundary are retained, that is, only abnormal data points within the most recent Q hours are retained. This results in the output of standardized detection results, including: abnormal threshold, number of abnormalities in the last Q hours, abnormal details list, and total number of data points in the last Q hours.

[0092] For example: Set the time boundary to the last 24 hours: last_24h = current_time - 24h, where 24h is the preset duration; only retain outliers with a time greater than or equal to last_24h, and filter out redundant historical data.

[0093] Where: current_time represents the current time, for example: 10:00 on April 3, 2026;

[0094] Calculate the time 24 hours prior to the current time: 10:00 AM on April 2, 2026, and set this time as the time boundary;

[0095] If outlier A occurs at 15:00 on April 2, 2026, which is later than the time boundary 10:00 on April 2, 2026, i.e., within the last 24 hours, then outlier A will be retained.

[0096] If anomaly A occurs at 9:00 on April 2, 2026, which is earlier than the time boundary 10:00 on April 2, 2026, meaning it did not occur within the last 24 hours, then anomaly A will be discarded.

[0097] Step 10: Based on the standardized test results, determine whether the slope monitoring and early warning has become an interference.

[0098] Specifically, based on a comparison of the percentage of anomalies within the past 24 hours with a preset threshold, a decision is made on whether to convert the slope monitoring warning into a disturbance event. When the percentage of anomalies falls below the threshold, the warning is converted into a disturbance event.

[0099] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for judging interference in slope monitoring and early warning based on time-series spike anomaly detection, characterized in that, include: Acquire the sensor data sequence obtained from the target slope monitoring, check the continuity of the time series data and fill in the time gap data, and then perform legality verification. The data sequences that pass the verification are standardized and preprocessed, and then time series samples are constructed using a sliding window. The time-series samples are input into the autoencoder network model, the sample data is reconstructed through model decoding, and the reconstruction error is calculated. The anomaly score of the sample data is obtained based on the reconstruction error, and a dual-condition judgment of spike anomalies is made based on the scores of adjacent points. The slope monitoring and early warning system filters out spike anomaly data by setting a preset time period, and determines whether to convert the slope monitoring and early warning into an interference event based on the proportion of the amount of spike anomaly data after filtering to the total number of monitoring data points within the preset time period.

2. The slope monitoring and early warning interference judgment method as described in claim 1, characterized in that, The process of checking the continuity of time-series data and filling in time gaps specifically involves: identifying empty data or time gaps, using an LSTM model to predict missing or gapped data, and then filling in the data; the LSTM model is trained based on historical sensor data.

3. The slope monitoring and early warning interference judgment method as described in claim 1, characterized in that, The sensor data sequence includes a time series time_list and a numerical sequence baseline_list; The legality verification includes: Length check: len(time_list) = len(baseline_list); Data volume verification: len(baseline_list)≥seq_len+1, where seq_len represents the length of the time series window; When the conditions for length verification and data volume verification are met, the verification is considered successful; otherwise, an exception is thrown.

4. The slope monitoring and early warning interference judgment method as described in claim 1, characterized in that, The data sequences that pass the verification are preprocessed using the StandardScaler.

5. The slope monitoring and early warning interference judgment method as described in claim 1, characterized in that, The autoencoder network model includes an encoder and a decoder. The encoder is used to map the input sample data into low-dimensional hidden layer features, and the decoder is used to reconstruct the original input from the hidden layer features.

6. The slope monitoring and early warning interference judgment method as described in claim 1, characterized in that, During training, the autoencoder network model uses mean squared error (MSE) as the loss function and employs the Adam optimizer to iteratively train for epochs, updating the model network parameters.

7. The slope monitoring and early warning interference judgment method as described in claim 1, characterized in that, The process of obtaining anomaly scores for sample data based on reconstruction errors is as follows: The squared error is calculated for each data point within each window, expressed as follows: ； Among them, This represents the j-th data point in the k-th window sample. The corresponding reconstructed value; The average error of all windows covering the same data point is used to obtain the anomaly score for each data point, expressed as: ; Where step_error[s][is] is the squared error of s windows covering the i-th point; count[i] is the number of times the i-th data point is covered by the window, and count[i]≥1.

8. The slope monitoring and early warning interference judgment method as described in claim 7, characterized in that, The spike anomaly is determined using a dual criteria based on the magnitude of the reconstruction error and the ratio of the errors of adjacent points, and must simultaneously meet the following two conditions: raw_error[i]>threshold; raw_error[i]>3*(raw_error[i-1]+raw_error[i+1]) / 2; Where threshold is the anomaly detection threshold, threshold=Quantile(raw_error,0.99), which means taking the 99th percentile of all raw_errors.

9. The slope monitoring and early warning interference judgment method as described in claim 1, characterized in that, The filtering of spike abnormal data by preset duration is specifically as follows: the preset duration is Q hours, the time boundary is obtained by subtracting Q hours from the current time, and only abnormal data points with timestamps greater than or equal to the time boundary are retained.

10. The slope monitoring and early warning interference judgment method as described in claim 1, characterized in that, When the abnormal spikes after filtering meet the following two criteria, it is determined to be transient interference from the sensor, and the slope warning is automatically converted into an interference event: Level 1: When the proportion of spike abnormal data to the total number of monitoring data points within a preset time period is lower than the preset threshold, the slope monitoring early warning will be converted into an interference event. Level 2: After filtering, the spikes show no continuous increasing trend and no time clustering.

Images