Slope displacement monitoring method and system based on reinforcement learning enhanced kalman filter

Abstract

The application provides a kind of based on reinforcement learning enhanced Kalman filtering slope displacement monitoring method and system, method includes: the displacement data of be collected to Beidou is preprocessed, executes abnormal value elimination, missing value interpolation and time consistency check;Establish the Kalman filtering model including displacement and speed state vector, initialize process noise covariance matrix Q and observation noise covariance matrix R as initial filtering parameter;By PPO reinforcement learning algorithm dynamic optimization Q, R matrix, realize parameter self-adapting adjustment;Displacement trend analysis is carried out to filtering output data, and the abnormal trend data is marked using statistical test and trend inflection point identification algorithm, and its root mean square error is fed back to PPO algorithm for parameter readjustment;Based on sliding window technology analysis data phase change, set independent experience playback buffer area for different stage data in PPO, realize the associated update of filtering parameter.The application improves the precision and reliability of slope displacement monitoring.

Description

Technical Field

[0001] This invention relates to the field of engineering testing technology, and in particular to a slope displacement monitoring method and system based on reinforcement learning-enhanced Kalman filtering. Background Technology

[0002] With the continuous expansion of infrastructure construction in my country, including the widespread construction of railways, highways, and water conservancy projects, slope safety issues have become increasingly prominent. Slope displacement monitoring is a crucial means of preventing geological disasters and ensuring traffic safety. In recent years, satellite navigation technology, represented by the BeiDou Navigation Satellite System (BDS), has developed rapidly, and displacement monitoring technology based on the BDS system has become one of the mainstream methods for slope monitoring.

[0003] Currently, conventional slope displacement monitoring methods based on the BeiDou satellite navigation system mainly employ the traditional Kalman filter to filter the displacement data collected by BeiDou monitoring stations. The traditional Kalman filter is an efficient linear optimal estimation algorithm, performing excellently in addressing position and velocity state estimation problems in linear systems. However, for slope displacement monitoring in complex geological environments, displacement changes typically exhibit nonlinear and non-stationary characteristics, leading to significant limitations in the practical application of the traditional Kalman filter.

[0004] Specifically, in traditional Kalman filtering algorithms, the process noise covariance matrix Q and the observation noise covariance matrix R are generally determined by manual experience or fixed values. This static setting is difficult to adapt to the dynamic changes in noise characteristics in the actual monitoring environment. When environmental factors (such as temperature and rainfall) change during slope displacement monitoring, the noise characteristics of the observed data also change. The fixed Q and R matrices cannot dynamically adapt to this change, resulting in limited filtering effect on the displacement monitoring data and seriously affecting the accuracy of displacement monitoring.

[0005] To address these issues, some scholars have proposed using adaptive Kalman filtering algorithms, adjusting filter parameters through empirical formulas or heuristics. However, these methods lack theoretical support for parameter adjustment and still suffer from problems such as strong subjectivity, poor generalization, and insufficient real-time adaptability. In practical applications, when significant changes occur in the geological environment or sudden displacement anomalies occur, traditional adaptive methods struggle to quickly and effectively adjust filter parameters, thus reducing monitoring accuracy. Summary of the Invention

[0006] This invention provides a slope displacement monitoring method and system based on reinforcement learning-enhanced Kalman filtering, which addresses the shortcomings of existing technologies, enables dynamic and adaptive adjustment of filtering parameters, thereby improving the accuracy and reliability of slope displacement monitoring and reducing the risk of false alarms and missed alarms.

[0007] In a first aspect, the present invention provides a slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering, comprising: setting up monitoring points at predetermined locations on the slope to collect displacement data and performing data quality assessment; performing data preprocessing on the qualified displacement data, wherein the data preprocessing methods include outlier removal, missing value imputation, and time consistency check; establishing a Kalman filter model containing displacement and velocity state vectors based on the preprocessed displacement data, and initializing the process noise covariance matrix Q and the observation noise covariance matrix R as initial filtering parameters; adaptively optimizing the filtering parameters through the PPO algorithm; performing displacement trend analysis on the displacement data output by adaptive filtering using statistical tests and trend inflection point identification algorithms, marking abnormal trend data, and using their root mean square error index as feedback information for the PPO algorithm to adjust the filtering parameters; and implementing data segmentation processing based on sliding window technology and data stage change analysis, setting corresponding experience playback buffers for different stages of data in the PPO algorithm to perform correlation updates of filtering parameters.

[0008] The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by this invention includes the following steps for data quality assessment: determining the signal-to-noise ratio of the collected displacement data based on the effective signal power and background noise power of the monitoring data; determining the sampling completeness rate based on the effective sampling points and the total number of samples of the collected displacement data; and determining the displacement data as qualified data if the signal-to-noise ratio and sampling completeness rate of the displacement data meet the preset requirements.

[0009] The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by this invention includes the following steps for data preprocessing of qualified displacement data: removing outlier data based on the three sigma criterion; imputing missing values ​​based on trend prediction interpolation; and checking time consistency based on the difference in timestamps between adjacent data.

[0010] The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by this invention establishes a Kalman filter model containing displacement and velocity state vectors based on preprocessed displacement data, including:

[0011] Define the state equation as: X(k|k-1)=AX(k-1|k-1)+Bμ(k-1)+ω(k-1),

[0012] Define the observation equation: Z(k) = HX(k) + ν(k);

[0013] Where X(k) is the state vector, Z(k) is the observation vector, A is the state transition matrix, B is the control input matrix, H is the observation matrix, ω(k-1) is the process noise, and ν(k) is the observation noise; the state vector X(k) is defined as the displacement and velocity of the slope in the X, Y, and Z directions; the observation vector Z(k) is defined as the actual monitored displacement data in the X, Y, and Z directions.

[0014] The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by this invention initializes the process noise covariance matrix Q and the observation noise covariance matrix R, including: determining the initial value of the process noise covariance matrix Q according to the state change rate of the initial monitoring data; and determining the initial value of the observation noise covariance matrix R according to the equipment accuracy parameters.

[0015] The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by this invention optimizes the Q and R matrices in real time through a near-end strategy optimization (PPO) algorithm to adaptively optimize the filtering parameters. This includes: designing the PPO algorithm using an Actor-Critic network: the Actor network generates actions, the Critic network estimates the state value function, and the action space is defined as the real-time adjustment of the Kalman filter covariance matrices Q and R; training the Actor-Critic network using a preset training optimization strategy; the training optimization strategy includes an exponential decay strategy for adjusting the learning rate, a Dropout strategy to prevent overfitting, a gradient clipping strategy to avoid gradient explosion, and an experience replay strategy.

[0016] The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by this invention uses statistical testing and trend inflection point identification algorithms to perform displacement trend analysis on the displacement data output by adaptive filtering, including: testing the significance of the displacement data trend using the Mann-Kendall test method; and identifying trend inflection points using cumulative sum control chart analysis method.

[0017] The slope displacement monitoring method based on reinforcement learning and enhanced Kalman filtering provided by this invention uses analysis of variance to analyze the phased changes in data.

[0018] The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by this invention further includes: adjusting the network parameters of the PPO algorithm based on the root mean square error index when the optimized filtering parameters fail to pass the verification.

[0019] Secondly, the present invention also provides a slope displacement monitoring system based on reinforcement learning-enhanced Kalman filtering, comprising:

[0020] The data acquisition module is used to set up monitoring points at preset locations on the slope to collect displacement data and perform data quality assessment.

[0021] The data preprocessing module is used to preprocess the displacement data that has passed the evaluation. The data preprocessing methods include outlier removal, missing value imputation, and time consistency check.

[0022] The slope displacement dynamic model construction module is used to build a Kalman filter model containing displacement and velocity state vectors based on preprocessed displacement data, and initialize the process noise covariance matrix Q and the observation noise covariance matrix R as initial filtering parameters.

[0023] The PPO algorithm parameter optimization module is used to adaptively optimize filter parameters using the PPO algorithm.

[0024] The displacement trend analysis module is used to perform displacement trend analysis on the displacement data output by adaptive filtering using statistical tests and trend inflection point identification algorithms, and to mark abnormal trend data so that the root mean square error index can be used as feedback information for the PPO algorithm to adjust the filtering parameters.

[0025] The segmented processing and real-time update module is used to segment data based on sliding window technology and data phase change analysis. In the PPO algorithm, corresponding experience playback buffers are set for data at different stages to perform correlation updates of filtering parameters.

[0026] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering as described above.

[0027] The slope displacement monitoring method and system based on reinforcement learning-enhanced Kalman filtering provided by this invention have the following advantages compared with the prior art:

[0028] (1) Accurate state-space dynamic modeling improves prediction accuracy

[0029] A well-defined state-space model was adopted, and the state transition matrix, observation matrix, and initial covariance matrix were determined in detail. Sensitivity analysis of initial conditions was performed to ensure the stability and accuracy of the dynamic model. This step effectively reduced state estimation errors and provided accurate initial predictions for the Kalman filter algorithm.

[0030] (2) Introducing the PPO reinforcement learning algorithm to achieve adaptive optimization of filtering parameters significantly improves filtering performance.

[0031] By introducing the PPO reinforcement learning algorithm with an Actor-Critic structure, the process noise covariance matrix Q and the observation noise covariance matrix R in the Kalman filter are dynamically adjusted in real time to achieve adaptive optimization of the parameters. The careful design of the network structure, the exponential decay strategy of the learning rate, gradient pruning, entropy regularization, sensitivity analysis of the action space adjustment amplitude, and constraint strategies all significantly improve the stability and adaptability of the filtering algorithm, greatly reduce filtering errors, and significantly improve the accuracy and real-time performance of displacement monitoring data.

[0032] (3) Real-time online verification and cross-validation mechanism to improve algorithm robustness

[0033] An online verification method for real-time error checking (RMSE calculation) was designed, which realizes real-time monitoring and feedback of the dynamic adjustment effect of filtering algorithm parameters, effectively ensuring the accuracy and stability of filtering parameter adjustment.

[0034] (4) Significantly improves the accuracy of displacement trend identification and quickly identifies abnormal slope changes.

[0035] The displacement trend analysis module explicitly proposes the Mann-Kendall trend significance test and the CUSUM trend inflection point identification method, which effectively identifies abnormal trend changes in slope displacement data and promptly feeds them back to the filtering system for parameter optimization. This method significantly improves the sensitivity and response capability to abnormal slope changes.

[0036] (5) Data segmentation and adaptive sliding window mechanism effectively capture the characteristics of displacement changes at different stages.

[0037] An adaptive adjustment mechanism for the sliding window length was designed. Different displacement stages were identified using analysis of variance (ANOVA), and different data stage types (stable period, slow change period, and rapid change period) were automatically labeled. At the same time, adaptive correlation updates and smooth transitions between stage data and PPO filter parameters were realized, effectively improving the response speed and accuracy to slope state changes. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is one of the flowcharts of the slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by the present invention;

[0040] Figure 2This is the second flowchart of the slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by the present invention;

[0041] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] It should be noted that, in the description of the embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0044] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0045] The following is combined with Figures 1-3 This invention describes the slope displacement monitoring method and apparatus based on reinforcement learning-enhanced Kalman filtering provided in the embodiments of the present invention.

[0046] Figure 1 This is one of the flowcharts of the slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by the present invention. Figure 2 This is the second flowchart of the slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided by this invention. See below for reference. Figure 1 and Figure 2 This invention will now be described, and includes, but is not limited to, the following steps:

[0047] Step 101: Set up monitoring points at the predetermined locations on the slope to collect displacement data and perform data quality assessment.

[0048] This method first clarifies the layout, quantity, and spatial distribution principles of slope monitoring points to ensure the accuracy and representativeness of the monitoring data.

[0049] ① Monitoring point layout method: The first step in the layout of monitoring points is to conduct on-site geological surveys to determine sensitive areas based on geological conditions. These areas typically include the top of the slope, the center of the slope, and the toe of the slope, which are prone to deformation or danger. The distance between monitoring points is usually between 20m and 50m, depending on the length, height, and deformation characteristics of the slope. At least three monitoring points should be set up within the monitoring area to ensure the validity of the spatial data.

[0050] ② Principles for the number and distribution of monitoring points: The number of slope monitoring points is mainly determined based on the scale and deformation of the slope. For slopes less than 100m in length, there should be no fewer than 3 monitoring points. For slopes longer than 100m, one additional monitoring point should be added for every additional 30m. Monitoring points should be laid out in a grid or cross-sectional pattern, with the cross-sectional direction perpendicular to the slope surface, to facilitate the analysis of data both horizontally and vertically.

[0051] ③ Monitoring data quality assessment method: To ensure the quality of monitoring data, two assessment indicators are introduced: signal-to-noise ratio (SNR) analysis and signal integrity evaluation.

[0052] The signal-to-noise ratio analysis method is as follows: Among them, P signal To monitor the effective signal power of the data, P noise This represents the background noise power. Data with an SNR greater than 20dB is generally considered to be more valid, while data with a SNR lower than this value should be processed with caution or discarded.

[0053] Signal integrity assessment method: Integrity checks are performed on the continuously collected time-series data at each monitoring point. Let the total number of samples be N, and the actual effective sampling points be N based on the signal-to-noise ratio (SNR). e The formula for calculating data integrity is: When data integrity is below 95%, data preprocessing or re-collection is required.

[0054] ④ Equipment hardware parameter requirements: To ensure high accuracy and stability of data acquisition, the specific hardware parameter requirements for the equipment must be clearly defined:

[0055] A high-precision BeiDou GNSS receiver is used, and a multi-frequency high-gain measurement antenna with a gain ≥40dB and good multipath resistance is selected. The antenna should be installed on a stable support structure, and the antenna height is usually set at 1.5-2.5m to reduce the impact of ground multipath effects on the observation data.

[0056] Step 102: Perform data preprocessing on the qualified displacement data. The data preprocessing methods include outlier removal, missing value imputation, and time consistency check.

[0057] Data preprocessing is a key step in ensuring the high quality of monitoring data. This step mainly includes outlier handling, missing value imputation, and time consistency checks.

[0058] ① Outlier Handling: To address potential outliers in BeiDou-monitored displacement data, this method employs the three-sigma criterion for outlier identification and handling. The specific steps are as follows:

[0059] First, calculate the mean and standard deviation of the original displacement data sequence:

[0060]

[0061] Then, outlier detection is performed on the data: |x i If -μ|>3σ, and the above formula is satisfied, it is considered an outlier and is removed.

[0062] ② Missing value imputation: To ensure data continuity, missing values ​​need to be effectively imputed.

[0063] Trend-based interpolation: This method uses historical data trends for prediction and interpolation, employing a first-order difference model to predict: x integ (t)=x(t-1)+[x(t-1)-x(t-2)].

[0064] ③ Data Time Consistency Check: The consistency of data collection timestamps is crucial for subsequent data analysis. The specific steps are as follows: Calculate the difference ΔT between adjacent data timestamps. i =|T i -T i-1 Ideally, the sampling interval should be constant, i.e., ΔT. i The timestamp is considered abnormal when the following condition is met: |ΔT ≈ Δt. i -Δt|>ε, where ε is the allowable error threshold, typically set to 10% of the sampling interval. When timestamps are abnormal, the data needs to be reordered or re-acquired to ensure the accuracy of data analysis.

[0065] Step 103: Based on the preprocessed displacement data, establish a Kalman filter model containing displacement and velocity state vectors, and initialize the process noise covariance matrix Q and the observation noise covariance matrix R as initial filter parameters.

[0066] This step uses a state-space model to describe in detail the dynamic change process of slope displacement. The accuracy of the model directly determines the performance of the subsequent filtering algorithm. The specific steps include model construction, parameter determination, initial setting of noise covariance matrix, and sensitivity analysis.

[0067] ① State-space model construction: The slope displacement state model is usually expressed in the following state-space equation form:

[0068] State equation: X(k|k-1)=AX(k-1|k-1)+Bμ(k-1)+ω(k-1), observation equation: Z(k)=HX(k)+ν(k), where X(k) is the state vector, Z(k) is the observation vector, A is the state transition matrix, B is the control input matrix. Since there is no external control input in this method, B is usually a zero matrix; H is the observation matrix, ω(k-1) is the process noise, and ν(k) is the observation noise. Both are assumed to be white noise following a Gaussian distribution.

[0069] The state vector X(k) is defined as the displacement and velocity of the slope in the X, Y, and Z directions: X(k) = [x(k), y(k), z(k), υ x (k),υ y (k),υ z (k)] T The observation vector Z(k) is defined as the actual monitored displacement data in the X, Y, and Z directions: Z(k) = [x(k), y(k), z(k)] T The state transition matrix is ​​explicitly defined as follows: The observation matrix is ​​defined as: H = [I3 03], where I is the identity matrix.

[0070] ② Method for determining state transition matrix parameters

[0071] In the state transition matrix A, the parameter Δt is the data sampling interval, which is determined by the data sampling frequency of the monitoring system. For example, if the monitoring frequency is 1Hz, then Δt = 1s.

[0072] ③ Determination of initial values ​​for the noise covariance matrix

[0073] The initial values ​​of the process noise covariance matrix Q and the observation noise covariance matrix R have a significant impact on the filtering performance. The method for determining the initial values ​​is as follows:

[0074] The initial value of the process noise covariance matrix Q is determined empirically based on the rate of change of the initial monitoring data. The calculation formula is as follows: In the formula, The initial value of the process noise variance is determined empirically. The initial value of the observed noise covariance matrix R is determined based on equipment accuracy parameters, using the following formula: In the formula, This refers to the nominal positioning accuracy of the BeiDou monitoring receiver.

[0075] ④Sensitivity analysis of initial conditions

[0076] To ensure model stability, initial condition sensitivity analysis is required. This involves changing the initial parameter values ​​and observing the changes in the state estimation results. The steps include: setting different initial values ​​for Q and R, and performing filtering calculations for each; calculating the mean and variance of the state estimation error under different parameters, using the following formulas:

[0077] e(k) = Z(k) - HX(k|k)

[0078]

[0079] Step 104: Adaptively optimize the filter parameters using the PPO algorithm.

[0080] To further improve the performance of the Kalman filter algorithm in slope displacement monitoring, this method proposes a dynamic adaptive optimization of filter parameters based on the Proximal Policy Optimization (PPO) reinforcement learning algorithm. The detailed process includes network structure design, training process optimization strategy, action space design, reinforcement learning convergence analysis, and online verification method.

[0081] ① Network Structure Design of PPO Algorithm The PPO algorithm adopts an Actor-Critic structure, and the specific design is as follows:

[0082] The Actor network generates actions, i.e., adjustments to the Kalman filter parameters; the Critic network estimates the state-value function. The network structures are described in detail below:

[0083] Input layer: Reinforcement learning state vector: S(k)=[Z(k),X(k|k-1),X(k-1|k-1),K(k),r(k)], the dimension depends on the total number of state variables; the input data is normalized using Min-Max.

[0084] Hidden layers: There are 2 hidden layers in total. The number of neurons in each layer can be set between 64 and 128 depending on the actual data characteristics. The activation function is the ReLU function, with the expression: f(x) = max(0,x). The network weights are initialized using the Xavier initialization method.

[0085] Output layer: The Actor network outputs ΔQ(k) and ΔR(k), and the Critic network outputs the state value function.

[0086] V(s)=f critic (s;θ v ), where f critic For the Critic neural network function, θ v The parameters and weights of the Critic network are defined as follows: the output value V(s) represents the estimated future return under state s, which includes observational data such as slope displacement, velocity, and residuals, and is vectorized and input into the neural network.

[0087] ② Training process optimization strategies: To ensure the stability and convergence speed of network training, the following training optimization strategies are adopted:

[0088] Learning rate adjustment strategy: The initial learning rate is set to 0.001, and an exponential decay strategy is adopted, with the formula: lr new =lr init ×γ epoch In the formula, γ epoch This is the attenuation coefficient (typically taken as 0.95 to 0.99).

[0089] Gradient clipping strategy: To avoid gradient explosion, the gradient clipping threshold is set to 5, i.e., ||g||=min(||g||,5).

[0090] Overfitting prevention strategy: Use the Dropout technique to randomly drop 20% of neurons in each hidden layer.

[0091] Experience replay buffer: Set the capacity to 10,000 samples, update in first-in-first-out (FIFO) mode, and set the mini-batch sampling size to 64 to 128.

[0092] ③ Reinforcement learning action space selection: The action space is defined as the real-time adjustment of the Kalman filter covariance matrices Q and R. The specific actions are represented as follows:

[0093] Action(k) = {ΔQ(k), ΔR(k)}, ΔQ(k), ΔR(k) ∈ [-0.01, 0.01], and an action constraint strategy is added: the adjustment range of each action step shall not exceed 50% of the range of the previous action step.

[0094] ④ Reinforcement Learning Convergence Analysis: During the PPO algorithm training process, the change in the loss function is monitored to determine whether the algorithm has reached convergence. The loss function is defined as:

[0095] In the formula, This is the probability ratio, which measures the difference in the probability of action between the current strategy and the old strategy. A tLet ε be the advantage function and ε be the pruning parameter (usually set to 0.2). The algorithm's convergence criterion is defined as the loss function changing less than 1e for 10 consecutive periods. -4 This is considered convergence. π is the policy function, defined as the probability distribution function of taking action a in a given state s. θ (a|s) represents the probability of choosing action a in state s based on the current policy parameters θ. a represents the adjustment action to the covariance matrices Q and R, such as ΔQ and ΔR.

[0096] An entropy regularization strategy is added to balance exploration and exploitation. The entropy regularization term is formulated as H(π)=-∑π(a|s)logπ(a|s), with a weighting coefficient set to 0.01.

[0097] ⑤ Real-time error verification

[0098] To verify the adaptive adjustment effect of the filter parameters, a real-time online verification step was set up: The RMSE is fed back to the PPO network for retraining and parameter fine-tuning.

[0099] Step 105: Using statistical tests and trend inflection point identification algorithms, perform displacement trend analysis on the displacement data output by adaptive filtering, mark abnormal trend data, and use their root mean square error index as feedback information for the PPO algorithm to adjust the filtering parameters.

[0100] To accurately identify slope displacement trends and promptly detect and effectively respond to anomalies, this section describes in detail the methods for constructing displacement trend analysis, including trend significance testing, trend inflection point identification methods, and anomaly trend feedback mechanisms.

[0101] ① Displacement trend significance test: To determine the significance of the displacement data trend, this method uses the Mann-Kendall test. The steps are as follows:

[0102] First, calculate the test statistic:

[0103] in, Calculate the variance: Then calculate the standardized statistic Z:

[0104] When |Z|≥Z 1-α / 2 When α is the significance level (usually 0.05), it indicates that the trend is significant.

[0105] ② Trend Inflection Point Identification Method

[0106] To quickly identify inflection points in displacement trends, this method introduces the Cumulative Sum Control Chart (CUSUM) analysis method:

[0107] Calculate the cumulative amount: In the formula, μ is the mean value of the displacement data.

[0108] The CUSUM control limits are defined as: h = kσ. Where k is a constant (usually taken as 3), and σ is the standard deviation of the displacement data.

[0109] When |C t When |≥h, it is determined to be a trend inflection point, and the time and location of the occurrence are recorded.

[0110] ③ Data response mechanism after anomaly trend identification

[0111] Upon identification of abnormal displacement trends, a feedback mechanism is immediately initiated, following the steps below:

[0112] Abnormal trend data marking: Automatically marks trend segments that are abnormal, and records the specific time and location of the abnormality.

[0113] Feedback PPO filtering system: Abnormal trend information is fed back to the PPO reinforcement learning model as the error metric RMSE, with the formula as follows:

[0114] Filtering parameter correction: Based on feedback information, the PPO algorithm dynamically readjusts the covariance matrix parameters Q and R to further improve the real-time adaptability of the filtering effect.

[0115] Step 106: Based on the sliding window technique and the analysis of data phase changes, data segmentation is implemented. In the PPO algorithm, corresponding experience replay buffers are set for data at different stages to perform correlation updates of filter parameters.

[0116] To more effectively handle the phased changes in slope displacement monitoring data and improve monitoring accuracy and response capabilities, this section details the methods for data segmentation, determination of the sliding window length, adaptive adjustment mechanisms, automatic labeling of phased data, and the method for updating the correlation between phased data and PPO filter parameters.

[0117] ① Determination and adaptive adjustment mechanism of sliding window length

[0118] This invention employs a sliding window technique for real-time segmented data analysis. The initial sliding window length is set to an empirical value, typically 50 to 100 sampling points. Data variation within the window is evaluated using variance.

[0119] Calculate the variance of the data within the window: If the variance of the data within the window remains consistently small, it indicates that the displacement is stable, and the window length can be increased: L new =min(L old +ΔL,L maxIf the variance of the data within the window is large, reduce the window length to improve sensitivity to changes: L new =max(L old -ΔL,L min ), where ΔL is the window length adjustment step, typically taken as 5 to 10; L max L min These are the maximum and minimum limits for the window length, respectively.

[0120] ② Specific identification methods for data segmentation

[0121] The periodic changes in data were statistically tested using analysis of variance (ANOVA) to determine the data breakpoints.

[0122] First, calculate the within-group variance: Then calculate the between-group variance: Then calculate the F-statistic: When the F-statistic exceeds the critical value

[0123] The value at which the data points are considered significant indicates a significant difference between different data stages and is thus identified as a breakpoint. Here, α represents the significance level, set to 0.05, and m⁻¹ represents the degrees of freedom between groups.

[0124] ③ Automatic labeling and analysis of data at different stages

[0125] After completing the sliding window segmentation and significance statistical test, the system will enter different stages of judgment and labeling. This process dynamically identifies the slope displacement state by classifying the statistical characteristics of each data segment. First, it is necessary to calculate the statistical indicators for each data segment, including the mean u of that segment. k ,variance And the mean drift rate. Where, the mean drift rate D k =|u k -u k-1 |

[0126] After obtaining the above statistical parameters, the data segments are classified according to the set thresholds. The specific classification rules are as follows:

[0127] Stable period: And D k <D stable ;

[0128] Slow change period: And D stable ≤D k <D fast ;

[0129] Period of rapid change: And D k ≥Dfast .

[0130] To ensure the scientific validity and reliability of the stage classification, the above thresholds need to be statistically determined based on historical monitoring data. First, a sliding window analysis is performed on the historical data to calculate the variance within all data windows. Then, its probability density distribution is plotted, and the thresholds are set based on the cumulative distribution function (CDF).

[0131] σ stable Take the 80th percentile value, σ fast Take the 95th percentile value, that is In addition, the system dynamically adjusts the threshold based on real-time data, using a weighted moving average formula:

[0132] σ stable,new =β·σ stable,current +(1-β)·σ stable,old , where β is the weighting coefficient, with a value of 0.7 to 0.9.

[0133] After completing the stage classification, the system will add labels to each stage data segment, including "stable," "slowly changing," and "rapidly changing." All label information is recorded in a time index table for subsequent analysis and filtering parameter scheduling.

[0134] ④ Method for updating the correlation between phased data and PPO filter parameters

[0135] The Kalman filter parameters Q and R are configured for different data segments at different stages. To avoid system oscillations caused by abrupt changes in the filter parameters, an exponentially weighted moving average algorithm is used to achieve a smooth transition of the parameters.

[0136] Q new =α·Q target +(1-α)·Q prev

[0137] R new =α·R target +(1-α)·R prev

[0138] Where α ranges from 0.1 to 0.3. The subscript target represents the target covariance or observation matrix, a predefined parameter for label matching at different stages. The subscript prev represents the covariance or observation matrix used in the previous time step.

[0139] In the PPO algorithm section, the system designs independent Replay Buffers for different data stages. The Replay Buffers are divided into "stable period buffers", "slow change period buffers" and "rapid change period buffers" according to the stage, and independently store the experience data of different stages to prevent the mixing of samples from different stages from causing training chaos.

[0140] The core data format stored in the Replay Buffer is a quadruple (s t ,a t ,r t ,s t+1 ), where s t Let a represent the current state vector. t 'r' represents the current action, 'r' represents the current reward value, and 's' represents the current reward value. t+1 This is the state at the next moment.

[0141] To improve the quality and efficiency of training samples, a prioritized replay mechanism is adopted. Priority is calculated based on TD-Error (Temporal Difference Error). Where δ i λ represents the TD-Error value of the sample, and λ controls the priority of sampling, ranging from 0.6 to 0.8. A first-in, first-out (FIFO) mechanism is used to periodically discard samples with low TD-Error. When the system detects a stage switch, it immediately switches the Replay Buffer and policy parameters, and calls the PPO policy of the corresponding stage to optimize the network and update the parameters.

[0142] On the other hand, the present invention also provides a slope displacement monitoring system based on reinforcement learning-enhanced Kalman filtering, the system comprising:

[0143] The data acquisition module is used to set up monitoring points at preset locations on the slope to collect displacement data and perform data quality assessment.

[0144] The data preprocessing module is used to preprocess the displacement data that has passed the evaluation. The data preprocessing methods include outlier removal, missing value imputation, and time consistency check.

[0145] The slope displacement dynamic model construction module is used to build a Kalman filter model containing displacement and velocity state vectors based on preprocessed displacement data, and initialize the process noise covariance matrix Q and the observation noise covariance matrix R as initial filtering parameters.

[0146] The PPO algorithm parameter optimization module is used to adaptively optimize filter parameters using the PPO algorithm.

[0147] The displacement trend analysis module is used to perform displacement trend analysis on the displacement data output by adaptive filtering using statistical tests and trend inflection point identification algorithms, and to mark abnormal trend data so that the root mean square error index can be used as feedback information for the PPO algorithm to adjust the filtering parameters.

[0148] The segmented processing and real-time update module is used to segment data based on sliding window technology and data phase change analysis. In the PPO algorithm, corresponding experience playback buffers are set for data at different stages to perform correlation updates of filtering parameters.

[0149] It should be noted that the slope displacement monitoring system based on reinforcement learning enhanced Kalman filtering provided in this embodiment of the invention can execute the slope displacement monitoring method based on reinforcement learning enhanced Kalman filtering described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0150] The slope displacement monitoring method and system based on reinforcement learning-enhanced Kalman filtering provided by this invention have the following advantages compared with the prior art:

[0151] (1) Accurate state-space dynamic modeling improves prediction accuracy

[0152] A well-defined state-space model was adopted, and the state transition matrix, observation matrix, and initial covariance matrix were determined in detail. Sensitivity analysis of initial conditions was performed to ensure the stability and accuracy of the dynamic model. This step effectively reduced state estimation errors and provided accurate initial predictions for the Kalman filter algorithm.

[0153] (2) Introducing the PPO reinforcement learning algorithm to achieve adaptive optimization of filtering parameters significantly improves filtering performance.

[0154] By introducing the PPO reinforcement learning algorithm with an Actor-Critic structure, the process noise covariance matrix Q and the observation noise covariance matrix R in the Kalman filter are dynamically adjusted in real time to achieve adaptive optimization of the parameters. The careful design of the network structure, the exponential decay strategy of the learning rate, gradient pruning, entropy regularization, sensitivity analysis of the action space adjustment amplitude, and constraint strategies all significantly improve the stability and adaptability of the filtering algorithm, greatly reduce filtering errors, and significantly improve the accuracy and real-time performance of displacement monitoring data.

[0155] (3) Real-time online verification and cross-validation mechanism to improve algorithm robustness

[0156] An online verification method for real-time error checking (RMSE calculation) was designed, which realizes real-time monitoring and feedback of the dynamic adjustment effect of filtering algorithm parameters, effectively ensuring the accuracy and stability of filtering parameter adjustment.

[0157] (4) Significantly improves the accuracy of displacement trend identification and quickly identifies abnormal slope changes.

[0158] The displacement trend analysis module explicitly proposes the Mann-Kendall trend significance test and the CUSUM trend inflection point identification method, which effectively identifies abnormal trend changes in slope displacement data and promptly feeds them back to the filtering system for parameter optimization. This method significantly improves the sensitivity and response capability to abnormal slope changes.

[0159] (5) Data segmentation and adaptive sliding window mechanism effectively capture the characteristics of displacement changes at different stages.

[0160] An adaptive adjustment mechanism for the sliding window length was designed. Different displacement stages were identified using analysis of variance (ANOVA), and different data stage types (stable period, slow change period, and rapid change period) were automatically labeled. At the same time, adaptive correlation updates and smooth transitions between stage data and PPO filter parameters were realized, effectively improving the response speed and accuracy to slope state changes.

[0161] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 3 As shown, the electronic device may include a processor 310, a communication interface 320, a memory 330, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions from the memory 330 to execute a slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering.

[0162] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0163] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer is able to execute the slope displacement monitoring method based on reinforcement learning enhanced Kalman filtering provided in the above embodiments.

[0164] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering provided in the above embodiments.

[0165] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering, characterized in that, include: Monitoring points were set up at predetermined locations on the slope to collect displacement data, and data quality assessment was performed. The displacement data that has passed the evaluation are preprocessed, and the preprocessing methods include outlier removal, missing value imputation, and time consistency check. A Kalman filter model containing displacement and velocity state vectors is established based on the preprocessed displacement data, and the process noise covariance matrix Q and the observation noise covariance matrix R are initialized as the initial filter parameters. The filtering parameters are adaptively optimized using the PPO algorithm. By using statistical tests and trend inflection point identification algorithms, displacement trend analysis is performed on the displacement data output by adaptive filtering, and abnormal trend data are marked so that their root mean square error index can be used as feedback information for the PPO algorithm to adjust the filtering parameters. Data segmentation is achieved based on sliding window technology and phased data change analysis. In the PPO algorithm, corresponding experience replay buffers are set for data at different stages to perform correlation updates of filter parameters.

2. The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering according to claim 1, characterized in that, The steps involved in performing a data quality assessment include: The signal-to-noise ratio of the collected displacement data is determined based on the effective signal power and background noise power of the monitoring data. The sampling completeness rate is determined based on the valid sampling points and the total number of samples of the collected displacement data; If the signal-to-noise ratio and sampling completeness of the displacement data meet the preset requirements, the displacement data is determined to be qualified data.

3. The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering according to claim 1, characterized in that, The steps for preprocessing qualified displacement data include: Remove outlier data based on the three Sigma criterion; Missing value imputation based on trend prediction interpolation method; Time consistency is checked based on the difference in timestamps between adjacent data.

4. The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering according to claim 1, characterized in that, A Kalman filter model containing displacement and velocity state vectors is established based on the preprocessed displacement data, including: Define the state equation: , Define the observation equation: ; in, For state vectors, Let A be the observation vector, B be the state transition matrix, and H be the control input matrix; H be the observation matrix. For process noise, For observation noise; state vector The displacement and velocity of the slope in the X, Y, and Z directions are defined as the displacement data in the X, Y, and Z directions, respectively; the observation vector Z(k) is defined as the actual monitored displacement data in the X, Y, and Z directions.

5. The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering according to claim 1, characterized in that, The initialization process noise covariance matrix Q and the observation noise covariance matrix R include: Based on the rate of state change from the initial monitoring data, determine the initial value of the process noise covariance matrix Q; and, Based on the equipment accuracy parameters, determine the initial value of the observation noise covariance matrix R.

6. The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering according to claim 1, characterized in that, The Proximal Policy Optimization (PPO) algorithm, a reinforcement learning method, optimizes the Q and R matrices in real time to adaptively optimize filter parameters, including: The PPO algorithm is designed using an Actor-Critic network: the Actor network is used to generate actions, the Critic network is used to estimate the state value function, and the action space is defined as the real-time adjustment of the Kalman filter covariance matrices Q and R. The Actor-Critic network is trained using a pre-defined training optimization strategy; The training optimization strategies include an exponential decay strategy for adjusting the learning rate, a Dropout strategy to prevent overfitting, a gradient clipping strategy to avoid gradient explosion, and an experience replay strategy.

7. The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering according to claim 1, characterized in that, Using statistical tests and trend inflection point identification algorithms, displacement trend analysis is performed on the displacement data output by adaptive filtering, including: The significance of the displacement data trend was tested using the Mann-Kendall test. Trend inflection points are identified using cumulative sum and control chart analysis methods.

8. The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering according to claim 1, characterized in that, Analysis of changes in data over time is achieved through analysis of variance.

9. The slope displacement monitoring method based on reinforcement learning-enhanced Kalman filtering according to claim 6, characterized in that, Also includes: If the optimized filtering parameters fail the validation, the network parameters of the PPO algorithm are adjusted based on the root mean square error index.

10. A slope displacement monitoring system based on reinforcement learning-enhanced Kalman filtering, characterized in that, include: The data acquisition module is used to set up monitoring points at preset locations on the slope to collect displacement data and perform data quality assessment. The data preprocessing module is used to preprocess the displacement data that has passed the evaluation. The data preprocessing methods include outlier removal, missing value imputation, and time consistency check. The slope displacement dynamic model construction module is used to build a Kalman filter model containing displacement and velocity state vectors based on preprocessed displacement data, and initialize the process noise covariance matrix Q and the observation noise covariance matrix R as initial filtering parameters. The PPO algorithm parameter optimization module is used to adaptively optimize filter parameters using the PPO algorithm. The displacement trend analysis module is used to perform displacement trend analysis on the displacement data output by adaptive filtering using statistical tests and trend inflection point identification algorithms, and to mark abnormal trend data so that the root mean square error index can be used as feedback information for the PPO algorithm to adjust the filtering parameters. The segmented processing and real-time update module is used to segment data based on sliding window technology and data phase change analysis. In the PPO algorithm, corresponding experience playback buffers are set for data at different stages to perform correlation updates of filtering parameters.

Images