Dam deformation monitoring data outlier adaptive identification method

Abstract

The adaptive outlier identification method for dam deformation monitoring data disclosed in this invention mainly involves introducing the Baseline Management System (BMS) method to optimize the key influencing factors in the dam deformation monitoring model. Then, a robust regression model for dam deformation is established using the optimized set of key influencing factors. Combined with the Longest Trigger Score (LTS) estimation method, robust regression analysis is performed on the dam deformation monitoring data. This allows for simultaneous regression modeling, outlier identification, and structural deformation prediction without the need for data preprocessing. This adaptive outlier identification method can adaptively overcome the misleading effect of outliers on regression, enhancing the significance of the regression results and effectively improving the accuracy of data prediction.

Description

Technical Field

[0001] This invention belongs to the technical field of dam safety monitoring methods, specifically relating to an adaptive identification method for outlier values ​​in dam deformation monitoring data. Background Technology

[0002] Establishing a safety monitoring model using dam deformation monitoring data is an important method for quantitatively analyzing dam deformation behavior. Currently, there are three main types of safety monitoring models that can be established for dam deformation. Among them, statistical models are widely used in the quantitative analysis of dam deformation monitoring data due to their advantages such as mature theory, simple modeling, and ease of use.

[0003] Common statistical models often use empirical influence factors to describe the impact of loads on dam deformation behavior. However, in actual engineering, the factors affecting dam deformation are complex and multifaceted. Redundant factors in empirical influence factors may actually lead to poorer goodness of fit and lower prediction accuracy in the statistical model. In addition, during the collection of dam deformation monitoring data, due to the complexity of the data collection environment, the limitations of the data collection equipment, and the influence of human factors, the raw monitoring data obtained contains not only information about the actual operating state of the dam but also outliers that do not conform to the specific physical and mechanical relationship between environmental quantities and deformation effects. This causes the statistical model to deviate significantly from reality. At the same time, the large volume of dam safety monitoring data also increases the cost and workload of outlier removal to some extent.

[0004] In summary, in order to effectively eliminate redundant factors in the empirical influencing factors of dam deformation and adaptively overcome the adverse effects of outlier interference on the performance of statistical models, it is essential to propose a new safety monitoring model and identify outliers in dam deformation monitoring data based on this model. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive method for identifying outliers in dam deformation monitoring data. This method constructs a robust regression model based on BMS-LTS, which can adaptively overcome the misleading effect of outliers on regression and effectively improve the accuracy of data prediction.

[0006] The technical solution adopted in this invention is an adaptive identification method for outlier values ​​in dam deformation monitoring data, which specifically includes the following steps:

[0007] Step 1: Based on the actual engineering situation, determine the dam deformation measurement points that need to be adaptively identified for outliers. Use the deformation and empirical influencing factor monitoring data of the corresponding measurement points as the dataset for building a robust regression model, and divide the dataset into training and test sets according to the size of the dataset and actual needs.

[0008] Step 2: The empirical influencing factors affecting dam deformation are reduced using the BMS method, and redundant factors in the empirical influencing factors are removed using the backward elimination method of BIC, thereby determining the key influencing factors for constructing a robust regression model.

[0009] Step 3: Construct a multiple linear regression model using the deformation of the training set and the measured values ​​of the monitoring data of its key influencing factors, and obtain the final regression coefficients through LTS estimation, thereby establishing a robust regression model based on BMS-LTS.

[0010] Step 4: Select several data points used for the final regression coefficient estimation as the optimal data group, and mark the remaining data as outliers. Identify potential horizontal shift anomalies in the data sequence and use a double wedge plot to visualize the changes in outliers in the data sequence.

[0011] Step 5: Input the key influencing factors of the test set into the robust regression model based on BMS-LTS trained in Step 3 to obtain the dam deformation prediction values ​​for the corresponding measuring points.

[0012] The invention is further characterized in that,

[0013] Step 2 specifically includes the following steps:

[0014] Step 2.1: Construct a complete linear regression model using all the empirical influencing factors obtained in Step 1. Randomly remove one empirical influencing factor from the model and fit a new linear regression model with the remaining factors, recording its BIC value. Repeat this process the same number of times as all empirical influencing factors, and select the linear regression model with the smallest BIC value for subsequent calculation and analysis.

[0015] Step 2.2: For the linear regression model with the smallest BIC value selected in Step 2.1, by removing one empirical influencing factor from the model again, further observe the BIC values ​​of several new linear regression models fitted, and select the model with the lowest BIC value for subsequent calculation and analysis.

[0016] Step 2.3: Repeat steps 2.1 to 2.2 until the BIC value of the fitted linear regression model no longer decreases. Finally, select the linear regression model with the global minimum BIC value as the best parsimony model, and its corresponding factor is the key influencing factor of the robust regression model to be constructed.

[0017] The formula for calculating the BIC value in step 2 is:

[0018] BIC=-2ln(likelihood)+(m+1)ln(n) (1)

[0019] In the formula: n is the number of observations in the linear regression model, and m is the number of predicted values;

[0020] Given the model M and its parameter θ, the likelihood function is as follows:

[0021] likelihood=P(data|θ,M)=L(θ,M) (2);

[0022] When the sample size is large enough and the data follows an exponential family distribution, the BIC value can be approximated as:

[0023] BIC≈-2 ln(P(data|M))=-2ln(∫P(data|θ,M)P(θ|M)dθ) (3)

[0024] In the formula: P(data|M) is the marginal likelihood function of data under model M, and P(θ|M) is the prior distribution of parameter θ.

[0025] Step 3 specifically includes the following steps:

[0026] Step 3.1: Use the modified training set of the corresponding measurement points and the monitoring data of key influencing factors selected by the BMS method to construct a multiple linear regression model. Let the number of measured values ​​of the monitoring data be n, then the multiple linear regression model can be expressed as:

[0027]

[0028] In the formula: x is the independent variable, x∈R n×p ; y is the dependent variable, y∈R n ; For the parameter to be estimated, ε is the random error term, ε∈R n p represents the number of independent variables;

[0029] Step 3.2: Use LTS estimation to estimate the parameters of the multiple linear regression model constructed in Step 3.1. Randomly select h different sample points from n measured values ​​to form a sample subset H0, and use the LS method to calculate the initial regression coefficients.

[0030] Step 3.3: Substitute the initial regression coefficients obtained in Step 3.2 into the multiple linear regression model established in Step 3.1 to calculate the squared residuals of the n measured values. Arrange the squared residuals in ascending order and denote the sum of the squared residuals of the first h measured values ​​as . Given any parameters Define the squared residual as:

[0031]

[0032] In the formula: (x i ,y i) represents the measured value of the monitoring data for the i-th sample point. Let be the squared residual corresponding to the i-th sample point;

[0033] Step 3.4: Retain the samples corresponding to the first h measured values ​​with least square residuals from Step 3.3, and use them as a new sample subset H1. Calculate the new regression coefficients using the LS method.

[0034] Step 3.5: Substitute the new regression coefficients obtained in Step 3.4 into the multiple linear regression model established in Step 3.1, calculate the squared residuals of the n measured values, and arrange the squared residuals in ascending order. Let the sum of the squared residuals of the first h measured values ​​be denoted as .

[0035] Step 3.6: Repeat steps 3.2 to 3.5 until the sum of the squared residuals of the first h measured values ​​is reached. Convergence, then The expression is:

[0036]

[0037] Wherein, the range of values ​​for h must satisfy...

[0038] Step 3.7, take the contents of step 3.6... When the final regression coefficients corresponding to the convergence are substituted into the multiple linear regression model established in step 3.1, a robust regression model based on BMS-LTS is obtained.

[0039] Step 4 specifically includes the following steps:

[0040] Step 4.1: Add the corresponding horizontal displacement anomaly detection factor, δ1I (w≥δ2), to the multiple linear regression model obtained in Step 3.1, where w=1,…,n; I(.) is the indicator function, δ2 is the exact location of the horizontal displacement anomaly, and w (1 ),…,w (S) The possible positions of δ2, w (s) ∈{w (1) ,…,w (S)};

[0041] Step 4.2, let δ2 = w (s) Where s is initially set to 1;

[0042] Step 4.3: Construct a subset E of elements containing p-1 different measured values, while maintaining δ2 = w. (s) Repeat steps 3.2 to 3.4 twice;

[0043] Step 4.4: Repeat step 4.3 several times from 1 to the number of element subsets E, performing steps 3.2 to 3.6 on the nbest element subsets that produce the minimum objective function until convergence; if s > 1, also start from w. (s-1) Starting with the nbest subset of elements discovered, δ2 is also set to w. (s) Continue with steps 3.2 through 3.6 until convergence.

[0044] Step 4.5: Select the regression coefficient with the minimum objective function from the 2×nbest data points, and use... This indicates that the corresponding residuals are stored.

[0045]

[0046] Step 4.6: Apply steps 4.2 to 4.5 to all w (s) The loop continues, where s = 1, ..., S, and multiple iterations will be generated thereafter. Choose a minimum objective function of And recorded as

[0047] Step 4.7: Apply a univariate outlier detection program to detect anomalies in the data sequence;

[0048] Step 4.8, from the initial estimate To begin, keeping δ2 constant, the LS method is applied to all points that were not marked as anomalies in step 4.7 to obtain the outlier detection results;

[0049] Step 4.9: Based on the outlier detection results obtained in Step 4.8, draw a double wedge diagram to visually represent the horizontal displacement anomalies and possible anomalies in the monitoring data.

[0050] In step 4.3, the subset E of elements should include w. (s) The corresponding measured value, a value satisfying w <w (s) The measured values ​​and another p-3 measured values ​​randomly selected from the entire data sequence.

[0051] The beneficial effects of this invention are:

[0052] (1) The adaptive identification method for outliers in dam deformation monitoring data of the present invention uses the BMS method to reduce the empirical influencing factors of dam deformation, eliminates redundant factors in the dam safety monitoring model, determines the key influencing factor set of the model, and effectively improves the reliability of parameter estimation results.

[0053] (2) The robust regression model based on BMS-LTS constructed by the adaptive identification method of outlier in dam deformation monitoring data of the present invention can overcome the adverse effects of outliers in deformation monitoring data on regression estimation results. It adaptively achieves robustness to outliers during the learning process, thereby enhancing the significance of regression, improving the goodness of fit and prediction accuracy.

[0054] (3) The adaptive identification method for outliers in dam deformation monitoring data of the present invention realizes the accurate identification of outliers and horizontal displacement anomalies in dam deformation monitoring data, and uses a double wedge diagram to intuitively visualize the possible anomalies in the data sequence, which has broad application prospects. Attached Figure Description

[0055] Figure 1 This is a flowchart of the adaptive identification method for outlier values ​​in dam deformation monitoring data according to the present invention;

[0056] Figure 2 This is a diagram showing the arrangement of deformation measuring points for the arch dam body in Embodiment 1 of the present invention;

[0057] Figure 3 This is a schematic diagram of the radial horizontal displacement process lines of each measuring point after noise addition in Embodiment 1 of the present invention;

[0058] Figure 4 This is a schematic diagram of the adaptive identification results of abnormal values ​​in the horizontal displacement monitoring data of PL3-2 measuring point in Embodiment 1 of the present invention;

[0059] Figure 5 This is a schematic diagram of the adaptive identification result of abnormal values ​​in the horizontal displacement monitoring data of PL5-1 measuring point in Embodiment 1 of the present invention;

[0060] Figure 6 This is a comparison chart of the measured horizontal displacement of PL3-2 measuring point in Embodiment 1 of the present invention with the fitted values ​​and predicted values ​​of different models;

[0061] Figure 7 This is a comparison chart of the measured horizontal displacement of PL5-1 measuring point in Embodiment 1 of the present invention with the fitted values ​​and predicted values ​​of different models. Detailed Implementation

[0062] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0063] This invention relates to an adaptive method for identifying outliers in dam deformation monitoring data, such as... Figure 1 As shown, the specific steps include:

[0064] Step 1: Based on the actual engineering situation, determine the dam deformation measurement points that need to be adaptively identified for outliers. Use the deformation and empirical influencing factor monitoring data of the corresponding measurement points as the dataset for building a robust regression model, and divide the dataset into training and test sets according to the size of the dataset and actual needs.

[0065] Step 2: The Bayesian Model Selection (BMS) method is used to reduce the empirical influencing factors affecting dam deformation. The Bayesian Information Criterion (BIC) method is then used to eliminate redundant factors from the empirical influencing factors, thereby determining the key influencing factors for constructing a robust regression model. This includes the following steps:

[0066] Step 2.1: Construct a complete linear regression model using all the empirical influencing factors obtained in Step 1. Randomly remove one empirical influencing factor from this model, and fit a new linear regression model using the remaining factors, recording its BIC value. Repeat this process the same number of times as all empirical influencing factors, selecting the linear regression model with the smallest BIC value for subsequent calculations and analyses. The formula for calculating the BIC value is:

[0067] BIC=-2 ln(likelihood)+(m+1)ln(n) (1)

[0068] In the formula: n is the number of observations in the linear regression model, and m is the number of predicted values. Given the model M and its parameter θ, the likelihood function is as follows:

[0069] likelihood=P(data|θ,M)=L(θ,M) (2);

[0070] When the sample size is large enough and the data follows an exponential family distribution, the BIC can be approximated as:

[0071] BIC≈-2 ln(P(data|M))=-2 ln(∫P(data|θ,M)P(θ|M)dθ) (3)

[0072] In the formula: P(data|M) is the marginal likelihood function of data under model M, and P(θ|M) is the prior distribution of parameter θ;

[0073] Step 2.2: For the linear regression model with the smallest BIC value selected in Step 2.1, by removing one empirical influencing factor from the model again, further observe the BIC values ​​of several new linear regression models fitted, and select the model with the lowest BIC value for subsequent calculation and analysis.

[0074] Step 2.3: Repeat steps 2.1 to 2.2 until the BIC value of the fitted linear regression model no longer decreases. Finally, select the linear regression model with the global minimum BIC value as the best parsimony model, and its corresponding factor is the key influencing factor of the robust regression model to be constructed.

[0075] Step 3: Construct a multiple linear regression model using the modified training set and the measured values ​​of key influencing factor monitoring data, and obtain the final regression coefficients through LTS estimation, thereby establishing a robust regression model based on BMS-LTS. This specifically includes the following steps:

[0076] Step 3.1: Use the modified training set of the corresponding measurement points and the monitoring data of key influencing factors selected by the BMS method to construct a multiple linear regression model. Let the number of measured values ​​of the monitoring data be n, then the multiple linear regression model can be expressed as:

[0077]

[0078] In the formula: x is the independent variable, x∈R n×p ; y is the dependent variable, y∈R n ; For the parameter to be estimated, P is the random error term, ε∈R n p represents the number of independent variables;

[0079] Step 3.2: Use LTS estimation, i.e., least squares estimation, to estimate the parameters of the multiple linear regression model constructed in step 3.1. Randomly select h different sample points from n measured values ​​to form a sample subset H0, and use the LS method, i.e., least squares method, to calculate the initial regression coefficients.

[0080] Step 3.3: Substitute the initial regression coefficients obtained in Step 3.2 into the multiple linear regression model established in Step 3.1 to calculate the squared residuals of the n measured values. Arrange the squared residuals in ascending order and denote the sum of the squared residuals of the first h measured values ​​as . Given any parameters Define the squared residual as:

[0081]

[0082] In the formula: (x i ,y i ) represents the measured value of the monitoring data for the i-th sample point. Let be the squared residual corresponding to the i-th sample point;

[0083] Step 3.4: Retain the samples corresponding to the first h measured values ​​with least square residuals from Step 3.3, and use them as a new sample subset H1. Calculate the new regression coefficients using the LS method. Generally speaking, the new regression coefficients have a smaller objective function than the initial regression coefficients.

[0084] Step 3.5: Substitute the new regression coefficients obtained in Step 3.4 into the multiple linear regression model established in Step 3.1, calculate the squared residuals of the n measured values, and arrange the squared residuals in ascending order. Let the sum of the squared residuals of the first h measured values ​​be denoted as .

[0085] Step 3.6: Repeat steps 3.2 to 3.5 until the sum of the squared residuals of the first h measured values ​​is reached. Convergence, then The expression is:

[0086]

[0087] Wherein, the range of values ​​for h must satisfy...

[0088] Step 3.7, take the contents of step 3.6... When the final regression coefficients corresponding to the convergence are substituted into the multiple linear regression model established in step 3.1, a robust regression model based on BMS-LTS is obtained.

[0089] Step 4: Select several data points used for the final regression coefficient estimation as the optimal data set, and mark the remaining data as outliers. Identify potential horizontal shift anomalies in the data sequence and visualize the changes in outliers using a double wedge plot. This includes the following steps:

[0090] Step 4.1: Add the corresponding horizontal shift anomaly detection factor, δ1I (w≥δ2), to the multiple linear regression model obtained in Step 3.1, where w=1,…,n; I( . ) is the indicator function, δ2 is the exact location of the horizontal displacement anomaly, and w (1) ,…,w (S) The possible positions of δ2, w (s) ∈{w (1) ,…,w (S)};

[0091] Step 4.2, let δ2 = w (s) Where s is initially set to 1;

[0092] Step 4.3: Construct a subset E of elements containing p-1 different measured values, while maintaining δ2 = w. (s)Without changing the order, execute steps 3.2 to 3.4 twice, where the element subset E should include w. (s) The corresponding measured value, a value satisfying w <w (s) The measured values ​​and another p-3 measured values ​​randomly selected from the entire data sequence;

[0093] Step 4.4: Repeat step 4.3 several times from 1 to the number of element subsets E, performing steps 3.2 to 3.6 on the nbest element subsets that produce the minimum objective function until convergence; if s > 1, also start from w. (s-1) Starting with the nbest subset of elements discovered, δ2 is also set to w. (s) Continue with steps 3.2 through 3.6 until convergence.

[0094] Step 4.5: Select the regression coefficient with the minimum objective function from the 2×nbest data points, and use... This indicates that the corresponding residuals are stored.

[0095]

[0096] Step 4.6: Apply steps 4.2 to 4.5 to all w (s) The loop continues, where s = 1, ..., S, and multiple iterations will be generated thereafter. Choose a minimum objective function of And recorded as

[0097] Step 4.7: Apply a univariate outlier detection program to detect anomalies in the data sequence;

[0098] Step 4.8, from the initial estimate To begin, keeping δ2 constant, the LS method is applied to all points that were not marked as anomalies in step 4.7 to obtain the outlier detection results;

[0099] Step 4.9: Based on the outlier detection results obtained in Step 4.8, draw a double wedge diagram to visually represent the horizontal displacement anomalies and possible anomalies in the monitoring data.

[0100] Step 5: Input the key influencing factors of the test set into the robust regression model based on BMS-LTS trained in Step 3 to obtain the dam deformation prediction values ​​for the corresponding measuring points.

[0101] Example 1

[0102] The adaptive identification method for outlier values ​​in dam deformation monitoring data, as described in this invention, is used to identify horizontal displacement anomalies in a hydropower station dam. The method includes the following steps:

[0103] Step 1: The water-retaining structure of a certain hydropower station is a concrete double-curvature arch dam with a maximum height of 250m. Multiple deformation monitoring items, including horizontal displacement, vertical displacement, and deflection, are installed on this arch dam. The horizontal displacement of the dam body is monitored using the plumb line method. Figure 2 This is a diagram showing the layout of deformation measuring points for the arch dam. To ensure that the data used can accurately reflect the evolution of the dam's operational behavior, the horizontal displacement and empirical influence factor monitoring data of measuring points PL3-2 and PL5-1, which are located near the middle of the dam and have different elevations, were selected as the model dataset. The model dataset was then divided according to a training set:test set ratio of 8:2.

[0104] Furthermore, to verify the effectiveness of the outlier adaptive identification method proposed in this invention, noise was added to the horizontal displacement data sequences of the two measuring points, PL3-2 and PL5-1. The radial horizontal displacement process lines of each measuring point after noise addition are shown below. Figure 3 As shown.

[0105] Noise 1: The data sequence of PL3-2 measurement point was noise-added by adding a horizontal shift anomaly and three isolated outliers. Specifically, 3.0 was added to the 231st data point (October 18, 2017) and subsequent data points, 2.5 was subtracted from the 130th data point (March 4, 2015), 1.5 was added to the 202nd data point (October 19, 2016), and 2.0 was subtracted from the 268th data point (October 4, 2018).

[0106] Noise 2: The data sequence of PL5-1 measurement point was noise-added by adding three sets of consecutive outliers. Specifically, 8.0 was added to the 171st data point (October 16, 2015) to the 192nd data point (March 3, 2016), 8.0 was subtracted from the 193rd data point (March 15, 2016) to the 212th data point (October 19, 2016), and 7.0 was added to the 264th data point (June 6, 2018) to the 282nd data point (November 14, 2018).

[0107] Step 2: The empirical influencing factors affecting dam deformation are reduced using the BMS method, and redundant factors are eliminated using the backward elimination method of BIC, thereby determining the key influencing factors for constructing the robust regression model. In this embodiment, a total of 12 initial empirical influencing factors are selected to represent the horizontal displacement of the arch dam, as shown in Table 1.

[0108] Table 1 Summary of Initial Empirical Influence Factors

[0109]

[0110] In the table: H and H0 represent the upstream water head corresponding to the monitoring date and the initial measurement date, respectively; a i t is the regression coefficient of the water pressure factor; t is the cumulative number of days from the monitoring date to the initial measurement date; t0 is the cumulative number of days from the first monitoring date to the initial measurement date in the modeling data series; b 1i b 2i θ is the regression coefficient for the temperature factor; θ is the cumulative number of days t from the monitoring date to the initial measurement date divided by 100; θ0 is the cumulative number of days t0 from the first monitoring date to the initial measurement date in the modeling data series divided by 100; c1 and c2 are the regression coefficients for the time factor. Considering the influence of the coordinate displacement benchmark value and the initial measurement value, a constant term a0 is generally needed to be added to the regression model.

[0111] In order to find 4096 (i.e. 2) 12 =4096) The best linear regression model among the possible models was selected, and the influencing factors were reduced using the backward elimination method of BIC. The specific process of model selection using the backward elimination method of BIC in this embodiment is shown in Table 2.

[0112] Table 2. Specific process of model selection using the backward elimination method of BIC.

[0113] step Model Factors Elimination factor BIC value Complete model <![CDATA[(x1,x2,…,x 12 )]]> 501.69 Step 1 <![CDATA[(x1,x2,…,x8,x 10 ,x 11 ,x 12 )]]> <![CDATA[x9]]> 500.90 Step 2 <![CDATA[(x1,x2,…,x8,x 11 ,x 12 )]]> <![CDATA[x 10 ]]> 498.61

[0114] First, by removing one of the 12 influencing factors, the study found that the BIC value was minimized when the temperature-related influencing factor x9 was removed from the complete model. Therefore, the model reduced to x9 was chosen as the new model in this step. Second, by removing another influencing factor, the model with 11 remaining influencing factors was further analyzed. It was found that removing factor x9 from the model... 10 After that, the BIC value will decrease further; further, removing any of the remaining 10 influencing factors will not significantly reduce the model's BIC value. Therefore, after optimization, the model excluding x9 and x... 10 The 10 influencing factors are the key influencing factors affecting dam deformation.

[0115] Step 3: Construct a multiple linear regression model using the 10 key influencing factors selected from the training set and the measured values ​​of horizontal displacement monitoring data. Obtain the final regression coefficient estimate of the model through LTS estimation, thereby establishing a robust regression model based on BMS-LTS. The final form of the model is shown in Equation (8). The final regression coefficients of the horizontal displacement statistical model of PL3-2 and PL5-1 measuring points are detailed in Table 3.

[0116]

[0117] Table 3 Final Regression Coefficients of the Statistical Model for Horizontal Displacement of Measuring Points PL3-2 and PL5-1

[0118] Regression coefficient PL3-2 PL5-1 <![CDATA[a0]]> -3848.7 -1508.6 <![CDATA[a1]]> 871.3 362.1 <![CDATA[a2]]> -73.05 -31.71 <![CDATA[a3]]> 2.71 1.23 <![CDATA[a4]]> -0.038 -0.018 <![CDATA[c1]]> 2.13 -9.46 <![CDATA[c2]]> 0.096 0.98 <![CDATA[b 11 ]]> -1.19 -5.36 <![CDATA[b 21 ]]> 0.74 -1.19 <![CDATA[b 12 ]]> -0.026 0.45 <![CDATA[b 22 ]]> 0.019 0.015 <![CDATA[b ls ]]> 3.91 /

[0119] Step 4: Construct an adaptive outlier identification method for dam deformation monitoring data to monitor data sequences that may contain different types of outliers. The outlier identification capability is tested by applying this method to two different datasets: a test set and a training set. The adaptive outlier identification results for the horizontal displacement monitoring data of PL3-2 and PL5-1 measuring points are shown below. Figure 4 , Figure 5 As shown in the figure. The results show that the robust regression model based on BMS-LTS adaptively identifies outliers added to the horizontal displacement data sequences of PL3-2 and PL5-1 measuring points, effectively avoiding the adverse effects of outliers on the performance of the regression model;

[0120] Step 5: Input the 10 key influencing factors of the test set into the robust regression model based on BMS-LTS trained in Step 3 to obtain the dam deformation prediction values ​​of PL3-2 and PL5-1 measuring points. At the same time, in order to verify the performance of the robust regression model based on BMS-LTS in dam deformation prediction, the horizontal displacement and key influencing factor monitoring data of PL3-2 and PL5-1 measuring points after noise addition are used to construct a regression model. The data samples of the training set and the test set are fitted and predicted, and the results are compared with the traditional multiple linear regression model based on LS fitting. Figure 6 , Figure 7 The figures show a comparison between the measured horizontal displacement values ​​at PL3-2 and PL5-1 and the fitted and predicted values ​​from different models. As can be seen, when outliers exist in the training data, the regression error of the LS-based multiple linear regression model at these outliers dominates the increase or decrease of the overall loss function value. This causes the trained model to be biased towards outlier samples, making it highly sensitive to outliers and lacking robustness. In contrast, the robust regression model based on LTS exhibits higher fitting and prediction accuracy and shows stable performance on both the training and test sets. Compared to traditional non-robust regression methods, the application of LTS estimation technology in the model can adaptively identify outliers in dam deformation monitoring data and accurately uncover the complex interaction between dam deformation effect quantities and their influencing factors while overcoming the interference of outliers. This allows for the determination of the dam deformation behavior evolution trend and the achievement of the goal of dam deformation safety monitoring.

Claims

1. An adaptive method for identifying outliers in dam deformation monitoring data, characterized in that, Specifically, the following steps are included: Step 1: Based on the actual engineering situation, determine the dam deformation measurement points that need to be adaptively identified for outliers. Use the deformation and empirical influencing factor monitoring data of the corresponding measurement points as the dataset for building a robust regression model, and divide the dataset into training and test sets according to the size of the dataset and actual needs. Step 2: The empirical influencing factors affecting dam deformation are reduced using the BMS method, and redundant factors in the empirical influencing factors are removed using the backward elimination method of BIC, thereby determining the key influencing factors for constructing a robust regression model. Step 3: Construct a multiple linear regression model using the deformation of the training set and the measured values ​​of the monitoring data of its key influencing factors, and obtain the final regression coefficients through LTS estimation, thereby establishing a robust regression model based on BMS-LTS. Step 4: Select several data points used for the final regression coefficient estimation as the optimal data group, and mark the remaining data as outliers. Identify potential horizontal shift anomalies in the data sequence and use a double wedge plot to visualize the changes in outliers in the data sequence. Step 5: Input the key influencing factors of the test set into the robust regression model based on BMS-LTS trained in Step 3 to obtain the dam deformation prediction values ​​for the corresponding measuring points. Step 3 specifically includes the following steps: Step 3.1: Use the modified training set of the corresponding measurement points and the monitoring data of key influencing factors selected by the BMS method to construct a multiple linear regression model. Let the number of measured values ​​of the monitoring data be... Then the multiple linear regression model is expressed as: （4） In the formula: As the independent variable, ; As the dependent variable, ; For the parameter to be estimated, ; For random error term, ; p The number of independent variables; Step 3.2: Use LTS estimation to estimate the parameters of the multiple linear regression model constructed in Step 3.

1. Randomly selected from the measured values A subset of samples consists of 10 different sample points. The initial regression coefficients were calculated using the LS method. Step 3.3: Substitute the initial regression coefficients obtained in Step 3.2 into the multiple linear regression model established in Step 3.1 to calculate... The squared residuals of the measured values ​​are obtained and sorted in ascending order, with the first few being recorded. The sum of the squared residuals of the measured values ​​is Given any parameters The squared residual is defined as: （5） In the formula: For the first i Measured values ​​of monitoring data from sample points For the first i The squared residuals corresponding to each sample point; Step 3.4: Retain the first few with the least squared residual from Step 3.

3. The samples corresponding to each measured value are used as a new sample subset. The new regression coefficients were calculated using the LS method. Step 3.5: Substitute the new regression coefficients obtained in Step 3.4 into the multiple linear regression model established in Step 3.1 to calculate... n The squared residuals of the measured values ​​are obtained and sorted in ascending order, with the first few being recorded. h The sum of the squared residuals of the measured values ​​is ; Step 3.6: Repeat steps 3.2 to 3.5 until the previous step is completed. h The sum of the squared residuals of each measured value Convergence, then The expression is: （6） in, The range of values ​​must satisfy ; Step 3.7, take the contents of step 3.6... When the final regression coefficients corresponding to the convergence are substituted into the multiple linear regression model established in step 3.1, a robust regression model based on BMS-LTS is obtained.

2. The adaptive identification method for outliers in dam deformation monitoring data according to claim 1, characterized in that, Step 2 specifically includes the following steps: Step 2.1: Construct a complete linear regression model using all the empirical influencing factors obtained in Step 1. Randomly remove one empirical influencing factor from the model and fit a new linear regression model with the remaining factors, recording its BIC value. Repeat this process the same number of times as all empirical influencing factors, and select the linear regression model with the smallest BIC value for subsequent calculation and analysis. Step 2.2: For the linear regression model with the smallest BIC value selected in Step 2.1, by removing one empirical influencing factor from the model again, further observe the BIC values ​​of several new linear regression models fitted, and select the model with the lowest BIC value for subsequent calculation and analysis. Step 2.3: Repeat steps 2.1 to 2.2 until the BIC value of the fitted linear regression model no longer decreases. Finally, select the linear regression model with the global minimum BIC value as the best parsimony model, and its corresponding factor is the key influencing factor of the robust regression model to be constructed.

3. The adaptive identification method for outliers in dam deformation monitoring data according to claim 2, characterized in that, The formula for calculating the BIC value in step 2 is as follows: （1） In the formula: It is the number of observations in the linear regression model. It is the number of predicted values; Given model and its parameters The likelihood function is as follows: （2）； When the sample size is large enough and the data follows an exponential family distribution, the BIC value is approximately: （3） In the formula: It is a model The following is about The marginal likelihood function, It's about parameters. The prior distribution of .

4. The adaptive identification method for outliers in dam deformation monitoring data according to claim 3, characterized in that, Step 4 specifically includes the following steps: Step 4.1: Add the corresponding horizontal shift anomaly detection factor to the multiple linear regression model obtained in Step 3.1, i.e. ,in ; I ( . ) is an indicator function. To pinpoint the exact location of the horizontal displacement anomaly. yes Possible locations, ; Step 4.2, set The initial design ; Step 4.3, construct the included A subset of elements with different measured values On this basis, maintain Repeat steps 3.2 to 3.4 twice; Step 4.4: From step 4.3, convert 1 to a subset of elements. E The algorithm iterates several times within a given range to find the minimum objective function. Perform steps 3.2 to 3.6 on each subset of elements until convergence; if Also from research Discovered at the time Starting with a subset of elements, the same settings are applied. Continue with steps 3.2 through 3.6 until convergence. Step 4.5, in Select the regression coefficients with the minimum objective function from the data, and use them... This indicates that the corresponding residuals are stored. : （7）； Step 4.6: Transfer steps 4.2 to 4.5 to all... The upper loop, where After that, multiple Choose one with the minimum objective function. of , and record as ; Step 4.7: Apply a univariate outlier detection program to detect anomalies in the data sequence; Step 4.8, from the initial estimate Begin, keep Without changing the previous steps, the LS method is applied to all points that were not marked as anomalies in step 4.7 to obtain the outlier detection results; Step 4.9: Based on the outlier detection results obtained in Step 4.8, draw a double wedge diagram to visually represent the horizontal displacement anomalies and possible anomalies in the monitoring data.

5. The adaptive identification method for outliers in dam deformation monitoring data according to claim 4, characterized in that, The element subset mentioned in step 4.3 Should include The corresponding measured value, a satisfying The measured values ​​and other values ​​randomly sampled from the entire data series One measured value.

Images