Dam deformation prediction method based on embedded feature selection

Abstract

The invention relates to a dam deformation prediction method with embedded feature selection, and belongs to the field of dam safety monitoring and artificial intelligence, and the method comprises the following steps: obtaining the original monitoring data of an environment quantity and a dam deformation quantity; performing preprocessing of data cleaning, data filling and data normalization on the original data; evaluating the importance of environmental features based on a random forest algorithm, and selecting an optimal feature set in combination with a recursive feature method; constructing a dam deformation prediction model based on a convolutional neural network, a long and short term memory neural network and an attention mechanism deep learning algorithm; and the optimal feature is used as the input of CNN-LSTM-Attention to predict the deformation of the dam. Through the method in the application, the environmental characteristics having significant influence on dam deformation can be effectively determined, the complex mapping relation between the deformation and the environmental characteristics is fully excavated, and accurate dam deformation prediction is realized.

Description

Technical Field

[0001] The invention relates to a dam deformation prediction method embedded with feature selection, and belongs to the field of dam safety monitoring and artificial intelligence. Background Art

[0002] As an important water conservancy facility, dams play an important role in ensuring the stable development of social economy and the safety of people's lives and property. However, dam operation is affected by environmental characteristics such as water pressure, temperature, and time aging, and there may be safety hazards after long-term operation. Therefore, it is very important to monitor the operation status of the dam. Deformation is an intuitive indicator that comprehensively reflects the structural status of the dam. Accurate analysis and reliable prediction of deformation data have become an important means of evaluating dam safety.

[0003] At present, when determining the environmental characteristics of dam deformation, manual feature selection is time-consuming and subjective, and it is often difficult to accurately screen the features that have a significant impact on deformation. Too many input variables will introduce redundant information, increase model complexity and overfitting risks; and insufficient input features cannot capture the nonlinear characteristics of deformation. In addition, dam deformation presents complex dynamic characteristics. Traditional statistical models and machine learning algorithms are difficult to explore the deep causal relationship between input features and dam deformation, resulting in poor model generalization ability and reduced accuracy of dam deformation prediction.

[0004] Therefore, a dam deformation prediction method embedded with feature selection is needed to solve the problems of difficult feature selection and insufficient prediction accuracy in dam deformation prediction. Summary of the invention

[0005] The technical problem to be solved by the present invention is: in order to overcome the deficiencies of the prior art, a dam deformation prediction method with embedded feature selection is provided to solve the difficulty of feature selection and insufficient prediction accuracy in dam deformation prediction.

[0006] The technical solution adopted by the present invention to solve the above problem is: a dam deformation prediction method embedded with feature selection, comprising the following steps: Step 1: synchronously collect sensor data of various environmental characteristics and deformation measurement points on the dam to obtain input and output raw data, that is, obtain environmental characteristics that affect dam deformation and dam deformation monitoring data.

[0007] Step 2: Preprocess the collected monitoring data and divide the processed data into training set and test set. The specific steps are as follows: Step 2.1, pass 3 The original data is cleaned using the three-times standard deviation criterion. For any variable, the data set is located in Data points outside of these are identified as outliers and removed. and is the mean and standard deviation of the corresponding variables, that is, the abnormal environment and displacement data in the original data are identified and removed to eliminate sensor noise and random errors; Step 2.2, estimate the missing data and the data removed in step 2.1 by using the cubic spline interpolation method, and use the interpolation results as the filling of the estimated data; Step 2.3: To prevent the differences in dimensions and magnitudes between different variables, the filled variable data are normalized using the Min-Max method. The formula is as follows: (1) In the formula, represents the normalized variable value, represents the variable value after interpolation. is the maximum value in the variable, is the minimum value among the variables; Step 2.4: Divide the data set into a training set and a test set in a ratio of 8:2.

[0008] Step 3: Use the random forest algorithm to obtain the importance of environmental features, and combine it with the recursive feature elimination strategy to select input features to generate the optimal feature set. The random forest algorithm quantifies the importance of features by randomly permuting feature values ​​and comparing the prediction errors of out-of-bag data before and after permutation. The recursive elimination strategy recursively eliminates features that are not important for prediction until the optimal feature set is selected. Specifically, feature selection is completed on the training set based on RF-RFE (Random Forest - Recursive Feature Elimination, a feature selection method based on random forest). The random forest algorithm evaluates the importance of environmental features, and the recursive feature elimination method gradually eliminates redundant features to improve the prediction performance of the model. The specific steps are as follows: Step 3.1: Use Boostrap (a statistical technique widely used in machine learning and data analysis to estimate the distribution characteristics of a data set and the performance of a model through repeated sampling) to extract samples from the training set and construct m decision trees. These m decision trees together form a random forest. l After the training samples are extracted, the samples that are not extracted are called Sample (out-of-bag sample), Since the sample did not participate in the training process, it can be used to evaluate the performance of the decision tree on new data after training. Based on the training sample, some features are randomly selected as training features to complete the training of the decision tree, and the features are recorded in The prediction mean square error on the sample is the baseline out-of-bag error, calculated as: (2) In the formula, It is l Out-of-bag mean square error before tree feature permutation, is the number of out-of-bag samples, It is l The true value of the out-of-bag sample before feature replacement, It is l The predicted value of the out-of-bag sample of each tree before feature replacement; Step 3.2: l The out-of-bag samples of the tree are randomly replaced j The value of each feature is taken, and the out-of-bag samples after replacement are re-predicted. The out-of-bag error after replacement is recorded as the replacement out-of-bag error. The calculation formula is: (3) In the formula, It is l The tree is in j The out-of-bag mean square error after feature permutation is is the number of out-of-bag samples, It is l The tree is in j The true value of the out-of-bag sample after feature permutation, It is l The tree is in j The predicted value of the out-of-bag sample after feature permutation; Step 3.3, compare j The out-of-bag error after feature replacement and the original out-of-bag error are used to calculate the increase in error. The greater the increase in error, the greater the increase in error. j The more important a feature is to the decision tree prediction, the calculation formula is: (4) In the formula, It is l The first tree j The importance of a feature; Step 3.4: j The feature importance of each feature in all trees in the random forest is averaged to obtain the permutation importance of each feature in the entire random forest. The calculation formula is: (5) In the formula, It is the first j The permutation importance of the feature is calculated in the same way as the permutation importance of the remaining features; Step 3.5: Sort the features according to the permutation importance. After removing the least important feature each time, use the remaining features to retrain the random forest model. Then reorder the features according to the permutation importance. Repeat this process recursively until the prediction performance no longer improves and the feature set with the smallest prediction error is selected as the optimal feature set.

[0009] Step 4: Construct a prediction model for dam deformation based on convolutional neural network, long short-term memory neural network, and attention mechanism, and train the network parameters of the model on the training set. The specific steps are as follows: Step 4.1: After the optimal feature set is fused and extracted through the convolution operation in the CNN layer, the ReLU activation function is used Perform non-linear activation, where the convolution operation involved is one-dimensional; Step 4.2: The LSTM layer introduces a gating mechanism and a memory cell structure to effectively capture the time dependency of the input data and overcome the gradient vanishing problem of traditional RNN in long sequence modeling. The LSTM unit gradually updates the hidden state and memory cell state at the current moment through the synergy of the gating mechanism and the memory cell. The specific calculation process is as follows: Calculate the current time The forgetting coefficient , used to decide how much information from the previous moment to retain: (6) Calculate the current time The input coefficient , and update the candidate memory cell state : (7) (8) according to and Output, update the memory cell state at the current moment : (9) Calculate the output coefficient at the current moment , and generate a new hidden state : (10) (11) In formulas (6)-(11), , , They are the forgetting coefficient, input coefficient, and output coefficient. , are the candidate memory cell state and the memory cell state respectively, , Separate moments and The hidden state of the LSTM output is It's time is the input of LSTM, , are activation functions, and They are dot product and element-wise multiplication, with subscripts and are the weight matrix and bias vector respectively.

[0010] Step 4.3: The attention mechanism layer dynamically assigns weights to highlight the time steps that have a key contribution to the dam deformation prediction in the time series features, so as to focus on the time series information that is useful for the dam deformation prediction. The specific calculation process is as follows: The hidden state received by the attention layer is , , …, , for the current query vector (that is, hidden state ), the attention mechanism calculates the attention score at each time step in the following way : (12) In the formula, It's time i The attention score, It's time t The query vector is It's time i The hidden state of is the activation function; and are the weight matrix and weight vector of the attention layer respectively.

[0011] In order to Converted into weight coefficients, the attention scores of all time steps are normalized by the softmax function. i The attention weight of time steps The calculation formula is: (13) Multiply the attention weight by the corresponding input feature and sum it up. t Output of the attention layer for: (14) Step 5: Input the test set data into the trained prediction model for prediction, use its denormalized value as the final prediction result, and select multiple evaluation indicators to evaluate the performance of the prediction model; Specifically, the optimal feature set after feature selection is used as the input of the prediction model, and the network parameters are trained on the training set. The mean square error (MSE) is used as the loss function in the model training, and minimizing the MSE is the optimization goal. Based on the characteristics of the deep learning model, the training data is divided according to the fixed sliding window and input into the model in batches for training, thereby obtaining the CNN-LSTM-Attention model for dam deformation prediction.

[0012] The mean square error (MSE), mean absolute error (MAE), and determination coefficient (R) are used. 2 The three indicators comprehensively evaluate the performance of the prediction model, and the calculation formulas are: (15) (16) (17) In formulas (15)-(17), n is the total number of samples, yes i The measured deformation value of the samples, yes i The deformation prediction value of samples, is the mean of the measured values ​​of dam deformation.

[0013] Compared with the prior art, the advantages of the present invention are: (1) The present invention considers that too many input features may generate redundant information in dam deformation prediction, while too few input feature information is insufficient to reflect the nonlinearity of deformation. It invents a feature selection method that can automatically determine the optimal input feature set and simplifies the prediction model structure. The random forest algorithm evaluates the importance of features, and the recursive feature elimination method eliminates redundant features. The combination of the two can efficiently identify environmental features that have a significant impact on dam deformation, ensuring the comprehensiveness of the feature selection process; (2) The present invention constructs a CNN-LSTM-Attention deep learning prediction model to establish the mapping relationship between environmental features and dam deformation. Compared with the traditional single modeling method, the model fully explores the deep correlation between environmental features and dam deformation from both spatial and temporal dimensions, and effectively extracts key time series information, achieving a more accurate prediction of dam deformation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1is a schematic flow chart of a dam deformation prediction method embedded with feature selection in this embodiment; Figure 2 is a process line diagram of environmental characteristics and deformation monitoring data in this embodiment; Figure 3 is a diagram of the RF-RFE recursive feature selection process in this embodiment; Figure 4 : is a schematic diagram of the structure of the CNN-LSTM-Attention prediction model in this embodiment; Figure 5 It is a comparison chart between the actual value of dam deformation and the model predicted value in this embodiment. DETAILED DESCRIPTION

[0015] like Figure 1 As shown, a dam deformation prediction method embedded with feature selection in this embodiment includes the following steps: Step 1: Collect sensor monitoring data of environmental characteristics and deformation (radial horizontal displacement of measuring point PL13-4) of a certain arch dam from September 1, 2013 to September 1, 2018 as original data of the embodiment; Step 2: Preprocess the raw data including data cleaning, data filling and data normalization. The preprocessed data are divided into training set and test set at a ratio of 8:2. The preprocessed water level, air temperature and radial horizontal displacement process lines at PL13-4 are shown in Figure 2. Figure 2 ; Step 3: In the embodiment, the input features in the feature selection method are based on the hydrostatic-season-time (HST) theory, and the arch dam displacement influence features are obtained according to the upstream water level and the number of days the dam is in service. The water pressure feature is selected , , , , is the upstream water level; temperature feature selection , , , , is the cumulative number of days from the initial monitoring date; time-effect feature selection , , Is the cumulative number of days Divide by 100.

[0016] Step 4: Select the optimal input feature set based on the training set based on the random forest algorithm and recursive feature elimination strategy. The random forest algorithm evaluates the importance of different input features and recursively eliminates the least important features in combination with recursive feature elimination. When the model prediction performance is no longer improved, the corresponding optimal feature combination is the input of the prediction model. The specific process based on recursive feature selection is shown in Figure 3 , where the number of decision trees in the random forest is set to 200. As can be seen from the figure, as the number of features decreases from ten to eight, the prediction performance of the model on out-of-bag samples gradually improves as the number of features decreases, because some input features are redundant. However, after continuing to reduce the number of features, the information contained in fewer features is not enough to reflect the nonlinear characteristics of the dam, and the prediction performance of the model on out-of-bag samples begins to deteriorate. Therefore, the number of features in the optimal feature combination is 8, which are , , , , , , , ; Step 5: The proposed CNN-LSTM-Attention model structure is shown in Figure 4 , using the optimal feature set selected in step 4, train the network parameters of the CNN-LSTM-Attention prediction model on the training set, where the prediction model receives the input of the training set according to a fixed sliding window size, compares the loss function values ​​under different parameters, and saves the optimal model as the prediction model; Step 6. In addition to the CNN-LSTM-Attention prediction model, MLR (Multiple Linear Regression, the dependent variable is predicted by a linear combination of multiple independent variables), XGBoost (eXtreme Gradient Boosting, a tree model based on ensemble learning), and CNN-GRU model (a deep learning model that combines convolutional neural network CNN and gated recurrent unit GRU) are selected as comparison models. These three models are statistical models, machine learning models, and deep learning models, respectively. The predicted values ​​on the test set are denormalized and recorded as model predicted values. The true value of the radial horizontal displacement of PL13-4 and the predicted values ​​of each model are shown in the embodiment. Figure 5 ; Step 7: Calculate the mean square error (MSE), mean absolute error (MAE) and coefficient of determination (R) between the true value and the predicted value of each model. 2 To comprehensively verify the superiority of the prediction method in the invention. The smaller the MSE and MAE, the better the R 2 The larger the value is, the better the model prediction performance is. The prediction results of each model are shown in Table 1.

[0017] Table 1:

[0018] like Figure 5As shown in Table 1, the prediction results of each model are basically consistent with the data size and change trend of the measured radial horizontal displacement process line of the PL13-4 measuring point, that is, these models can reflect the causal relationship between the dam deformation and the optimal feature combination to a certain extent. Among them, compared with the statistical model based on MLR, the machine learning model established by the XGBoost algorithm has better prediction accuracy because it can model nonlinear relationships. The CNN-GRU model based on the deep learning algorithm has more advantages than the machine learning model in mapping nonlinear relationships and extracting deep features of dam deformation, so the prediction accuracy is higher. In contrast, CNN-LSTM-Attention has the same advantages as CNN-GRU in feature extraction, and pays more attention to the time series information that is important for dam deformation prediction, achieving the best prediction performance.

[0019] In summary, the environmental characteristics that have a significant impact on dam deformation are effectively determined, and the complex mapping relationship between deformation and environmental characteristics is fully explored to achieve accurate dam deformation prediction.

[0020] In addition to the above embodiments, the present invention also includes other implementation modes. Any technical solutions formed by equivalent transformation or equivalent replacement should fall within the protection scope of the claims of the present invention.

Claims

1. A dam deformation prediction method embedded with feature selection, characterized in that: The following steps are involved: Step 1: Obtain environmental characteristics affecting dam deformation and dam deformation monitoring data; Step 2: preprocess the collected monitoring data and divide the processed data into a training set and a test set; Step 3: Select input features before building the prediction model to eliminate the influence of irrelevant environmental features on the accuracy of deformation prediction; Step 4: Based on the convolutional neural network, long short-term memory neural network, and attention mechanism algorithm, a prediction model for dam deformation is constructed, and the network parameters of the model are trained on the training set; Step 5: Input the test set data into the trained prediction model for prediction, and use its denormalized value as the final prediction result. Select multiple evaluation indicators to evaluate the performance of the prediction model.

2. A dam deformation prediction method embedded with feature selection according to claim 1, characterized in that: In step 1, monitoring data is collected from various sensors installed inside and outside the dam body, including water level, temperature, and dam deformation monitoring data.

3. The dam deformation prediction method embedded with feature selection according to claim 1 is characterized in that: In step 2, the specific steps include: Step 2.1, data cleaning: Identify and remove abnormal environmental and deformation data in the original data to eliminate sensor noise and random errors; Step 2.2, data filling: Estimate the missing data and the data removed in step 2.1 by using the cubic spline interpolation method, and use the interpolation results as the filling of the estimated data; Step 2.3, normalization: The interpolated monitoring data are processed by Min-Max normalization to eliminate the influence of dimension and magnitude differences between different variables; Step 2.4: Divide the data set into a training set and a test set in a ratio of 8:

2.

4. The dam deformation prediction method embedded with feature selection according to claim 1 is characterized in that: In step 3, the importance of environmental features is obtained by using a random forest algorithm, and the selection of input features is realized in combination with a recursive feature elimination strategy to generate an optimal feature set, wherein the random forest algorithm quantifies the importance of features by randomly permuting feature values ​​and comparing the prediction errors of out-of-bag data before and after permutation, and the recursive elimination strategy gradually eliminates features that are not important for prediction until the optimal feature set is selected; The specific steps include: Step 3.1: Extract samples from the training set using the Boostrap sampling method and construct m decision trees. These m decision trees together form a random forest. l After the training samples are extracted, the samples that are not extracted are called sample, The sample is used to evaluate the performance of the decision tree on new data after training. Based on the training sample, some features are randomly selected as training features to complete the training of the decision tree. The prediction mean square error on the sample is the baseline out-of-bag error, calculated as: （1） In the formula, It is l Out-of-bag mean square error before tree feature permutation, is the number of out-of-bag samples, It is l The true value of the out-of-bag sample before feature replacement, It is l The predicted value of the out-of-bag sample of each tree before feature replacement; Step 3.2: l The out-of-bag samples of the tree are randomly replaced j The value of each feature is taken, and the out-of-bag samples after replacement are re-predicted. The out-of-bag error after replacement is recorded as the replacement out-of-bag error. The calculation formula is: （2） In the formula, It is l The tree is in j The out-of-bag mean square error after feature permutation is is the number of out-of-bag samples, It is l The tree is in j The true value of the out-of-bag sample after feature permutation, It is l The tree is in j The predicted value of the out-of-bag sample after feature permutation; Step 3.3, compare j The out-of-bag error after feature replacement and the original out-of-bag error are used to calculate the increase in error. The greater the increase in error, the greater the increase in error. j The more important a feature is to the decision tree prediction, the calculation formula is: （3） In the formula, It is l The first tree j The importance of a feature; Step 3.4: j The feature importance of each feature in all trees in the random forest is averaged to obtain the permutation importance of each feature in the entire random forest. The calculation formula is: （4） In the formula, It is the first j The permutation importance of the feature is calculated, and the feature importance of the remaining features is calculated in the same way; Step 3.5: Sort the features according to the permutation importance. After removing the least important feature each time, use the remaining features to retrain the random forest model. Then reorder the features according to the permutation importance. Repeat this process recursively until the prediction performance no longer improves and the feature set with the smallest prediction error is selected as the optimal feature set.

5. The dam deformation prediction method embedded with feature selection according to claim 1 is characterized in that: In step 4, the dam deformation prediction model is a CNN-LSTM-Attention model, wherein CNN extracts input features through a one-dimensional convolution operation to obtain a spatial feature map; the spatial feature map extracted by CNN is used as the input of the LSTM layer to capture the long-term dependency of the time series data and generate a hidden state of the time dimension; the Attention layer weights the time series information output by the LSTM layer, calculates the weight of each time step, and obtains the weighted time series information. The specific steps include: Step 4.1: After the optimal feature set is fused and extracted through the convolution operation in the CNN layer, the ReLU activation function is used Perform non-linear activation, where the convolution operation involved is one-dimensional; Step 4.2: The LSTM layer introduces a gating mechanism and a memory unit structure to effectively capture the time dependency of the input data and overcome the gradient vanishing problem of traditional RNN in long sequence modeling. The LSTM unit gradually updates the hidden state and memory unit state at the current moment through the synergy of the gating mechanism and the memory unit. The specific calculation process is as follows: Calculate the current time The forgetting coefficient , used to decide how much information from the previous moment to retain: （5） Calculate the current time The input coefficient , and update the candidate memory cell state : （6） （7） according to and Output, update the memory cell state at the current moment : （8） Calculate the output coefficient at the current moment , and generate a new hidden state : （9） （10） In formulas (5)-(10), , , They are the forgetting coefficient, input coefficient, and output coefficient. , are the candidate memory cell state and the memory cell state respectively, , Separate moments and The hidden state of the LSTM output is It's time is the input of LSTM, , are activation functions, and They are dot product and element-wise multiplication, with subscripts and are the weight matrix and bias vector respectively; Step 4.3: The attention mechanism layer dynamically assigns weights to highlight the time steps that have a key contribution to the dam deformation prediction in the time series features, so as to focus on the time series information that is useful for the dam deformation prediction. The specific calculation process is as follows: The hidden state received by the attention layer is , , …, , for the current query vector , the attention mechanism calculates the attention score at each time step in the following way : （11） In the formula, It's time i The attention score, It's time t The query vector is It's time i The hidden state of is the activation function; and They are the weight matrix and weight vector of the attention layer respectively; In order to Converted into weight coefficients, the attention scores of all time steps are normalized by the softmax function. i The attention weight of time steps The calculation formula is: （12） Multiply the attention weight by the corresponding input feature and sum it up. t Output of the attention layer for: （13）。 6. The dam deformation prediction method embedded with feature selection according to claim 1 is characterized in that: In step 5, the mean square error MSE, mean absolute error MAE, and determination coefficient R are used. 2 The three evaluation indicators are used to comprehensively evaluate the prediction model, and the calculation formulas are: （14） （15） （16） In formulas (14)-(16), n is the total number of samples, yes i The measured deformation value of the samples, yes i The deformation prediction value of samples, It is the mean of the measured values ​​of dam deformation.

Images