Tunnel abnormal data early warning method based on LSTM model

Abstract

The invention relates to the technical field of tunnel engineering monitoring, in particular to a tunnel abnormal data early warning method based on an LSTM model. Comprising the following steps: S1, data preprocessing; s2, building a model, wherein the structure of the model comprises two layers of LSTM networks and one layer of full connection layer; s3, model hyper-parameters are selected; s4, network training: performing parameter iteration training through optimization calculation and loss calculation; s5, establishing a dynamic prediction model: continuously inputting measured data, and comparing the measured data with a mean square error standard value to judge whether parameters are retrained or not; and S6, early warning judgment: calculating a difference value between a predicted value and a measured value of the LSTM model, comparing the difference value with a preset threshold value, and carrying out graded early warning. Compared with the prior art, the method has the advantages that the prediction curve is more stable, the calculation standard deviation is larger, the prediction confidence interval range is wider, the change of the data trend can be predicted in advance, the change of the data trend can be identified in advance, and the early warning advancement is better.

Description

Technical Field

[0001] The invention relates to the technical field of tunnel engineering monitoring, and in particular to a tunnel abnormal data early warning method based on an LSTM model. Background Art

[0002] Tunnels are important infrastructure and key control projects for traffic operation, and their operation and maintenance are crucial during their service. With the advancement of digital transformation of tunnel operation and maintenance, data-driven anomaly detection and early warning methods have become a research hotspot.

[0003] LSTM comes from the RNN model. RNN is a classic network model for processing serialized input problems in deep learning. Its core is to use the historical information of the input sequence to process the sequence number. In traditional RNN, due to the calculation problems of gradient disappearance and explosion in the time back propagation algorithm, the network can only learn the impact of recent sequences on current values, and it is very difficult to learn the long-term dependency of long-term sequences. LSTM is an improved version of the RNN network, which solves the training problem of the traditional RNN model and is widely used in the modeling and prediction of time series.

[0004] Current technical means may have the risk of missing detection when processing more complex long-term data. Usually, warnings are issued just after an anomaly occurs, and the identification of changes in data trends is poor. Summary of the invention

[0005] In order to solve the problems raised in the above background technology, the object of the present invention is to provide a tunnel abnormal data early warning method based on LSTM model (Long Short-Term Memory, long short-term memory network), comprising the following steps:

[0006] S1, data preprocessing;

[0007] S2. Build the model: The model structure is a two-layer LSTM network and a fully connected (Dense) layer;

[0008] S3. Select model hyperparameters;

[0009] S4, network training: perform parameter iterative training through optimization calculation and loss calculation;

[0010] S5. Establish a dynamic prediction model: continuously input measured data and compare it with the standard value of mean square error to determine whether to retrain parameters;

[0011] S6. Warning judgment: Calculate the difference between the LSTM model prediction value and the measured value, and make a graded warning after comparing the difference with the preset threshold.

[0012] The step S1 includes time series difference transformation, sample division and training sample construction;

[0013] The step S1 specifically includes the following steps:

[0014] S11. Obtain the time series and perform differential analysis on the original time series. The differential is defined as the difference between the last item and the first item of the original time series, that is, X′(t)=X(t+1)-X(t);

[0015] S12. Obtain sample data, and divide the sample data into a training set and a test set in a ratio of 5:1.

[0016] S13, construct training samples and process the time series into a form that can be learned by the neural network. The input sequence of the training sample X t is the sequence value within the time window, and the training label Y is the sequence value of the next second in the time window.

[0017] The LSTM network of step S2 includes a cell state that tracks the information content of the previous node, a forget gate, a memory gate, and an output gate that control the input and output of information. When the cell state progresses from time t-1 to time t, the input value at the current moment and the output of the model at the previous moment are added or removed to the cell state at the previous moment through the forget gate and the memory gate, completing an update of the cell state. The calculation formula in the LSTM network includes the following:

[0018] (1) Setting

[0019] (2) Forget gate: Γ f =σ(W f [X t ,h t-1 ]+b f );

[0020] (3) Memory Gate:

[0021] (4) Output gate: Γ o =σ(W o [X t ,h t-1 ]+b o );

[0022] (5) Cell state: c t =tanh(W c [X t ,h t-1 ]+b c ), c t =Γ u *c t +Γ f *ct-1 ;

[0023] (6) Output value: h t =Γ o *c t .

[0024] Among them, tanh(*) represents the hyperbolic tangent function, σ(*) represents the sigmoid function, and c t is the memory cell state, h t is the hidden state at time t, h t-1 is the hidden state at the moment before time t, W f , W u , W o , W c is the correlation weight matrix, b f , b u , b o , b c is the relevant bias term.

[0025] The step S3 specifically comprises the following steps:

[0026] S21. Set up a candidate set for the number of neurons in each layer of the network and the input time window, and search for hyperparameters in the network;

[0027] S22, cycle the model under each parameter combination in the candidate set ten times, and take the ten averages of the mean absolute error on the test set as the measurement standard;

[0028] S23. Select the hyperparameter combination by comparing the measurement criteria, that is, the one with the smallest ten-fold mean of the mean absolute error of the test set is the optimal hyperparameter combination.

[0029] The step S4 specifically comprises the following steps:

[0030] S41, set random numbers, and perform iterative training on parameters through optimization calculation and loss calculation. The optimization calculation uses the adaptive moment estimation algorithm (Adam algorithm), and the loss calculation uses the mean squared error (MSE);

[0031] S42, the network is repeatedly trained 30 times, and the distribution of the mean square error on the test set is observed;

[0032] S43, determine the test set error, if the mean square error is greater than 2*10 -8 , then returns to step S41, otherwise the network training is completed.

[0033] The specific process of step S5 is: set the interval of model update to one hour, retrain the LSTM model every hour based on the data of the previous hour, and check whether it exceeds the mean square error standard 2*10 -8 , if it exceeds, retrain the LSTM model.

[0034] Described step S6 specifically comprises the following steps:

[0035] S61. The standard deviation of the data in the previous hour is used as the benchmark for the fluctuation range of the monitoring data, and 5.5 times, 7 times and 8 times the standard deviation are taken as thresholds to determine abnormal points.

[0036] S62. Calculate the error between the measured value and the predicted value. If the error is greater than a certain graded dynamic threshold, output the corresponding graded warning result. Otherwise, store the monitoring data and the error.

[0037] Compared with the prior art, the present invention has a smoother prediction curve, a larger calculated standard deviation, a wider prediction confidence interval, can predict changes in data trends in advance, can identify changes in data trends one step earlier, and has better advance warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0039] Figure 2 The concrete strain data sample division in the embodiment;

[0040] Figure 3 The training sample format in the embodiment;

[0041] Figure 4 This is a structural diagram of the concrete strain LSTM model in the embodiment;

[0042] Figure 5 is the training set error distribution in the embodiment;

[0043] Figure 6 The loss function (MSE) of the iterative process in the embodiment changes;

[0044] Figure 7 The LSTM dynamic modeling error sequence in the embodiment;

[0045] Figure 8 The error of the dynamic LSTM model in the embodiment changes over time;

[0046] Fig. 9 These are abnormal points monitored at different threshold levels in the embodiment. DETAILED DESCRIPTION

[0047] The present invention is further described below with reference to the accompanying drawings and specific embodiments.

[0048] Example

[0049] like Figure 1 As shown, a tunnel abnormal data warning method based on the LSTM model includes the following steps:

[0050] 1. Data preprocessing;

[0051] 2. Build the model: The model structure is a two-layer LSTM network and a fully connected (Dense) layer;

[0052] 3. Select model hyperparameters;

[0053] 4. Network training: Iterative parameter training is performed through optimization calculation and loss calculation;

[0054] 5. Establish a dynamic prediction model: continuously input measured data and compare it with the standard value of mean square error to determine whether to retrain parameters;

[0055] 6. Warning judgment: Calculate the difference between the LSTM model prediction value and the measured value, and then compare the difference with the preset threshold to make a graded warning.

[0056] Step 1 specifically includes the following steps:

[0057] S11. Obtain the time series and perform differential analysis on the original time series. The differential is defined as the difference between the last item and the first item of the original time series, that is, X′(t)=X(t+1)-X(t);

[0058] S12. Obtain sample data. Since the network parameters are iteratively optimized based on the error between the predicted value and the actual value of the training set data, the error of the model on the training set is often lower than the error of the model on the entire data set. Divide the sample data into a training set and a test set. Considering the model training time and prediction accuracy, after debugging, the first 3000 seconds of data are used as the training set, and the last 600 seconds of data are used as the test set. The division results are as follows: Figure 2 shown.

[0059] S13, construct training samples and process the time series into a form that can be learned by the neural network. The input sequence X of the training sample t is the sequence value in the time window, and the training label Y is the sequence value of the next second in the time window. The input sequence of the training sample is the sequence value X in the time window t, the training label is the sequence value Y of the next second in the time window. You can build multiple moving time windows, and use the values ​​of each window except the last value as the input sequence, and the last value as the training label. At each time step, move the time window to obtain a new input sequence and target value one step forward, thus forming a series of training data and training labels. An example of this process is shown in the figure below. Figure 3 shown.

[0060] The LSTM model structure of step 2 is a two-layer LSTM network and a fully connected (Dense) layer. The structure diagram is as follows: Figure 4 As shown. The LSTM network includes a cell state that tracks the information content of the previous node, a forget gate, a memory gate, and an output gate that control the input and output of information. When the cell state progresses from time t-1 to time t, the input value at the current moment and the output of the model at the previous moment are added or removed to the cell state at the previous moment through the forget gate and the memory gate, completing an update of the cell state. The calculation formula in the LSTM network includes the following:

[0061] (1) Setting

[0062] (2) Forget gate: Γ f =σ(W f [X t ,h t-1 ]+b f );

[0063] (3) Memory Gate:

[0064] (4) Output gate: Γ o =σ(W o [X t ,h t-1 ]+b o );

[0065] (5) Cell state: c t =tanh(W c [X t ,h t-1 ]+b c ), c t =Γ u *c t +Γ f *c t-1 ;

[0066] (6) Output value: h t =Γ o *c t .

[0067] Among them, tanh(*) represents the hyperbolic tangent function, σ(*) represents the sigmoid function, and c t is the memory cell state, h t is the hidden state at time t, h t-1 is the hidden state at the moment before time t, W f , W u , W o , W c is the correlation weight matrix, b f , b u , b o , b c is the relevant bias term.

[0068] Specifically, in the first LSTM layer, the dimension of the input data at this moment is (1*1), the dimension of the state cell at the previous moment is the number of neurons in the first layer (16*1), and the combined input dimension of the two is (17*1); the dimension of the output data is the number of neurons in the first layer (16*1); therefore, the dimension of the neuron to be trained is the product of the input dimension and the output dimension, i.e. (17*16) dimension, plus the dimension of the bias, i.e. (1*16) dimension, for a total of (18*16) dimensional vectors; a single neuron has a cell state and three gate functions that need to be weighted, so the estimated training parameter quantity of the first LSTM layer is (18*16*4) dimension, i.e. 1152 parameters. The input of the second LSTM layer at this moment is the output of the first LSTM layer, with a dimension of (16*1). The calculation method of the remaining parameter dimensions is similar to that of the first layer, and a total of (4*((16+8)*8+8)) or 800 parameters can be inferred. The input of the Dense layer is the (8*1) dimension of the second layer output, and the output is the dimension of the label (1*1), plus the dimension of the deviation, a total of 9 parameters need to be trained. In this embodiment, it is estimated that a total of 1961 parameters need to be trained.

[0069] Step three includes the following steps:

[0070] S21. Set up a candidate set for the number of neurons in each layer of the network and the input time window, and search for hyperparameters in the network;

[0071] S22, cycle the model under each parameter combination in the candidate set ten times, and take the ten averages of the mean absolute error on the test set as the measurement standard;

[0072] S23. Select the hyperparameter combination by comparing the measurement criteria, that is, the one with the smallest ten-fold mean of the mean absolute error of the test set is the optimal hyperparameter combination.

[0073] Specifically, in order to avoid the influence of random initialization weights and random gradient descent factors on the model quality, this embodiment repeatedly trains the network 30 times and observes the distribution of MSE errors on its test set. Figure 5 As shown in the figure, there is a long tail on the right side, which means that the error of most random models is small, but there are also a few cases where the error is significantly greater than the average error. Considering the training time and model quality, the training evaluation standard of the model is set to MSE < 2*10 -8 When the test set error is greater than this value, the random number will be set again and the model will be trained until the model meets the error requirement, and the network training is considered complete.

[0074] Step 4 includes the following steps:

[0075] S41, set random numbers, and perform iterative training of parameters through optimization calculation and loss calculation. The optimization calculation uses the adaptive moment estimation algorithm (Adam algorithm), and the loss calculation uses the mean squared error (MSE). In this embodiment, the batch size of a single iteration is set to 50, the number of iterations is 50, and the training set and the test set are iterated 50 times. Figure 6 It can be seen that the MSE changes during the iteration process. After about 30 times, the test set error has stabilized and is close to the training error, indicating that the model has no underfitting or overfitting problems. The number of iterations can be set to 50.

[0076] S42, the network is repeatedly trained 30 times, and the distribution of the mean square error on the test set is observed;

[0077] S43, determine the test set error, if the mean square error is greater than 2*10 -8 , then returns to step S41, otherwise the network training is completed.

[0078] Step 5: Set the model update interval to one hour, retrain the LSTM model every hour based on the data from the previous hour, and check whether it exceeds the mean square error standard of 2*10 -8 , if it exceeds, retrain the LSTM model.

[0079] Specifically, the process of dynamic LSTM modeling is as follows Figure 7 As shown in the figure, the change of the error of the dynamic LSTM model over time in this embodiment is as follows Figure 8 As shown in the figure, due to the large amount of data required for training the network, the denoised data of concrete strain of a certain project on a certain day and the denoised data of the previous day are added for training. It can be seen that the absolute value of the error in the figure gradually decreases in the early stage, and then fluctuates within a narrow range in the later stage. This shows that the neural network continuously learns the characteristics of the data in the long term and can continuously optimize itself in the case of large samples.

[0080] Step six includes the following steps:

[0081] S61. The standard deviation of the data in the previous hour is used as the fluctuation range benchmark of the monitoring data. 5.5 times, 7 times and 8 times the standard deviation are used as thresholds to determine abnormal points. The abnormal points monitored under different threshold levels are as follows: Fig. 9 shown.

[0082] S62. Calculate the error between the measured value and the predicted value. If the error is greater than a certain graded dynamic threshold, output the corresponding graded warning result. Otherwise, store the monitoring data and the error.

Claims

1. A tunnel abnormal data early warning method based on LSTM model, characterized by: The following steps are involved: S1, data preprocessing; S2. Build the model: The model structure is a two-layer LSTM network and a fully connected (Dense) layer; S3. Select model hyperparameters; S4, network training: perform parameter iterative training through optimization calculation and loss calculation; S5. Establish a dynamic prediction model: continuously input measured data and compare it with the standard value of mean square error to determine whether to retrain parameters; S6. Warning judgment: Calculate the difference between the LSTM model prediction value and the measured value, and make a graded warning after comparing the difference with the preset threshold.

2. According to a tunnel abnormal data early warning method based on LSTM model according to claim 1, it is characterized in that: The step S1 includes time series difference transformation, sample division and training sample construction; the step S1 specifically includes the following steps: S11. Obtain the time series and perform differential analysis on the original time series. The differential is defined as the difference between the last item and the first item of the original time series, that is, X′(t)=X(t+1)-X(t); S12. Obtain sample data, and divide the sample data into a training set and a test set in a ratio of 5:

1. S13, construct training samples and process the time series into a form that can be learned by the neural network. The input sequence X of the training sample t is the sequence value within the time window, and the training label Y is the sequence value of the next second in the time window.

3. According to a tunnel abnormal data early warning method based on LSTM model according to claim 1, it is characterized in that: The LSTM network of step S2 includes a cell state that tracks the information content of the previous node, a forget gate, a memory gate, and an output gate that control the input and output of information. When the cell state progresses from time t-1 to time t, the input value at the current moment and the output of the model at the previous moment are added or removed to the cell state at the previous moment through the forget gate and the memory gate, completing a z update of the cell state. The calculation formula in the LSTM network includes the following: (1) Setting (2) Forget gate: Γ f =σ(W f [X t ,h t-1 ]+b f ); (3) Memory gate: Γ u =σ(W u [X t ,h t-1 ]+b u ); (4) Output gate: Γ o =σ(W o [X t ,h t-1 ]+b o ); (5) Cell state: c t =tanh(W c [X t ,h t-1 ]+b c ), c t =Γ u *c t +Γ f *c t-1 ; (6) Output value: h t =Γ o *c t . Among them, tanh(*) represents the hyperbolic tangent function, σ(*) represents the sigmoid function, and c t is the memory cell state, h t is the hidden state at time t, h t-1 is the hidden state at the moment before time t, W f , W u , W o , W c is the correlation weight matrix, b f , b u , b o , b c is the relevant bias term.

4. The tunnel abnormal data early warning method based on the LSTM model according to claim 1 is characterized in that: The step S3 specifically comprises the following steps: S21. Set up a candidate set for the number of neurons in each layer of the network and the input time window, and search for hyperparameters in the network; S22, cycle the model under each parameter combination in the candidate set ten times, and take the ten averages of the mean absolute error on the test set as the measurement standard; S23. Select the hyperparameter combination by comparing the measurement criteria, that is, the one with the smallest ten-fold mean of the mean absolute error of the test set is the optimal hyperparameter combination.

5. The tunnel abnormal data early warning method based on the LSTM model according to claim 1 is characterized in that: The step S4 specifically comprises the following steps: S41, set random numbers, and perform iterative training on parameters through optimization calculation and loss calculation. The optimization calculation uses the adaptive moment estimation algorithm (Adam algorithm), and the loss calculation uses the mean squared error (MSE); S42, the network is repeatedly trained 30 times, and the distribution of the mean square error on the test set is observed; S43, determine the test set error, if the mean square error is greater than 2*10 -8 , then returns to step S41, otherwise the network training is completed.

6. The method for early warning of abnormal tunnel data based on LSTM model according to claim 1, characterized in that: The specific process of step S5 is: set the interval of model update to one hour, retrain the LSTM model every hour based on the data of the previous hour, and check whether it exceeds the mean square error standard 2*10 -8 , if it exceeds, retrain the LSTM model.

7. The method for early warning of tunnel abnormal data based on LSTM model according to claim 1, characterized in that: Described step S6 specifically comprises the following steps: S61. The standard deviation of the data in the previous hour is used as the benchmark for the fluctuation range of the monitoring data, and 5.5 times, 7 times and 8 times the standard deviation are taken as thresholds to determine abnormal points. S62. Calculate the error between the measured value and the predicted value. If the error is greater than a certain graded dynamic threshold, output the corresponding graded warning result. Otherwise, store the monitoring data and the error.

Images