Method and device for realizing settlement prediction based on ground surface settlement data of intelligent recovery

Abstract

A method and a device for realizing settlement prediction based on intelligent recovery ground surface settlement data relate to the field of settlement prediction, and comprise the following steps: acquiring observation data of continuous settlement in a certain period of time, and defining a mask matrix to mark information of missing data positions in the observation data; constructing a multi-channel settlement data repair deep learning model comprising a generative network and an adversarial network, and inputting the incomplete data matrix, the mask matrix and the random noise matrix into the generative network to obtain a recovered complete settlement data matrix; generating a prompt matrix based on the mask matrix, and inputting the complete settlement data matrix and the prompt matrix into an adversarial network to obtain a numerical value true and false probability matrix in the complete settlement data matrix; and constructing a settlement prediction model based on an MHSAM-LSTM deep learning model, performing settlement prediction on the settlement prediction model input by the complete settlement data matrix, and outputting a prediction result, so that relatively high prediction precision can be realized, and a more excellent prediction effect is achieved.

Description

Technical Field

[0001] The present invention relates to the field of settlement prediction, and in particular to a method and device for realizing settlement prediction based on intelligently restored surface settlement data. Background Art

[0002] Due to the limited space in subway construction, shield tunneling operations inevitably cause damage to sensors or data collectors installed on site, resulting in data loss or even sensor failure. Even if the corresponding settlement monitoring device is installed on the surface, exposure to the external environment will also cause data incompleteness or data loss. At present, most surface settlement prediction models established using artificial intelligence methods use complete data sets, but it is very difficult to obtain complete sensor data during the actual collection process. Incomplete or missing data is inevitable, which requires further analysis and consideration.

[0003] The most important thing is that data loss of data collectors is a random process, especially when all sensors have data loss, which greatly increases the difficulty of data recovery. Most existing studies still need to use some normal sensors to restore the complete data set through related technologies, and few studies have studied data recovery when all sensors have data loss.

[0004] In the past decade, most studies have focused on predicting the maximum surface settlement induced by shield tunneling. However, shield tunneling is a continuous process, and only predicting the maximum surface settlement cannot reflect the complex internal relationship between the shield and the stratum. Therefore, it is necessary to conduct in-depth research on the development process of surface settlement during shield tunneling. In recent years, scholars have begun to focus on the prediction of the development process of surface settlement. At present, the most widely used is the long short term memory (LSTM) neural network model, which is a variant structure of the recurrent neural network and can capture relevant information in time series tasks, but it may be ineffective for long sequence data and cannot capture the global features between data. Summary of the invention

[0005] The main purpose of the present invention is to overcome the above-mentioned defects in the prior art, and propose a method and device for realizing settlement prediction based on intelligently restored surface settlement data, which jointly restores multi-channel incomplete settlement data sets to generate a complete data set. Then, the prediction model is established using the restored complete data set, and the LSTM neural network model is improved by using the Multi-head Self-attention Mechanism (MHSAM) to improve its prediction accuracy of shield-induced surface settlement.

[0006] The present invention adopts the following technical solution:

[0007] A method for realizing settlement prediction based on intelligently restored surface settlement data.

[0008] Obtaining observation data of continuous settlement over a period of time, wherein the observation data is an incomplete surface settlement data matrix with certain random missing points, defining a mask matrix to mark the information of the missing data positions in the observation data, and using the mask matrix to simulate the positions of the missing points to generate an incomplete data matrix;

[0009] Constructing a multi-channel sedimentation data patching deep learning model, the multi-channel sedimentation data patching deep learning model includes a generative network and an adversarial network, inputting the incomplete data matrix, the mask matrix and the random noise matrix into the generative network to obtain a restored complete sedimentation data matrix; generating a prompt matrix based on the mask matrix, inputting the complete sedimentation data matrix and the prompt matrix into the adversarial network to obtain a probability matrix of true and false values ​​in the complete sedimentation data matrix;

[0010] Construct a settlement prediction model based on the MHSAM-LSTM deep learning model, and perform settlement prediction on the settlement prediction model using the complete settlement data matrix that meets the requirements according to the probability matrix of true and false values ​​in the complete settlement data matrix, and output the prediction result.

[0011] Monitoring points are arranged above the shield tunnel, and a fiber Bragg grating data collector is used to obtain observation data of continuous settlement over a period of time.

[0012] The generation network includes a first input layer, a first feature extraction module, a first pooling layer, a first self-attention layer, a second feature extraction module, a second self-attention layer, a first upsampling layer, a third feature extraction module and a first output layer; the incomplete data matrix, the mask matrix and the random noise matrix are input into the first feature extraction module via the first input layer for feature extraction and output of a first transition matrix; the first transition matrix is ​​respectively passed to the first pooling layer and the first upsampling layer; the first transition matrix is ​​input into the first self-attention layer via the first pooling layer, and then passed to the second feature extraction module for feature extraction and output of a second transition matrix; the first transition matrix and the second transition matrix are input into the first upsampling layer and then passed to the third feature extraction module for feature extraction, and the restored complete sedimentation data matrix is ​​output from the first output layer.

[0013] The first feature extraction module includes a first convolutional layer, a first densely connected layer, a first normalization layer, a second convolutional layer and a second normalization layer arranged in sequence; the second feature extraction module includes a third convolutional layer, a second densely connected layer, a third normalization layer, a fourth convolutional layer and a fourth normalization layer arranged in sequence; the third feature extraction module includes a fifth convolutional layer and a fifth batch normalization layer arranged in sequence; the first densely connected layer and the second densely connected layer respectively include four convolutional layers arranged in sequence, and the feature output of the previous convolutional layer is passed and used as the input of the next convolutional layer.

[0014] The adversarial network includes a second input layer, a fourth feature extraction module, a second pooling layer, a fifth feature extraction module, a second upsampling layer, a sixth feature extraction module and a second output layer; the complete sedimentation data matrix and the prompt matrix are input into the fourth feature extraction module via the second input layer for feature extraction and outputting a third transition matrix; the third transition matrix is ​​respectively passed to the second pooling layer and the second upsampling layer; the third transition matrix is ​​input into the fifth feature extraction module via the second pooling layer for feature extraction and outputting a fourth transition matrix; the third transition matrix and the fourth transition matrix are input into the second upsampling layer and then passed to the sixth feature extraction module for feature extraction, and the probability matrix is ​​obtained from the output of the second output layer.

[0015] The fourth feature extraction module includes a first convolutional layer, a first densely connected layer, a first normalization layer, a second convolutional layer and a second normalization layer arranged in sequence; the fifth feature extraction module includes a second densely connected layer, a third normalization layer, a third convolutional layer, a third densely connected layer, a fourth normalization layer and a fourth convolutional layer arranged in sequence; the sixth feature extraction module includes a fifth convolutional layer and a fifth batch normalization layer arranged in sequence; the first densely connected layer, the second densely connected layer and the third densely connected layer respectively include four convolutional layers, and the feature output of each convolutional layer is passed and used as the input of the subsequent convolutional layer.

[0016] The generative network includes binary cross entropy loss and RMSE loss, and its loss function is:

[0017] ;

[0018] The loss function of the adversarial network is:

[0019] ;

[0020] in, represents the minimization objective function, and represents the objective function, Represents a variable , , expectations, is the complete settlement data matrix; is an incomplete data matrix; is the mask matrix; For Hadamard; is the prompt matrix; D is the adversarial network operation; Represents a hyperparameter.

[0021] The data set consisting of the complete settlement data matrix is ​​divided into a training set, a validation set and a test set to train, validate and test the settlement prediction model to determine the optimal hyperparameter combination; the settlement prediction model includes three LSTM layers and two MHSAM layers, the LSTM layer is used to capture the specific features of the complete settlement data matrix, and the MHSAM layer is arranged between two adjacent LSTM layers to capture global features and enrich the features of the captured data from different angles; the settlement prediction model outputs prediction results and an evaluation of the prediction results, and the evaluation includes mean square error, mean absolute error and correlation coefficient.

[0022] The LSTM layer includes a forget gate, an update gate, an input gate, a cell unit and an output gate, as follows:

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] ;

[0029] -c <t>< / t> Indicates the memory cells involved in the calculation at the current moment; c <t-1>< / t-1> is the memory cell of the next moment; <t>< / t> is the hidden state at time t; x <t>< / t> 、h <t -1> 、c <t -1> is the vector input to the LSTM layer; a <t>< / t> and c <t>< / t> represents the output vector; b (b f , b u , b c ,b o ) represents the corresponding bias values ​​of the forget gate, update gate, cell unit and output gate; W (Wf , W u ,W c , W o ) represents the corresponding weight values ​​of the forget gate, update gate, cell unit and output gate; σ and tanh are sigmoid activation function and tanh activation function respectively; Γ f is the forget gate, Γ u is the update gate, Γ o is the output gate;

[0030] The MHSAM layer includes Q, K and K, which represent query, key function and value respectively, as follows:

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] Where X is the input sequence of length N, and the matrix X∈R N×d represents, d represents the dimension of each position in the sequence, d k represents the dimension of the kth key; Wq, Wk, Wv are the weight matrices corresponding to Q, K, V; A is the attention score, softmax represents the normalization function; Head[.] represents the multi-head self-attention operation; Attention(X) is the output; Concat is the merging of multiple multi-head self-attentions.

[0037] A device for realizing settlement prediction based on intelligently restored surface settlement data, comprising:

[0038] A data acquisition module is used to acquire observation data of continuous settlement over a period of time, wherein the observation data is an incomplete surface settlement data matrix with certain random missing data, and a mask matrix is ​​defined to mark the information of the missing data positions in the observation data, and the positions of the missing points are simulated by the mask matrix to generate an incomplete data matrix;

[0039] A data patching module is used to construct a multi-channel sedimentation data patching deep learning model, wherein the multi-channel sedimentation data patching deep learning model includes a generative network and an adversarial network, wherein the incomplete data matrix, the mask matrix and the random noise matrix are input into the generative network to obtain a restored complete sedimentation data matrix; a prompt matrix is ​​generated based on the mask matrix, and the complete sedimentation data matrix and the prompt matrix are input into the adversarial network to obtain a probability matrix of true and false values ​​in the complete sedimentation data matrix;

[0040] The settlement prediction module constructs a settlement prediction model based on the MHSAM-LSTM deep learning model, and performs settlement prediction on the settlement prediction model inputted with the complete settlement data matrix that meets the requirements according to the probability matrix of the true and false values ​​in the complete settlement data matrix, and outputs the prediction result.

[0041] It can be seen from the above description of the present invention that, compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) The multi-channel settlement data patching deep learning model DCL-SAM-GAIN proposed in the present invention addresses the common problem of random missing data and reconstructs the complete response of failed settlement observations in the case of incomplete data sets.

[0043] (2) The generative network in the GAIN of the present invention uses a convolutional network and combines SAM to extract data correlation features, and introduces dense connection technology to promote information flow between layers and improve network parameter utilization. The addition of normalization technology in GAIN is conducive to the preservation of underlying details and can effectively alleviate the problem of gradient disappearance. By comparing the relative errors of the GAIN and DCL-SAM-GAIN models for the restored data set, it is proved that the DCL-SAM-GAIN in the present invention is effective for missing data recovery.

[0044] (3) Compared with the self-attention mechanism SAM, MHSAM can integrate information of multiple time scales and process data in parallel through multiple attention heads, thereby improving the prediction effect of the model. By comparing the LSTM, MHSAM-LSTM and SAM-LSTM prediction models, it is found that the MHSAM-LSTM model has better prediction effect than the other two models.

[0045] (4) The settlement data recovery and prediction model of DCL-SAM-GAIN and MHSAM-LSTM proposed in the present invention is based on reality and fully considers the situation that settlement data is incomplete or missing in actual projects. Based on this technology, a joint repair method of multi-channel data sets is developed, and settlement prediction is performed on the restored complete data set through MHSAM-LSTM, achieving a high prediction accuracy. Compared with the traditional research that mostly uses complete data sets for settlement prediction, it is more in line with actual application scenarios, and at the same time, it has a better prediction effect than the LSTM model improved by the self-attention mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a diagram of the architecture of the multi-channel sedimentation data patching deep learning model of the present invention;

[0047] Figure 2 It is a densely connected layer structure diagram;

[0048] Figure 3 Data feature processing for the MHSAM layer;

[0049] Figure 4 It is the settlement prediction model of MHSAM-LSTM deep learning model.

[0050] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0051] The present invention is further described below through specific implementation modes.

[0052] A method for realizing settlement prediction based on intelligently restored surface settlement data comprises the following steps:

[0053] S1 obtains the observation data of continuous settlement over a period of time. The observation data is an incomplete surface settlement data matrix with certain random missing points. A mask matrix is ​​defined to mark the information of the missing data positions in the observation data. The mask matrix is ​​used to simulate the positions of the missing points to generate an incomplete data matrix.

[0054] In this embodiment, monitoring points are arranged above the shield tunnel to form a surface settlement observation system, and a fiber Bragg grating data collector is used to obtain observation data of continuous settlement over a period of time.

[0055] S2 constructs a multi-channel sedimentation data patching deep learning model, which includes a generative network and an adversarial network. The incomplete data matrix, mask matrix and random noise matrix are input into the generative network to obtain the restored complete sedimentation data matrix; the prompt matrix is ​​generated based on the mask matrix, and the complete sedimentation data matrix and the prompt matrix are input into the adversarial network to obtain the probability matrix of the true and false values ​​in the complete sedimentation data matrix. The probability matrix is ​​used to judge whether the pseudo data generated by the generative network conforms to the distribution of the real data, and the generation result of the generator is represented by probability.

[0056] See also Figure 1 , the generation network includes a first input layer, a first feature extraction module, a first pooling layer, a first self-attention layer, a second feature extraction module, a second self-attention layer, a first upsampling layer, a third feature extraction module and a first output layer. The incomplete data matrix, mask matrix and random noise matrix are input into the first feature extraction module through the first input layer for feature extraction and output of the first transition matrix; the first transition matrix is ​​respectively passed to the first pooling layer and the first upsampling layer; the first transition matrix is ​​input into the first self-attention layer through the first pooling layer, and then passed to the second feature extraction module for feature extraction and output of the second transition matrix; the first transition matrix and the second transition matrix are input into the first upsampling layer and then passed to the third feature extraction module for feature extraction, and the restored complete sedimentation data matrix is ​​output from the first output layer. The size of the matrix output by the first output layer is consistent with the size of the matrix input by the first input layer.

[0057] The first feature extraction module includes a first convolutional layer, a first densely connected layer, a first normalization layer, a second convolutional layer, and a second normalization layer arranged in sequence. The second feature extraction module includes a third convolutional layer, a second densely connected layer, a third normalization layer, a fourth convolutional layer, and a fourth normalization layer arranged in sequence. The third feature extraction module includes a fifth convolutional layer and a fifth batch normalization layer arranged in sequence.

[0058] The adversarial network includes a second input layer, a fourth feature extraction module, a second pooling layer, a fifth feature extraction module, a second upsampling layer, a sixth feature extraction module and a second output layer; the complete sedimentation data matrix and the prompt matrix are input into the fourth feature extraction module through the second input layer for feature extraction and output of the third transition matrix; the third transition matrix is ​​respectively transmitted to the second pooling layer and the second upsampling layer; the third transition matrix is ​​input into the fifth feature extraction module through the second pooling layer for feature extraction and output of the fourth transition matrix; the third transition matrix and the fourth transition matrix are input into the second upsampling layer and then into the sixth feature extraction module for feature extraction, and the probability matrix is ​​output from the second output layer. The input complete sedimentation data matrix contains the data matrix of continuous sedimentation of multiple data collectors for a period of time, and the probability matrix corresponds to the probability value of the complete sedimentation data of each data collector.

[0059] The composition of the fourth feature extraction module may be the same as that of the first feature extraction module, and the fourth feature extraction module includes a first convolutional layer, a first densely connected layer, a first normalization layer, a second convolutional layer, and a second normalization layer arranged in sequence. The fifth feature extraction module includes a second densely connected layer, a third normalization layer, a third convolutional layer, a third densely connected layer, a fourth normalization layer, and a fourth convolutional layer arranged in sequence. The composition of the sixth feature extraction module may be the same as that of the third feature extraction module, including a fifth convolutional layer and a fifth batch normalization layer arranged in sequence.

[0060] In this embodiment, the first densely connected layer, the second densely connected layer and the third densely connected layer include four convolutional layers respectively, and the feature output of each convolutional layer is transmitted and used as the input of the subsequent convolutional layer.

[0061] In this embodiment, each layer is described as follows:

[0062] (1) Convolutional layer (Conv): By sliding the convolution kernel on the data, the local optimal features of the input data in the time domain and frequency domain can be extracted, and the original data can be converted into output data with a higher level of abstract feature representation.

[0063] (2) Pooling layer (PL): Downsample the input data to reduce the dimension, reduce computational complexity and help prevent the model from overfitting the training data. The present invention uses the maximum pooling technique to reduce the dimension of the feature.

[0064] (3) Self-attention layer: By calculating the weights between different positions, the model can establish global correlation when processing long sequences and capture the dependencies between different parts of the sequence.

[0065] (4) Batch Normalization Layer (BNL): accelerates the training convergence of the model, reduces the problem of gradient vanishing or exploding, and enhances the generalization ability and robustness of the model.

[0066] (5) Upsampling layer (USL): By interpolating or repeating the input data, the receptive field of the model is improved, thereby increasing the representation and generalization capabilities of the model.

[0067] (6) Activation function: Add activation function after convolution layer and batch normalization layer, and the activation function of hidden layer is ReLU function:

[0068] ;

[0069] in represents the output of the ReLU function, is the input of the ReLU function, when When that is itself. , the output of the ReLU function is 0.

[0070] (7) Densely connected layer (DCL): The main feature is to establish dense connections between all previous layers and the current layer, achieve feature reuse through feature connections in the channel dimension, effectively utilize shallow features, and reduce network parameters to a certain extent, preventing problems such as gradient vanishing or gradient explosion. By introducing a densely connected layer (composed of four convolutional layers), the feature output of each layer is passed and used as the input of the subsequent convolutional layer, and the number of parameters is reduced by sharing shallow and deep features, increasing the network's ability to extract hidden high-level features, effectively preventing problems such as gradient vanishing or gradient explosion.

[0071] In this embodiment, the loss function is mainly used to measure the error between the output result and the target, and the network updates the parameters of the network through back propagation to make the network output close to the target value. The multi-channel sedimentation data repair deep learning model DCL-SAM-GAIN constructed in this embodiment uses a binary cross entropy function to train the network.

[0072] Specifically, the loss of the generated network includes binary cross entropy loss and RMSE loss. The binary cross entropy loss is used to calculate the loss between the generated network and the adversarial network, thereby optimizing the network structure. The RMSE loss is to ensure that the data in the non-missing part is as close to the real data as possible, and to ensure that the hidden layer can effectively learn to capture the main information in the original observation signal to better generate real data. The details are as follows:

[0073] ;

[0074] The adversarial network outputs 1 for real data in the input and close to 0 for pseudo data (data interpolated by the generated network). Its loss function is:

[0075] ;

[0076] in, represents the minimization objective function, and represents the objective function, Represents a variable , , expectations, is the complete settlement data matrix; is an incomplete data matrix; is the mask matrix; For Hadamard; is the prompt matrix; D is the adversarial network operation; Represents a hyperparameter.

[0077] The multi-channel sedimentation data repair deep learning model in this embodiment adopts a convolutional neural network architecture (Convolution Neural Network, CNN). An up-sampling module (Up Sample Layer, USL) is set to decompress the compressed high-level features and restore their dimensions. The introduction of batch normalization technology (Batch Norm Layer, BNL) is conducive to the preservation of underlying details and effectively alleviates the problem of gradient disappearance, thereby speeding up the training process of the network. The introduction of the self-attention mechanism (Self-attention Mechanism, SAM) enables the network to capture more important elements and generate more realistic sedimentation data. The introduction of a dense connection layer (DenseConnectionLayer, DCL) promotes information exchange between network structure layers and effectively utilizes the parameters of the network, greatly improving the network's ability to extract features from data sets.

[0078] In this embodiment, the multi-channel sedimentation data repair deep learning model performs data intelligent recovery implementation process as follows:

[0079] The input of the generative network is an incomplete surface settlement data matrix with some random missing , the corresponding mask matrix and the random noise matrix The output of the generation network is the restored complete sedimentation data matrix The final output of the network is the complete sedimentation data matrix It consists of the missing data generated by the generator and the real data of the original data.

[0080] Define the input matrix X: There are x settlement data collectors represented by X={X 1 ,X 2 ,X 3 …,X i …,X n}, where X i represents the time series data of the i-th collector. The sequence data of each collector is expressed as, X i ={x 1i ,x 2i ,x 3i …,x ti} T Where i represents the collector number, t represents the time, and x represents the ti It is expressed as the surface settlement data value of the i-th collector at time t.

[0081] Define the mask matrix M: mark the missing data location information in the data set, the data set that is not missing is consistent with the original data, and the data values ​​of the missing part are marked as non-existent or missing. If m=1, then the data x ti No missing, if m=0, then the data x ti There are some missing parts.

[0082] Defining an incomplete data matrix : Use the mask matrix to simulate the position of missing points and generate an incomplete data matrix ,as follows:

[0083] ;

[0084] Where X represents the complete data set, i.e. the input matrix, and M represents the mask matrix. Represents Hadamard.

[0085] Define a random noise matrix Z: its dimension is the same as the mask matrix and random noise is added only at the missing positions on the incomplete dataset.

[0086] Define the data matrix after generator interpolation : Incomplete data matrix , the mask matrix and the random noise matrix The input of the generated network is as follows:

[0087] ;

[0088] in is the data matrix after interpolation; To generate network operations; is the mask matrix; is the random noise matrix; For Hadamard; is an incomplete data matrix.

[0089] Define the output of the generated network, which is the restored complete sedimentation data matrix : Generate the complete settlement data matrix , should be composed of two parts: the data that are not missing in the original data and the data generated by the generator at the missing position of the original data, then it is as follows;

[0090] ;

[0091] in, represents the final complete settlement data matrix; is the mask matrix; For Hadamard; is the data matrix after interpolation.

[0092] The input of the adversarial network is the complete sedimentation data matrix (one part is real data, the other part is fake data) and the prompt matrix The output dimension of the adversarial network is the same as the input, representing the probability of the corresponding value in the input matrix being true or false.

[0093] Define the prompt matrix H: The prompt matrix with the missing data position information is also passed to the adversarial network. The generation of the prompt matrix depends on the mask matrix. The position of the pseudo data part is marked with a value of 0.5 so that the network can learn the distribution of the real data. The final output of the adversarial network is as follows:

[0094] ;

[0095] in is the adversarial network output, D is the adversarial network operation, is the prompt matrix.

[0096] For example, based on the collected sedimentation data set, a total of 6 rows and 4400 columns correspond to the sedimentation data values ​​of 6 different points (M1-M6). In order to verify the rationality of this model, the corresponding missing rate of the data set is manually assigned, and the recovery effect of the multi-channel sedimentation data repair deep learning model DCL-SAM-GAIN and GAIN on the data set is compared. The specific steps are as follows:

[0097] (1) The collected data sets are normalized by the data logger, and all surface settlement data values ​​are converted to the range of 0-1 using the maximum-minimum normalization method, as follows:

[0098] ;

[0099] where x norm is the normalized sedimentation value, x max is the maximum value of the settlement data, x min is the minimum value of the settlement data.

[0100] (2) Use the mask matrix to specify random missing values ​​on the collected complete sedimentation data set, and each collector randomly discards a certain proportion of sampling points. Consider the cases where the data missing rate is 15%, 30%, and 45%. Generate a mask vector that can represent the missing position information of each sedimentation data channel, and then merge it into a mask matrix, that is, the data missing of each collector is independent.

[0101] (3) The incomplete data sets with different data missing rates are obtained through the mask matrix, and the incomplete data sets after missing data are used to train the deep learning network.

[0102] (4) The processed incomplete data set is input into the DCL-SAM-GAIN network, and the corresponding mask matrix and random noise matrix are also input at the same time. After adversarial training, the network outputs the restored complete data set (note that the normalized data set needs to be denormalized to restore it to the original range). Finally, the real data and the repaired data are used to analyze and evaluate the results.

[0103] The hyperparameters used in this model The parameter is set to 50, the learning rate is set to 0.001, the prompt rate in the mask matrix is ​​set to 0.90, the Adam (Adaptive Moment Estimation) optimizer, the number of iterations is set to 1500, and the final result is measured by the relative error between the true value and the predicted value.

[0104] Comparative analysis of results:

[0105] Compare the data recovery effects of GAIN (a model in existing research) and DCL-SAM-GAIN under different data missing rates (15%, 30%, and 45%). The comparison results are shown in Table 1:

[0106] Table 1 Relative error values ​​of data recovery of subsidence observation points under different data missing rates (%)

[0107]

[0108] The comparison results show that: as the data missing rate increases, the error of the model's recovery effect on incomplete data sets gradually increases, but under the condition that the missing rate is no more than 45%, the relative error of the DCL-SAM-GAIN model in recovering missing data is no more than 10%. Under the condition of the same data missing rate, the data recovery effect of DCL-SAM-GAIN is higher than that of the GAIN model.

[0109] S3 builds a settlement prediction model based on the MHSAM-LSTM deep learning model, and performs settlement prediction on the settlement prediction model inputted with the complete settlement data matrix that meets the requirements according to the probability matrix of the true and false values ​​in the complete settlement data matrix, and outputs the prediction result. That is, through step S2, the probability matrix of the true and false values ​​in the complete settlement data matrix is ​​obtained, and it is judged whether the complete settlement data matrix meets the requirements, and the settlement prediction is performed on the complete settlement data matrix that meets the requirements.

[0110] In this step, the complete sedimentation dataset restored by the DCL-SAM-GAIN deep learning model is modeled and normalized (using the maximum and minimum normalization method).

[0111] The data set consisting of the complete settlement data matrix is ​​divided into training set, validation set and test set to train, validate and test the settlement prediction model to determine the optimal hyperparameter combination; the settlement prediction model includes three LSTM layers and two MHSAM layers. The LSTM layer is used to capture the specific features of the complete settlement data matrix. The MHSAM layer is set between two adjacent LSTM layers to capture global features and enrich the features of the captured data from different angles; the settlement prediction model outputs the prediction results and the evaluation of the prediction results. The prediction results are time series data, representing the settlement monitoring results of each collector at different times.

[0112] The multi-channel joint input is the settlement prediction results of different collectors at different times, which is a two-dimensional matrix (number of collectors * time point). The evaluation includes mean square error, mean absolute error and correlation coefficient.

[0113] Among them, the LSTM layer includes a forget gate, an update gate, an input gate, a cell unit, and an output gate, as follows:

[0114] ;

[0115] ;

[0116] ;

[0117] ;

[0118] ;

[0119] ;

[0120] -c <t>< / t> Indicates the memory cells involved in the calculation at the current moment; c <t-1>< / t-1> is the memory cell of the previous moment; <t>< / t> is the hidden state at time t; x <t>< / t> 、h <t -1> 、c <t -1> is the vector input to the LSTM layer; a <t>< / t> and c <t>< / t> represents the output vector; b (b f , b u , b c ,b o ) represents the corresponding bias values ​​of the forget gate, update gate, cell unit and output gate; W (W f ,W u , W c , W o) represents the corresponding weight values ​​of the forget gate, update gate, cell unit and output gate; σ and tanh are sigmoid activation function and tanh activation function respectively; Γ f is the forget gate, Γ u is the update gate, Γ o is the output gate.

[0121] The LSTM layer is defined as a bidirectional loop structure, namely Bi-LSTM, and its corresponding output values ​​are as follows:

[0122] O t-Bi = softmax(W o ×[ <t>< / t> , <t>< / t> ]+b o ) ;

[0123] in <t>< / t> and <t>< / t> represents the hidden state of the cell at each moment in forward and backward propagation, W o , b o denote the weight and bias vectors respectively.

[0124] Define the architecture and weight distribution of the multi-head attention layer:

[0125] The present invention uses 3 LSTM layers. Due to the bidirectional architecture of the network, there are a total of 6 LSTM units to capture the specific features of time series data. The MHSAM in the present invention is set in the middle of the Bi-LSTM layer, that is, a multi-head self-attention layer is set in the first and second LSTM layers, the second and third LSTM layers to capture the global features of the data and enrich the features of the captured data from different angles.

[0126] The structure of Multi-Head Self-Attention (MHSAM) is attached Figure 3 . In the self-attention module (i.e. when the number of attention heads is 1, head=1), there are three functions, namely "query (Q)", "key (K)" and "value (K)", which are obtained by multiplying the input with the weight matrix. Through the attention layer, a "Q" and a set of "K" and "V" pairs are mapped, and the weight scores calculated by "K" and "Q" are assigned to each value. The attention weight score (A) between each pair of positions in the input sequence is calculated by softmax and Q, K as follows:

[0127] ;

[0128] ;

[0129] ;

[0130] ;

[0131] ;

[0132] Where X is the input sequence of length N, and the matrix X∈R N×d represents, d represents the dimension of each position in the sequence, d k represents the dimension of the kth key; Wq, Wk, Wv are the weight matrices corresponding to Q, K, V; A is the attention score, softmax represents the normalization function; Head[.] represents the multi-head self-attention operation; Attention(X) is the output; Concat is the merging of multiple multi-head self-attentions.

[0133] When using a multi-head self-attention mechanism, if head is set to m, the above process is repeated m times. At this time, the output of the multi-head self-attention [Attention(X)] is calculated by summarizing the value of V according to the attention score (A), and the results of multiple heads are combined through the Concat operation, and finally the output is obtained through a complete connection layer, see the two formulas in the dropout technology below. In this embodiment, the number of self-attention heads is set to 4 (head=4).

[0134] Define the dropout technique:

[0135] In the application of deep learning methods, overfitting is a recognized challenge. Dropout technology is one of the most effective and direct neural network regularization techniques. The main purpose of dropout is to prevent the network from over-relying on a single neuron and to reduce the co-adaptability between neurons by discarding a part of the neurons in the model architecture to reduce the risk of overfitting during training. Specifically, the neuron is multiplied by a random variable in a Bernoulli distribution with a probability of p, resulting in a dropout rate of (1-p), as shown in the following formula:

[0136] m r = x r1 w 1 + x r2 w 2 + x r3 w 3 +……+h rt w t + b 0 ;

[0137] M r = f (m r );

[0138] where r t is a random vector based on Bernoulli distribution, f() is the activation function of the output layer, x rt is the output of the hidden layer after implementing the dropout technique, x t is the output of the hidden layer, w t and b 0 is the weight matrix and bias, M rt is the final output of the model, m r is the output of the model at time t. The dropout value in this embodiment can be set to 0.40, which can prevent overfitting and gradient vanishing and exploding problems for each LSTM layer.

[0139] Define the sliding window technique:

[0140] The sliding window technique systematically moves across the time series, creating multiple overlapping sequences to extract characteristic relationships and trends on various time scales, enhancing the robustness and predictive performance of the model. The sliding window size in this embodiment is set to 50. That is, if the sliding window size is set to k=50, the first k=50 consecutive data points are used to input into the model to predict the 51st settlement data value.

[0141] Introducing the multi-head self-attention mechanism (MHSAM) into LSTM for sedimentation prediction can improve the model's prediction ability, especially when the data has complex temporal dependencies and correlations. MHSAM can calculate attention in parallel through multiple attention heads, and can extract information from different angles. Each head can learn different feature representations, and finally combine these features to provide richer feature representations. MHSAM can directly calculate the relationship between any two positions in the sequence without being restricted by the length of the sequence, so as to better capture the long-range dependencies between different parts of the sequence. By setting MHSAM in different LSTM layers to improve the model's analysis and screening of features at different levels or dimensions of the data, the prediction effect of the LSTM model can be further improved.

[0142] In addition, the dropout regularization technology is implemented in the MHSAM-LSTM model to avoid the occurrence of overfitting problems, and the sliding window technology is introduced to create multiple overlapping data combinations. The window range is expanded or reduced as needed to adapt to different analysis needs while maximizing the use of time series data.

[0143] The MHSAM-LSTM deep network model architecture is attached Figure 4 , the following compares the improvement effect of MHSAM on the LSTM model:

[0144] (1) Using the dataset generated by DCL-SAM-GAIN, the prediction performance of MHSAM-LSTM and LSTM models is compared to illustrate the improvement of MHSAM on the performance of LSTM model in predicting surface subsidence.

[0145] (2) Using the dataset generated by DCL-SAM-GAIN, we compare the prediction performance of MHSAM-LSTM and the long short-term memory model (SAM-LSTM) enhanced by the self-attention mechanism, which is commonly used at present. The result shows that MHSAM improves the performance of LSTM model better than SAM does.

[0146] (3) The MHSAM-LSTM model was used to predict the settlement of the original data set and the data set restored by the DCL-SAM-GAIN model. Combined with the specific settlement prediction model, it was verified that the settlement data set restored by the DCL-SAM-GAIN model can be effectively used in specific predictions.

[0147] The data set is divided into three parts: training set, validation set and test set. In the present invention, the 5-fold cross validation method is first used to train the MHSAM-LSTM model to determine its optimal hyperparameter combination, wherein 70% of the data is used for cross validation training and the remaining 30% of the data is used for testing the model. The five-fold cross validation method allows each sample to be used as a validation set once, and the process is repeated 5 times, thereby generating the average value of 5 prediction errors, and the average value of all results is selected as the prediction result of the final model.

[0148] See also Figure 4 In the present invention, three LSTM layers are set in the LSTM model, namely LSTM1, LSTM2 and LSTM3, and the multi-head self-attention layer is set in the middle of the adjacent LSTM layers, namely MHSA1 and MHSA2. The initial values ​​of various hyperparameters are set, including the number of hidden units in the LSTM layer, the number of hidden units in the MHSAM layer, the sliding window size, the learning rate, the dropout value and the number of iterations.

[0149] In the first stage, the MHSAM-LSTM model first uses the settlement values ​​from the i-th to the (i+m)-th time steps as inputs passed to the model to predict the predicted settlement at the (i+m+1)-th time step, where i represents the i-th time step and m is the sliding window size.

[0150] In the second stage, after the first LSTM layer (LSTM1), the input vector is transferred to the first multi-head self-attention layer (MHSA1), which extracts the features of the data and calculates the weight score of the element by combining the information from the i-th to (i+m)-th time steps, as follows:

[0151] ;

[0152] in represents the multi-head self-attention weight value of the model at time step i, Tj represents the feature vectors of other time steps derived from the original dataset, t=1,2,3,…m.

[0153] In the third stage, the output of the first MHSA layer at each time step is represented as Di, Di+1, Di+2, …, Di+m-1, Di+m, where Di indicates that the output after the multi-head self-attention mechanism at the i-th moment is a combination of different weights at each time step. Subsequently, these results are used as the input vector of the next LSTM layer (LSTM2) and further processed by the second MHSA2. Finally, the third LSTM layer (LSTM3) generates the results for each time step.

[0154] The final model was debugged and trained through the cross-validation method, and the optimal architecture of the model was obtained as follows: the number of hidden layer units in the LSTM was set to 64, 128 and 64 respectively, the number of hidden units in the multi-head attention layer was set to 64 and 128, the dropout value was 0.50, the learning rate was 0.001, the data batch size was 128, the sliding window size was 50, and the number of iterations was 800.

[0155] In this process, the mean square error (MSE), mean absolute error (MAE), and correlation coefficient (R 2 ) and other evaluation indicators to evaluate the prediction effect of the model, as shown in the following formula:

[0156] ;

[0157] ;

[0158] ;

[0159] Where yi represents the measured value; Represents the predicted value represents the average value of the measured value; N is the total number of data sets.

[0160] Comparative Example 1 compares the settlement prediction performance of MHSAM-LSTM, SAM-LSTM and LSTM models:

[0161] Based on the DCL-SAM-GAIN deep learning model, a new data set was obtained by restoring the data set with a data missing rate of 30%. The MHSAM-LSTM and LSTM models were used to build a settlement prediction model. The results are shown in Table 2.

[0162] Table 2 Prediction error at different settlement points - taking the restored data set with a data loss rate of 30% as an example

[0163]

[0164] The conclusions show that: (1) MHSAM-LSTM has a higher fit and a smaller prediction error for different settlement observation points, and its performance is better than the unimproved LSTM model. (2) MHSAM-LSTM has a higher fit and a smaller error for different settlement observation points, and its performance is better than the SAM-LSTM model.

[0165] Comparative Example 2: Comparison of the prediction errors of the MHSAM-LSTM model on the original data set (no missing data) and the data set restored after missing data (restored by the DCL-SAM-GAIN model):

[0166] Based on the DCL-SAM-GAIN deep learning model, the data sets with data missing rates of 15% and 30% are jointly restored to obtain a new restored data set. Then, based on this data set, the MHSAM-LSTM model is used to build a settlement prediction model for prediction. The results are shown in Table 3:

[0167] Table 3. Settlement prediction errors of different categories of data sets based on the MHSAM-LSTM model

[0168]

[0169] The conclusion shows that after setting different degrees of missing values ​​for the data set, the data is restored through DCL-SAM-GAIN, and then the results of settlement prediction using the MHSAM-LSTM model are not much different from the results of modeling and prediction using the original data set directly, which further verifies that the DCL-SAM-GAIN deep learning model has a good data recovery effect. At the same time, it also proves that the proposed MHSAM-LSTM not only has a good prediction effect on the original data set, but also has a better prediction effect on the data set restored by the DCL-SAM-GAIN deep learning model, indicating that it has strong generalization ability, robust performance and small prediction error.

[0170] Based on this, the present invention also proposes a device for realizing settlement prediction based on intelligently restored surface settlement data, comprising:

[0171] The data acquisition module obtains the observation data of continuous settlement over a period of time. The observation data is an incomplete surface settlement data matrix with certain random missing points. A mask matrix is ​​defined to mark the information of the missing data locations in the observation data. The mask matrix is ​​used to simulate the locations of missing points to generate an incomplete data matrix.

[0172] The data patching module constructs a multi-channel sedimentation data patching deep learning model. The multi-channel sedimentation data patching deep learning model includes a generative network and an adversarial network. The incomplete data matrix, mask matrix and random noise matrix are input into the generative network to obtain the restored complete sedimentation data matrix; the prompt matrix is ​​generated based on the mask matrix, and the complete sedimentation data matrix and the prompt matrix are input into the adversarial network to obtain the probability matrix of true and false values ​​in the complete sedimentation data matrix.

[0173] The settlement prediction module builds a settlement prediction model based on the MHSAM-LSTM deep learning model. According to the probability matrix of true and false values ​​in the complete settlement data matrix, the settlement prediction model input with the complete settlement data matrix that meets the requirements performs settlement prediction and outputs the prediction results.

[0174] The device of the present invention is used to execute the above-mentioned method for realizing settlement prediction based on intelligently restored surface settlement data, the data acquisition module is used to execute the above-mentioned step S1, the data repair module is used to execute the above-mentioned step S2, and the settlement prediction module is used to execute the above-mentioned step S3.

[0175] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A method for realizing settlement prediction based on intelligently restored surface settlement data, characterized in that: Obtaining observation data of continuous settlement over a period of time, wherein the observation data is an incomplete surface settlement data matrix with certain random missing points, defining a mask matrix to mark the information of the missing data positions in the observation data, and using the mask matrix to simulate the positions of the missing points to generate an incomplete data matrix; Constructing a multi-channel sedimentation data repair deep learning model, the multi-channel sedimentation data repair deep learning model includes a generation network and an adversarial network, inputting the incomplete data matrix, the mask matrix and the random noise matrix into the generation network to obtain a restored complete sedimentation data matrix; Generate a prompt matrix based on the mask matrix, input the complete sedimentation data matrix and the prompt matrix into the adversarial network to obtain a probability matrix of true and false values ​​in the complete sedimentation data matrix; A settlement prediction model based on the MHSAM-LSTM deep learning model is constructed. According to the probability matrix of true and false values ​​in the complete settlement data matrix, the complete settlement data matrix that meets the requirements is input into the settlement prediction model for settlement prediction and outputs the prediction result.

2. A method for realizing settlement prediction based on intelligently restored surface settlement data according to claim 1, characterized in that: Monitoring points are arranged above the shield tunnel, and a fiber Bragg grating data collector is used to obtain observation data of continuous settlement over a period of time.

3. The method for realizing settlement prediction based on intelligently restored surface settlement data according to claim 1, characterized in that: The generation network includes a first input layer, a first feature extraction module, a first pooling layer, a first self-attention layer, a second feature extraction module, a second self-attention layer, a first upsampling layer, a third feature extraction module and a first output layer; the incomplete data matrix, the mask matrix and the random noise matrix are input into the first feature extraction module via the first input layer for feature extraction and output of a first transition matrix; the first transition matrix is ​​respectively passed to the first pooling layer and the first upsampling layer; the first transition matrix is ​​input into the first self-attention layer via the first pooling layer, and then passed to the second feature extraction module for feature extraction and output of a second transition matrix; the first transition matrix and the second transition matrix are input into the first upsampling layer and then passed to the third feature extraction module for feature extraction, and the restored complete sedimentation data matrix is ​​output from the first output layer.

4. A method for realizing settlement prediction based on intelligently restored surface settlement data as claimed in claim 3, characterized in that: The first feature extraction module includes a first convolutional layer, a first densely connected layer, a first normalization layer, a second convolutional layer and a second normalization layer arranged in sequence; the second feature extraction module includes a third convolutional layer, a second densely connected layer, a third normalization layer, a fourth convolutional layer and a fourth normalization layer arranged in sequence; the third feature extraction module includes a fifth convolutional layer and a fifth batch normalization layer arranged in sequence; the first densely connected layer and the second densely connected layer respectively include four convolutional layers arranged in sequence, and the feature output of the previous convolutional layer is passed and used as the input of the next convolutional layer.

5. The method for realizing settlement prediction based on intelligently restored surface settlement data according to claim 1, characterized in that: The adversarial network includes a second input layer, a fourth feature extraction module, a second pooling layer, a fifth feature extraction module, a second upsampling layer, a sixth feature extraction module and a second output layer; the complete sedimentation data matrix and the prompt matrix are input into the fourth feature extraction module via the second input layer for feature extraction and outputting a third transition matrix; the third transition matrix is ​​respectively passed to the second pooling layer and the second upsampling layer; the third transition matrix is ​​input into the fifth feature extraction module via the second pooling layer for feature extraction and outputting a fourth transition matrix; the third transition matrix and the fourth transition matrix are input into the second upsampling layer and then passed to the sixth feature extraction module for feature extraction, and the probability matrix is ​​obtained from the output of the second output layer.

6. A method for realizing settlement prediction based on intelligently restored surface settlement data according to claim 5, characterized in that: The fourth feature extraction module includes a first convolutional layer, a first densely connected layer, a first normalization layer, a second convolutional layer and a second normalization layer arranged in sequence; the fifth feature extraction module includes a second densely connected layer, a third normalization layer, a third convolutional layer, a third densely connected layer, a fourth normalization layer and a fourth convolutional layer arranged in sequence; the sixth feature extraction module includes a fifth convolutional layer and a fifth batch normalization layer arranged in sequence; the first densely connected layer, the second densely connected layer and the third densely connected layer respectively include four convolutional layers, and the feature output of each convolutional layer is passed and used as the input of the subsequent convolutional layer.

7. A method for realizing settlement prediction based on intelligently restored surface settlement data according to claim 1, characterized in that: The generative network includes binary cross entropy loss and RMSE loss, and its loss function is: ； The loss function of the adversarial network is: ； in, represents the minimization objective function, G represents the generator, and represents the objective function, Represents a variable , , The expectation of , E[.] means to find the expected value, is the complete settlement data matrix; is an incomplete data matrix; is the mask matrix; For Hadamard; is the prompt matrix; D is the adversarial network operation; Represents a hyperparameter.

8. The method for realizing settlement prediction based on intelligently restored surface settlement data according to claim 1, characterized in that: The data set consisting of the complete settlement data matrix is ​​divided into a training set, a validation set and a test set to train, validate and test the settlement prediction model to determine the optimal hyperparameter combination; the settlement prediction model includes three LSTM layers and two MHSAM layers, the LSTM layer is used to capture the specific features of the complete settlement data matrix, and the MHSAM layer is arranged between two adjacent LSTM layers to capture global features and enrich the features of the captured data from different angles; the settlement prediction model outputs prediction results and an evaluation of the prediction results, and the evaluation includes mean square error, mean absolute error and correlation coefficient.

9. A method for realizing settlement prediction based on intelligently restored surface settlement data according to claim 8, characterized in that: The LSTM layer includes a forget gate, an update gate, an input gate, a cell unit and an output gate, as follows: ； ； ； ； ； ； -c <t>< / t> Indicates the memory cells involved in the calculation at the current moment; c <t-1>< / t-1> is the memory cell of the next moment; <t>< / t> is the hidden state at time t; x <t>< / t> 、h <t -1> 、c <t -1> is the vector input to the LSTM layer; a <t>< / t> and c <t>< / t> represents the output vector; b(b f , b u , b c ,b o ) represents the corresponding bias values ​​of the forget gate, update gate, cell unit and output gate; W (W f , W u , W c ,W o ) represents the corresponding weight values ​​of the forget gate, update gate, cell unit and output gate; σ and tanh are sigmoid activation function and tanh activation function respectively; Γ f is the forget gate, Γ u is the update gate, Γ o is the output gate; The MHSAM layer includes Q, K and K, which represent query, key function and value respectively, as follows: ； ； ； ； ； Where X is the input sequence of length N, and the matrix X∈R N×d represents, d represents the dimension of each position in the sequence, d k represents the dimension of the kth key; Wq, Wk, Wv are the weight matrices corresponding to Q, K, and V; A is the attention score, softmax represents the normalization function; Head[.] represents the multi-head self-attention operation; Attention(X) is the output; Concat is the merging of multiple multi-head self-attention operations.

10. A device for realizing settlement prediction based on intelligently restored surface settlement data, characterized in that: include: A data acquisition module is used to acquire observation data of continuous settlement over a period of time, wherein the observation data is an incomplete surface settlement data matrix with certain random missing data, and a mask matrix is ​​defined to mark the information of the missing data positions in the observation data, and the positions of the missing points are simulated by the mask matrix to generate an incomplete data matrix; A data patching module is used to construct a multi-channel sedimentation data patching deep learning model, wherein the multi-channel sedimentation data patching deep learning model includes a generation network and an adversarial network, and the incomplete data matrix, the mask matrix and the random noise matrix are input into the generation network to obtain a restored complete sedimentation data matrix; Generate a prompt matrix based on the mask matrix, input the complete sedimentation data matrix and the prompt matrix into the adversarial network to obtain a probability matrix of true and false values ​​in the complete sedimentation data matrix; The settlement prediction module constructs a settlement prediction model based on the MHSAM-LSTM deep learning model, and inputs the complete settlement data matrix that meets the requirements into the settlement prediction model according to the probability matrix of the true and false values ​​in the complete settlement data matrix to perform settlement prediction and output the prediction result.

Images