Deep foundation pit support pile micro-deformation prediction method based on multi-head attention mechanism

Abstract

The invention relates to the technical field of deep foundation pit engineering, in particular to a deep foundation pit support pile micro-deformation prediction method based on a multi-head attention mechanism, and the method comprises the steps: collecting the monitoring data of a target deep foundation pit support pile; the monitoring data are input into a preset prediction model, a micro-deformation prediction result of the target deep foundation pit support pile is output, the prediction model is obtained based on training of a training set, the training set comprises horizontal deformation data of a plurality of groups of deep foundation pit support piles at different time and spatial positions, and the horizontal deformation data of the deep foundation pit support piles at different time and spatial positions is obtained. The prediction model is constructed by adopting a convolutional neural network, a long-short-term memory network and a multi-head attention mechanism. According to the method, key information is extracted from multi-dimensional data through a multi-head attention mechanism, factors which have the largest influence on micro-deformation can be effectively recognized, and accurate prediction in a complex environment is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep foundation pit engineering, and in particular to a method for predicting micro-deformation of deep foundation pit supporting piles based on a multi-head attention mechanism. Background Art

[0002] In recent years, with the acceleration of urbanization and the extensive development of underground space, deep foundation pit support technology has become particularly important in modern urban construction. As a key structure to maintain the stability of the foundation pit and avoid damage to the surrounding environment, deep foundation pit support piles play a vital role in areas with complex geology and dense high-rise buildings. However, in actual use, support piles will undergo micro-deformations due to factors such as soil pressure, groundwater level changes, and ground loads. The accumulation of these micro-deformations may lead to the gradual degradation of the performance of the foundation pit support structure and even cause safety hazards. Therefore, the monitoring and evaluation of support pile micro-deformation has become a research focus in the current engineering field. Traditional support pile deformation monitoring methods mainly rely on field observations and numerical simulations, but these methods have problems such as insufficient detection accuracy, poor real-time performance, and difficulty in coping with complex environments. It is impossible to effectively cope with the dynamic changes of support pile micro-deformation and the coupling of complex factors. In addition, the traditional method has low data utilization efficiency and it is difficult to accurately identify key factors under the influence of multiple factors, resulting in limited prediction of micro-deformation in foundation pit support engineering. These deficiencies increase the safety risks of foundation pit support structures and limit the level of intelligent monitoring of deep foundation pit projects.

[0003] In recent years, with the development of artificial intelligence technology, the monitoring method based on neural network has gradually attracted attention due to its powerful data processing and self-learning capabilities. Neural networks can predict and monitor the micro-deformation of support piles through the training of a large amount of data, providing more accurate and real-time evaluation results. Compared with traditional methods, the prediction effect of neural network methods is more significant under multi-factor coupling environments, and has strong adaptive capabilities. Therefore, establishing a neural network system suitable for micro-deformation monitoring of deep foundation pit support piles can not only improve the accuracy of deformation monitoring, but also provide reliable protection for the safety of foundation pit support structures, and provide new technical means for the design and management of deep foundation pit support projects. Summary of the invention

[0004] The purpose of the present invention is to provide a deep foundation pit support pile micro-deformation prediction method based on a multi-head attention mechanism. By extracting key information from multi-dimensional data through the multi-head attention mechanism, the factors that have the greatest impact on micro-deformation can be effectively identified, and accurate prediction in complex environments can be achieved.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for predicting micro-deformation of deep foundation pit support piles based on a multi-head attention mechanism, comprising:

[0007] Collect monitoring data of target deep foundation pit support piles;

[0008] The monitoring data is input into a preset prediction model, and the micro-deformation prediction result of the target deep foundation pit supporting pile is output, wherein the prediction model is obtained based on training of a training set, and the training set contains horizontal deformation data of deep foundation pit supporting piles at different time and space positions. The prediction model is constructed by integrating convolutional neural network, long short-term memory network and multi-head attention mechanism.

[0009] Optionally, before training the prediction model based on the training set, the step further includes preprocessing the training set, and the preprocessing includes:

[0010] Data cleaning and normalization are performed on the horizontal deformation data of several groups of deep foundation pit support piles at different time and space positions in the training set, and the processed data are constructed into a spatiotemporal matrix of support structure deformation.

[0011] Optionally, the prediction model includes: an input layer, a first feature extraction module, a second feature extraction module, a third feature extraction module, and an output layer, wherein the input layer is used to input the monitoring data; the first feature extraction module is used to use a convolutional neural network to perform local feature extraction on the monitoring data to obtain spatial features; the second feature extraction module is used to use a long short-term memory network to extract time dependencies on the spatial features to obtain time features; the third feature extraction module is used to use a multi-head attention mechanism to perform global feature extraction on the time features to obtain key features; the output layer is used to predict micro-deformation of deep foundation pit support piles based on the key features.

[0012] Optionally, the first feature extraction module includes K layers of convolutional layers, pooling layers, and fully connected layers connected in sequence. After extracting spatial features through the convolutional layers and pooling layers, the spatial features are output through the fully connected layers.

[0013] Optionally, the second feature extraction module includes a forget gate, an input gate, and an output gate connected in sequence, and extracts the time dependency in the spatial features through the forget gate, input gate, and output gate to output time series features.

[0014] Optionally, the method of extracting the time dependency in the spatial feature through the forget gate, the input gate, and the output gate is:

[0015]

[0016] Among them, x t is the current input; i t is the input gate; f t For the forget gate;t is the output gate; c t For long-term memory; c t-1 It is the long-term memory of the previous moment; t For short-term memory; h t-1 It is the short-term memory of the previous moment; is a candidate state; W i , W f , W o , W c is the parameter matrix to be trained; b i 、b f 、b o 、b c is the bias term to be trained; σ is the sigmoid activation function.

[0017] Optionally, the third feature extraction module includes several encoders and decoders, and a multi-head attention mechanism is adopted in both the encoders and the decoders.

[0018] Optionally, the multi-head attention mechanism is expressed as:

[0019] Multi-Head(Q,K,V)

[0020] =Concat(head1,head2,...,head i ,...,head h )W O

[0021] Among them, head i =Attention(QW i Q ,KW i K ,VW i V ), W i Q , KW i K , VW i V is the linear transformation matrix corresponding to different heads, mapping the input vector to query Q, key K and value V; W O is a linear transformation matrix used to project the concatenated vector back to the original dimension of the input.

[0022] The beneficial effects of the present invention are:

[0023] (1) Improvement of prediction accuracy: Combining the characteristics of CNN, LSTM and Transformer, the prediction model of the present invention has excellent learning ability in the prediction of micro-deformation of supporting piles, and the extraction and modeling of key features are more accurate, which effectively reduces errors and improves the accuracy of prediction.

[0024] (2) Enhanced real-time and robustness: Due to the use of a multi-head attention mechanism, the prediction model can dynamically adjust the attention paid to different input features and adapt to changes in complex environments with multiple factors, thus ensuring the prediction stability and real-time performance under different conditions, making it more practical in actual applications.

[0025] (3) Ability to adapt to complex data: The prediction model architecture can efficiently process multi-dimensional data, adapt to the influence of complex factors such as geological conditions and load changes, meet the needs of micro-deformation prediction of deep foundation pit support piles under different working conditions, and improve the safety monitoring effect of deep foundation pit support structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A schematic diagram of the structure and workflow of a first feature extraction module according to an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of the structure and workflow of a second feature extraction module according to an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of the structure and workflow of the third feature extraction module in an embodiment of the present invention;

[0030] Figure 4 The figure is a schematic diagram of the structure and workflow of the prediction model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] This embodiment provides a method for predicting micro-deformation of deep foundation pit support piles based on a multi-head attention mechanism, including:

[0034] Collect monitoring data of target deep foundation pit support piles;

[0035] The monitoring data is input into a preset prediction model, and the micro-deformation prediction result of the target deep foundation pit supporting pile is output, wherein the prediction model is obtained based on training of a training set, and the training set contains horizontal deformation data of deep foundation pit supporting piles at different time and space positions. The prediction model is constructed by integrating convolutional neural network, long short-term memory network and multi-head attention mechanism.

[0036] Specifically, this embodiment combines the characteristics of convolutional neural networks, long short-term memory networks and multi-head attention mechanisms, and the constructed prediction model has excellent learning ability in the prediction of micro-deformation of support piles, and the extraction and modeling of key features are more accurate, which effectively reduces errors and improves the accuracy of prediction.

[0037] Furthermore, before training the prediction model based on the training set, the training set is also preprocessed, and the preprocessing includes:

[0038] Data cleaning and normalization are performed on the horizontal deformation data of several groups of deep foundation pit support piles at different time and space positions in the training set, and the processed data are constructed into a spatiotemporal matrix of support structure deformation.

[0039] Specifically, this embodiment collects and organizes the monitoring data of deep foundation pit support piles, including horizontal deformation data at different time and space positions. Through data cleaning and normalization preprocessing, the original data is converted into a standardized data set suitable for model training, and a spatiotemporal matrix of support structure deformation is constructed. The data set is divided into a training set and a test set in proportion to characterize the deformation characteristics of the support piles in various time and space dimensions.

[0040] Furthermore, the prediction model includes: an input layer, a first feature extraction module, a second feature extraction module, a third feature extraction module, and an output layer, wherein the input layer is used to input the monitoring data; the first feature extraction module is used to use a convolutional neural network to perform local feature extraction on the monitoring data to obtain spatial features; the second feature extraction module is used to use a long short-term memory network to extract time dependencies on the spatial features to obtain time features; the third feature extraction module is used to use a multi-head attention mechanism to perform global feature extraction on the time features to obtain key features; the output layer is used to predict micro-deformation of deep foundation pit support piles based on the key features.

[0041] Among them, the first feature extraction module includes K layers of convolutional layers, pooling layers, and fully connected layers connected in sequence. After the spatial features are extracted by the convolutional layers and pooling layers, they are output through the fully connected layers.

[0042] The second feature extraction module includes a forget gate, an input gate, and an output gate connected in sequence. The time dependency in the spatial features is extracted through the forget gate, the input gate, and the output gate, and a time series feature is output.

[0043] The third feature extraction module includes several encoders and decoders, and a multi-head attention mechanism is adopted in both the encoders and the decoders.

[0044] Specifically, the structure and workflow of the prediction model constructed in this embodiment are as follows: Figure 4 As shown in the figure, the CNN-LSTM-Transformer network model can gradually realize the spatial and temporal feature extraction and modeling of global dependency relationships. First, the first feature extraction module, namely the convolutional neural network (CNN), first extracts local features from the input data and obtains spatial features through convolutional layers and pooling layers; then, the second feature extraction module, namely the long short-term memory network (LSTM), is used to model the temporal dependency of deformation data and capture the dynamic change trend of micro-deformation of support piles; finally, the third feature extraction module, namely the Transformer layer, uses self-attention and multi-head attention mechanisms to identify important feature relationships in a global scope, further improving the accuracy and robustness of prediction.

[0045] Specifically, the structure and workflow of the first feature extraction module are as follows: Figure 1 As shown in Figure 1, a convolutional neural network (CNN) is used to extract one-dimensional features from the original monitoring data. CNN can automatically extract local features from the data and enhance the feature representation capability of the input data, thereby providing effective basic features for subsequent time series modeling. The mathematical relationship in the convolution calculation is shown in formula (1):

[0046]

[0047] Where l is the network layer, M j is the data matrix of the jth input, is the jth output of the lth layer, is the i-th output of the l-1th layer, f is the activation function, * is the convolution symbol, is the kernel parameter at position (i, j), is the bias term. The calculation method of the pooling layer neuron is shown in formula (2):

[0048]

[0049] In the formula, down() is the downsampling function, which selects the maximum value or average value of a certain area to replace that area.

[0050] Next, the structure and workflow of the second feature extraction module are as follows: Figure 2 As shown in Figure 3, the long short-term memory network (LSTM) is used to model the time series features. LSTM has significant advantages in processing data with strong time dependence, and can effectively capture the trend and law of the micro-deformation of the support piles over time, thereby improving the model's ability to predict dynamic changes. The specific data processing process is shown in formula (3):

[0051]

[0052] Where: x t is the current input; i t is the input gate; f t For the forget gate; t is the output gate; c t is the cell state (long-term memory); c t-1 is the cell state at the previous moment (long-term memory); h t is memory (short-term memory); h t-1 It is the memory of the last moment; is a candidate state; W i , W f , W o , W c is the parameter matrix to be trained; b i 、b f 、b o 、b c is the bias term to be trained; σ is the sigmoid activation function, σ=1 / (1+e -x ), where e is a natural constant.

[0053] Finally, the structure and workflow of the third feature extraction module are as follows: Figure 3 As shown in Figure 1, the Transformer with multi-head attention mechanism is introduced into the model architecture, and the multi-head attention mechanism is used to focus on the key factors affecting micro-deformation in the feature sequence. The Transformer model can capture the long-distance dependencies between different features in a global scope and assign weights to highlight important features, which helps to improve the prediction accuracy and robustness in complex environments. The formula of the multi-head attention mechanism is shown in formula (4):

[0054] Multi-Head(Q,K,V)

[0055] =Concat(head1,head2,...,head i ,...,head h )W O (4)

[0056] Where: head i =Attention(QW iQ ,KW i K ,VW i V ), W i Q , KW i K , VW i V is the linear transformation matrix corresponding to different heads, mapping the input vector to query, key and value; W O is a linear transformation matrix used to project the concatenated vector back to the original dimension of the input. This matrix is ​​used to integrate the results of all attention heads.

[0057] Furthermore, in the model training phase, the mean square error (MSE) is used as the loss function, and the model parameters are updated through back propagation and Adam optimization algorithm, and hyperparameters such as convolution kernel size and number of hidden layers are continuously adjusted until the model converges and reaches the preset prediction accuracy. During the training process, the validation set is used to evaluate the generalization ability of the model to avoid overfitting problems.

[0058] After training, the test set is input into the model for deformation prediction, and the prediction performance and error of the model are measured using evaluation indicators such as mean absolute error (MAE) and root mean square error (RMSE). The actual deformation data and the predicted value are compared to verify the accuracy and stability of the model.

[0059] Finally, the trained model is applied to the micro-deformation prediction of actual deep foundation pit support piles, and the tiny deformation changes of the support piles are monitored in real time to provide support for engineering safety monitoring and realize early warning of the deformation trend of the support piles.

[0060] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for predicting micro-deformation of deep foundation pit support piles based on a multi-head attention mechanism, characterized in that: include: Collect monitoring data of target deep foundation pit support piles; The monitoring data is input into a preset prediction model, and the micro-deformation prediction result of the target deep foundation pit supporting pile is output, wherein the prediction model is obtained based on training of a training set, and the training set contains horizontal deformation data of deep foundation pit supporting piles at different time and space positions. The prediction model is constructed by integrating convolutional neural network, long short-term memory network and multi-head attention mechanism.

2. The method for predicting micro-deformation of deep foundation pit support piles based on multi-head attention mechanism according to claim 1 is characterized in that: Before training the prediction model based on the training set, the training set is preprocessed, and the preprocessing includes: Data cleaning and normalization are performed on the horizontal deformation data of several groups of deep foundation pit support piles at different time and space positions in the training set, and the processed data are constructed into a spatiotemporal matrix of support structure deformation.

3. The method for predicting micro-deformation of deep foundation pit support piles based on multi-head attention mechanism according to claim 1 is characterized in that: The prediction model includes: an input layer, a first feature extraction module, a second feature extraction module, a third feature extraction module, and an output layer, wherein the input layer is used to input the monitoring data; the first feature extraction module is used to use a convolutional neural network to perform local feature extraction on the monitoring data to obtain spatial features; the second feature extraction module is used to use a long short-term memory network to extract time dependencies on the spatial features to obtain time features; the third feature extraction module is used to use a multi-head attention mechanism to perform global feature extraction on the time features to obtain key features; the output layer is used to predict the micro-deformation of deep foundation pit support piles based on the key features.

4. The method for predicting micro-deformation of deep foundation pit supporting piles based on multi-head attention mechanism according to claim 3 is characterized in that: The first feature extraction module includes K convolutional layers, pooling layers, and fully connected layers connected in sequence. After the spatial features are extracted by the convolutional layers and pooling layers, they are outputted by the fully connected layers.

5. The method for predicting micro-deformation of deep foundation pit supporting piles based on multi-head attention mechanism according to claim 3 is characterized in that: The second feature extraction module includes a forget gate, an input gate, and an output gate connected in sequence. The time dependency in the spatial features is extracted through the forget gate, the input gate, and the output gate, and a time series feature is output.

6. The method for predicting micro-deformation of deep foundation pit supporting piles based on multi-head attention mechanism according to claim 5 is characterized in that: The method of extracting the time dependency in the spatial feature through the forget gate, input gate, and output gate is: Among them, x t is the current input; i t is the input gate; f t For the forget gate; t is the output gate; c t For long-term memory; c t-1 It is the long-term memory of the previous moment; t For short-term memory; h t-1 It is the short-term memory of the previous moment; is a candidate state; W i , W f , W o , W c is the parameter matrix to be trained; b i , b f , b o , b c is the bias term to be trained; σ is the sigmoid activation function.

7. The method for predicting micro-deformation of deep foundation pit supporting piles based on multi-head attention mechanism according to claim 3 is characterized in that: The third feature extraction module includes several encoders and decoders, and a multi-head attention mechanism is adopted in both the encoders and the decoders.

8. The method for predicting micro-deformation of deep foundation pit supporting piles based on multi-head attention mechanism according to claim 7 is characterized in that: The multi-head attention mechanism is expressed as: Multi-Head(Q,K,V)=Concat(head1,head2,...,head i ,...,head h )W O Among them, head i =Attention(QW i Q ,KW i K ,VW i V ), W i Q , KW i K , VW i V is the linear transformation matrix corresponding to different heads, mapping the input vector to query Q, key K and value V; W O is a linear transformation matrix used to project the concatenated vector back to the original dimension of the input.

Images