A method for predicting micro-deformation of deep foundation pit support piles based on multi-head attention mechanism

Abstract

This invention relates to the field of deep foundation pit engineering technology, and in particular to a method for predicting the micro-deformation of deep foundation pit support piles based on a multi-head attention mechanism. The method includes: collecting monitoring data of the target deep foundation pit support piles; inputting the monitoring data into a preset prediction model; and outputting the micro-deformation prediction result of the target deep foundation pit support piles. The prediction model is trained on a training set containing several sets of horizontal deformation data of the deep foundation pit support piles at different times and spatial locations. The prediction model is constructed using a convolutional neural network, a long short-term memory network, and a multi-head attention mechanism. This invention extracts key information from multi-dimensional data through a multi-head attention mechanism, effectively identifying the factors that have the greatest impact on micro-deformation and achieving accurate prediction in complex environments.

Description

Technical Field

[0001] This invention relates to the field of deep foundation pit engineering technology, and in particular to a method for predicting the micro-deformation of deep foundation pit support piles based on a multi-head attention mechanism. Background Technology

[0002] In recent years, with the acceleration of urbanization and the widespread development of underground space, deep foundation pit support technology has become particularly important in modern urban construction. Deep foundation pit support piles, as key structures for maintaining foundation pit stability and preventing damage to the surrounding environment, play a crucial role in areas with complex geology and dense high-rise buildings. However, in actual use, support piles undergo minute deformations due to factors such as soil pressure, groundwater level changes, and ground loads. The accumulation of these micro-deformations can lead to the gradual degradation of the foundation pit support structure's performance and even cause safety hazards. Therefore, the monitoring and evaluation of micro-deformation of support piles has become a key research focus in the current engineering field. Traditional methods for monitoring support pile deformation mainly rely on on-site observation and numerical simulation, but these methods suffer from insufficient detection accuracy, poor real-time performance, and difficulty in handling complex environments. They cannot effectively address the dynamic changes in micro-deformation of support piles and the coupling effects of complex factors. Furthermore, traditional methods have low data utilization efficiency and struggle to accurately identify key factors under multiple influences, thus limiting their predictive ability for micro-deformation in foundation pit support engineering. These deficiencies increase the safety risks of foundation pit support structures and limit the level of intelligent monitoring in deep foundation pit projects.

[0003] In recent years, with the development of artificial intelligence technology, monitoring methods based on neural networks have gradually attracted attention due to their powerful data processing and self-learning capabilities. Neural networks can predict and monitor the micro-deformation of support piles through training on large amounts of data, providing more accurate and real-time assessment results. Compared with traditional methods, neural network methods show more significant prediction effects in multi-factor coupled environments and have stronger adaptive capabilities. Therefore, establishing a neural network system suitable for monitoring the micro-deformation of support piles in deep foundation pits can not only improve the accuracy of deformation monitoring but also provide reliable protection for the safety of foundation pit support structures, offering new technical means for the design and management of deep foundation pit support projects. Summary of the Invention

[0004] The purpose of this invention is to provide a method for predicting the micro-deformation of deep foundation pit support piles based on a multi-head attention mechanism. By extracting key information from multi-dimensional data through the multi-head attention mechanism, it can effectively identify the factors that have the greatest impact on micro-deformation and achieve accurate prediction in complex environments.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A method for predicting the micro-deformation of deep foundation pit support piles based on a multi-head attention mechanism includes:

[0007] Collect monitoring data on the support piles of the target deep foundation pit;

[0008] The monitoring data is input into a preset prediction model, and the micro-deformation prediction result of the target deep foundation pit support pile is output. The prediction model is obtained by training based on a training set, which contains horizontal deformation data of the deep foundation pit support pile at different times and spatial locations. The prediction model is constructed by integrating convolutional neural networks, long short-term memory networks and multi-head attention mechanisms.

[0009] Optionally, before training the prediction model based on the training set, the method further includes preprocessing the training set, which includes:

[0010] Data cleaning and normalization were performed on several sets of horizontal deformation data of deep foundation pit support piles at different times and spatial locations in the training set, and the processed data were used to construct a spatiotemporal matrix of support structure deformation.

[0011] Optionally, the prediction model includes: an input layer, a first feature extraction module, a second feature extraction module, a third feature extraction module, and an output layer, wherein the input layer is used to input the monitoring data; the first feature extraction module is used to perform local feature extraction on the monitoring data using a convolutional neural network to obtain spatial features; the second feature extraction module is used to extract the temporal dependencies of the spatial features using a long short-term memory network to obtain temporal features; the third feature extraction module is used to perform global feature extraction on the temporal features using a multi-head attention mechanism to obtain key features; and the output layer is used to predict the micro-deformation of deep foundation pit support piles based on the key features.

[0012] Optionally, the first feature extraction module includes K convolutional layers, pooling layers, and fully connected layers connected in sequence. Spatial features are extracted through the convolutional layers and pooling layers, and then output through the fully connected layers.

[0013] Optionally, the second feature extraction module includes a forget gate, an input gate, and an output gate connected in sequence. The forget gate, input gate, and output gate are used to extract the temporal dependencies in the spatial features and output time series features.

[0014] Optionally, the method for extracting the temporal dependencies in the spatial features using the forget gate, input gate, and output gate is as follows:

[0015]

[0016] Where, x t For the current input; i t For input gate; f t Forgotten Gate; ot For output gate; c t For long-term memory; c t-1 For long-term memory of the previous moment; h t For short-term memory; h t-1 It is the short-term memory of the previous moment; Candidate state; W i W f W o W c b is the parameter matrix to be trained; i b f b o b c σ represents the bias term to be trained; σ is the sigmoid activation function.

[0017] Optionally, the third feature extraction module includes several encoders and decoders, all of which employ a multi-head attention mechanism.

[0018] Optionally, the multi-head attention mechanism is represented as follows:

[0019] Multi-Head(Q,K,V)

[0020] =Concat(head1,head2,...,head i ,...,head h W O

[0021] Among them, head i =Attention(QW i Q ,KW i K VW i V ), W i Q KW i K VW i V For each different head, a linear transformation matrix is ​​used to map the input vector to the query Q, key K, and value V; W O This is a linear transformation matrix used to project the concatenated vector back to the original dimension of the input.

[0022] The beneficial effects of this invention are as follows:

[0023] (1) Improved prediction accuracy: Combining the characteristics of CNN, LSTM and Transformer, the prediction model of this invention has excellent learning ability in predicting micro deformation of support piles, and the extraction and modeling of key features are more accurate, effectively reducing errors and improving the accuracy of prediction.

[0024] (2) Enhanced real-time performance and robustness: Due to the adoption of the multi-head attention mechanism, the prediction model can dynamically adjust the attention to different input features, adapt to the complex changes of multiple factors, and ensure the prediction stability and real-time performance under different conditions, making it more practical in practical applications.

[0025] (3) Ability to adapt to complex data: The prediction model architecture can efficiently process multi-dimensional data, adapt to the influence of complex factors such as geological conditions and load changes, meet the needs of deep foundation pit support pile micro-deformation prediction under different working conditions, and improve the safety monitoring effect of deep foundation pit support structure. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram illustrating the structural composition and workflow of the first feature extraction module in an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram illustrating the structure and workflow of the second feature extraction module in an embodiment of the present invention.

[0029] Figure 3 This is a schematic diagram illustrating the structure and workflow of the third feature extraction module in an embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram illustrating the structure and workflow of the prediction model in an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] This embodiment provides a method for predicting the micro-deformation of deep foundation pit support piles based on a multi-head attention mechanism, including:

[0034] Collect monitoring data on the support piles of the target deep foundation pit;

[0035] The monitoring data is input into a preset prediction model, and the micro-deformation prediction result of the target deep foundation pit support pile is output. The prediction model is obtained by training based on a training set, which contains horizontal deformation data of the deep foundation pit support pile at different times and spatial locations. The prediction model is constructed by integrating convolutional neural networks, long short-term memory networks and multi-head attention mechanisms.

[0036] Specifically, this embodiment combines the characteristics of convolutional neural networks, long short-term memory networks, and multi-head attention mechanisms. The constructed prediction model has excellent learning ability in predicting the micro-deformation of support piles, and its extraction and modeling of key features are more accurate, effectively reducing errors and improving the accuracy of prediction.

[0037] Furthermore, before training the prediction model based on the training set, the method further includes preprocessing the training set, which includes:

[0038] Data cleaning and normalization were performed on several sets of horizontal deformation data of deep foundation pit support piles at different times and spatial locations in the training set, and the processed data were used to construct a spatiotemporal matrix of support structure deformation.

[0039] Specifically, this embodiment collects and organizes monitoring data of deep foundation pit support piles, including horizontal deformation data at different times and spatial locations. Through data cleaning and normalization preprocessing, the raw data is transformed into a standardized dataset suitable for model training, and a spatiotemporal matrix of support structure deformation is constructed. The dataset is divided into training and test sets proportionally to characterize the deformation features of the support piles in various time and spatial dimensions.

[0040] Furthermore, the prediction model includes: an input layer, a first feature extraction module, a second feature extraction module, a third feature extraction module, and an output layer. The input layer is used to input the monitoring data; the first feature extraction module uses a convolutional neural network to extract local features from the monitoring data to obtain spatial features; the second feature extraction module uses a long short-term memory network to extract the temporal dependencies of the spatial features to obtain temporal features; the third feature extraction module uses a multi-head attention mechanism to extract global features from the temporal features to obtain key features; and the output layer is used to predict the micro-deformation of deep foundation pit support piles based on the key features.

[0041] The first feature extraction module includes K convolutional layers, pooling layers, and fully connected layers connected in sequence. Spatial features are extracted through the convolutional layers and pooling layers, and then output through the fully connected layers.

[0042] The second feature extraction module includes a forget gate, an input gate, and an output gate connected in sequence. The forget gate, input gate, and output gate are used to extract the temporal dependencies in the spatial features and output time series features.

[0043] The third feature extraction module includes several encoders and decoders, and both encoders and decoders employ a multi-head attention mechanism.

[0044] Specifically, the predictive model structure and workflow constructed in this embodiment are as follows: Figure 4 As shown, the CNN-LSTM-Transformer network model can progressively extract spatial and temporal features and model global dependencies. First, the first feature extraction module, the Convolutional Neural Network (CNN), extracts local features from the input data, obtaining spatial features through convolutional and pooling layers. Then, the second feature extraction module, the Long Short-Term Memory Network (LSTM), models the temporal dependencies of the deformation data, capturing the dynamic trends of the micro-deformation of the support piles. Finally, the third feature extraction module, the Transformer layer, utilizes self-attention and multi-head attention mechanisms to identify important feature relationships globally, further improving the accuracy and robustness of predictions.

[0045] Specifically, the structure and workflow of the first feature extraction module are as follows: Figure 1 As shown, a convolutional neural network (CNN) is used to extract one-dimensional features from the original monitoring data. CNN can automatically extract local features from the data, enhance the feature representation capability of the input data, and thus provide effective basic features for subsequent time series modeling. The mathematical relationship in the convolution calculation is shown in equation (1):

[0046]

[0047] In the formula, l represents the network layer, and M represents the network layer. j It is the j-th input data matrix. It is the j-th output of the l-th layer. It is the i-th output of the (l-1)-th layer, f is the activation function, and * is the convolution symbol. The kernel parameter at position (i, j) This is the bias term. The calculation method for pooling layer neurons is shown in equation (2):

[0048]

[0049] In the formula, down() is a downsampling function, which selects the maximum or average value of a certain region to replace that region.

[0050] Next, the structure and workflow of the second feature extraction module are as follows: Figure 2 As shown, a Long Short-Term Memory (LSTM) network is used to model the time series features. LSTM has significant advantages in processing data with strong time dependence, and can effectively capture the trend and law of the micro deformation of the support piles over time, thereby improving the model's ability to predict dynamic changes. The specific data processing process is shown in equation (3).

[0051]

[0052] In the formula: x t For the current input; i t For input gate; f t Forgotten Gate; o t For output gate; c t For cellular state (long-term memory); c t-1 The previous cellular state (long-term memory); h t For memory (short-term memory); h t-1 It is the memory of the previous moment; Candidate state; W i W f W o W c b is the parameter matrix to be trained; i b f b o b c Here, σ represents the bias term to be trained; σ is the sigmoid activation function, σ = 1 / (1+e^(-σ / σ)). -x ), where e is the natural constant.

[0053] Finally, the structure and workflow of the third feature extraction module are as follows: Figure 3 As shown, the Transformer model incorporates a multi-head attention mechanism into its architecture. This mechanism focuses on key factors influencing subtle deformations within the feature sequence. The Transformer model can capture long-distance dependencies between different features globally and assign weights to highlight important features, thus improving prediction accuracy and robustness in complex environments. The multi-head attention mechanism formula is shown in equation (4).

[0054] Multi-Head(Q,K,V)

[0055] =Concat(head1,head2,...,head i ,...,head h W O (4)

[0056] In the formula: head i =Attention(QW iQ ,KW i K VW i V ), W i Q KW i K VW i V For each different head, a linear transformation matrix is ​​used to map the input vector to the query, key, and value; W O This is a linear transformation matrix used to project the concatenated vector back to the original dimension of the input. This matrix is ​​used to integrate the results of all attention heads.

[0057] Furthermore, during the model training phase, mean squared error (MSE) is used as the loss function. The model parameters are updated through backpropagation and the Adam optimization algorithm, continuously adjusting hyperparameters such as convolutional kernel size and the number of hidden layers until the model converges and reaches the preset prediction accuracy. During training, a validation set is used to evaluate the model's generalization ability, avoiding overfitting.

[0058] After training, the test set is input into the model for deformation prediction. Evaluation metrics such as Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) are used to measure the model's predictive performance and error. The actual deformation data and predicted values ​​are compared to verify the model's accuracy and stability.

[0059] Finally, the trained model is applied to the prediction of micro-deformation of actual deep foundation pit support piles, to monitor the minute deformation changes of the support piles in real time, to provide support for engineering safety monitoring, and to achieve early warning of the deformation trend of the support piles.

[0060] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for predicting the micro-deformation of deep foundation pit support piles based on a multi-head attention mechanism, characterized in that, include: Collect monitoring data on the support piles of the target deep foundation pit; The monitoring data is input into a preset prediction model, and the micro-deformation prediction result of the target deep foundation pit support pile is output. The prediction model is obtained by training based on a training set, which contains horizontal deformation data of the deep foundation pit support pile at different times and spatial locations. The prediction model is constructed by integrating convolutional neural networks, long short-term memory networks and multi-head attention mechanisms. The prediction model includes an input layer, a first feature extraction module, a second feature extraction module, a third feature extraction module, and an output layer. The input layer is used to input the monitoring data. The first feature extraction module uses a convolutional neural network to extract local features from the monitoring data to obtain spatial features. The second feature extraction module uses a long short-term memory network to extract the temporal dependencies of the spatial features to obtain temporal features. The third feature extraction module uses a multi-head attention mechanism to extract global features from the temporal features to obtain key features. The output layer is used to predict the micro-deformation of deep foundation pit support piles based on the key features. The second feature extraction module includes a forget gate, an input gate, and an output gate connected in sequence. The forget gate, input gate, and output gate are used to extract the temporal dependencies in the spatial features and output time series features. The method for extracting the temporal dependencies in the spatial features using the forget gate, input gate, and output gate is as follows: ； in, x t This is the current input; i t For input gates; f t Forgotten Gate; o t For output gate; c t For long-term memory; c t−1 For long-term memory of the previous moment; h t For short-term memory; h t−1 It is the short-term memory of the previous moment; Candidate state; W i , W f , W o , W c The parameter matrix to be trained; b i , b f , b o , b c The bias term to be trained; σ It is the sigmoid activation function; The third feature extraction module includes several encoders and decoders, and both encoders and decoders employ a multi-head attention mechanism.

2. The method for predicting micro-deformation of deep foundation pit support piles based on multi-head attention mechanism according to claim 1, characterized in that, Before training the prediction model based on the training set, the method further includes preprocessing the training set, which includes: Data cleaning and normalization were performed on several sets of horizontal deformation data of deep foundation pit support piles at different times and spatial locations in the training set, and the processed data were used to construct a spatiotemporal matrix of support structure deformation.

3. The method for predicting micro-deformation of deep foundation pit support piles based on multi-head attention mechanism according to claim 1, characterized in that, The first feature extraction module includes several convolutional layers, pooling layers, and fully connected layers connected in sequence. Spatial features are extracted through the convolutional layers and pooling layers, and then output through the fully connected layers.

4. The method for predicting micro-deformation of deep foundation pit support piles based on multi-head attention mechanism according to claim 1, characterized in that, The multi-head attention mechanism is represented as follows: ； Among them, head i =Attention( QW i Q , KW i K , VW i V ), W i Q , KW i K , VW i V For each different head, a linear transformation matrix is ​​used to map the input vector to the query. Q ,key K Sum V ; W O This is a linear transformation matrix used to project the concatenated vector back to the original dimension of the input.

Images