A highway slope displacement prediction method

Abstract

This application provides a method for predicting highway slope displacement, relating to the field of highway slope displacement prediction technology. By integrating data from multiple sensors, satellite remote sensing, and UAV images, a comprehensive data acquisition system is constructed. A convolutional neural network combined with a long short-term memory neural network is used to extract spatiotemporal features from the images, and a graph neural network combined with a long short-term memory neural network is used to extract spatiotemporal features from the sensor data. These features are then fused through an attention mechanism to form comprehensive spatiotemporal feature data. Ensemble empirical mode decomposition is applied to decompose and reconstruct the fused features, removing noise and retaining core features. The main spatiotemporal features are then input into a Transformer model for prediction. Leveraging the Transformer model's powerful ability to capture spatiotemporal dependencies, the prediction of highway slope displacement is achieved. This method improves prediction accuracy through the comprehensive utilization of multi-source data and advanced deep learning models.

Description

Technical Field

[0001] The invention relates to the technical field of highway slope displacement prediction, and in particular to a highway slope displacement prediction method. Background Art

[0002] With the rapid development of highway construction in my country, the number of highway slopes is also increasing, and the stability of highway slope service will directly affect the operational safety and maintenance costs of highways. Therefore, it is of great significance to carry out health monitoring of highway slopes and accurately predict the trend of slope displacement changes to ensure the safety of people's lives and property and the safe operation of highways. In recent years, many studies have been carried out on slope displacement prediction models. Most machine learning models lack explanatory internal mechanisms and their impact on output. Since slope displacement data has nonlinear and unstable characteristics, directly predicting it will reduce the prediction accuracy of the model.

[0003] The document with the prior art publication number CN117725368A provides a highway slope displacement prediction method, which adopts the variational mode decomposition of WT-PCA combined algorithm and particle swarm optimization to achieve high-precision and stable prediction. Through the WT-PCA combined algorithm and wavelet denoising technology, the X, Y, and Z displacement data of the highway slope are denoised and fused into cumulative displacement. Wavelet denoising is performed on a three-layer scale to improve data purity. The variational mode decomposition (VMD) algorithm is applied to decompose the cumulative displacement spatiotemporal feature data into trend, cycle, and random terms. The particle swarm algorithm is used to optimize the decomposition layer number and penalty factor of VMD to ensure prediction accuracy. Combining multiple algorithms, the accuracy and robustness of the slope displacement prediction model are improved. Experiments have proved that its prediction effect is highly accurate and stable, and it has important application value for highway slope safety monitoring.

[0004] Although this technical method has achieved remarkable results in highway slope displacement prediction by combining WT-PCA combined algorithm, wavelet denoising technology and variational mode decomposition (VMD) algorithm, as well as particle swarm optimization, with the continuous advancement of technology and the increasing abundance of data, it is necessary to explore more advanced and efficient prediction methods to meet the complex and changeable highway slope displacement monitoring needs. In practical applications, more influencing factors and complex actual environments need to be considered.

[0005] In view of this, we propose a highway slope displacement prediction method. Summary of the invention

[0006] The present invention provides a method for predicting highway slope displacement, and the technical solution includes the following aspects.

[0007] According to one aspect of an embodiment of the present application, a method for predicting highway slope displacement is provided, and the method includes:

[0008] S1. Deploy a variety of ground sensors to obtain slope geological information.

[0009] Deploy drones to obtain high-resolution images of the slopes.

[0010] Obtain satellite remote sensing images of the slopes regularly.

[0011] Establish a comprehensive data collection system to obtain comprehensive data.

[0012] S2. Use data cleaning algorithms to identify and process outliers and missing values ​​in the comprehensive data to ensure the quality of the comprehensive data.

[0013] S3. Use convolutional networks combined with long short-term memory neural networks to extract spatiotemporal features from satellite remote sensing images and drone high-resolution images. Use graph neural networks combined with long short-term memory neural networks to extract spatiotemporal features of slope geological information.

[0014] S4. Fuse the spatiotemporal feature 1 with the spatiotemporal feature 2 to obtain comprehensive spatiotemporal feature data.

[0015] S5. Perform integrated empirical mode decomposition on the spatiotemporal feature data, decompose the spatiotemporal feature data into different intrinsic mode functions, use the entropy method to select the intrinsic mode function with the highest entropy value for reconstruction, remove noise and unimportant components, and obtain the main spatiotemporal features.

[0016] S6. Input the main spatiotemporal features into the Transformer model for prediction, and use the powerful spatiotemporal dependency capture capability of the Transformer model to predict highway slope displacement.

[0017] Wherein, in step S1, the ground sensors include GPS, accelerometer, and inclinometer.

[0018] The highway slope geological information includes the precise displacement of the slope, the position information of the slope, the vibration and acceleration change of the slope, and the inclination of the slope.

[0019] GPS sensors are used to measure the precise displacement and location information of the slope. They are deployed at the top, middle, and bottom of the slope to cover the entire slope area and obtain precise displacement and location information in real time and / or periodically.

[0020] Accelerometers are used to monitor the vibration and acceleration changes of the slope and detect possible landslide precursors. They are arranged at different heights and key points along the slope to obtain vibration and acceleration data at high frequency.

[0021] Inclinometers are used to monitor changes in slope inclination and provide information on slope stability. They are placed in areas prone to landslides to obtain slope inclination regularly.

[0022] The slope geological information is transmitted to the data center in real time using a wireless network.

[0023] Wherein, in step S1, regularly acquiring satellite remote sensing images of the slope specifically includes regularly downloading the latest satellite remote sensing images from a satellite data platform; and preprocessing the satellite remote sensing images.

[0024] Wherein, in step S2, the comprehensive data is identified and processed using a data cleaning algorithm.

[0025] The machine learning model Isolation Forest is used to identify outliers in the synthetic data, and the detected outliers and missing values ​​are handled using interpolation methods.

[0026] Convert sensor data into a unified time series format, Dsensor = {(t i ,s i )} i=1 N , where t i is the timestamp, s i is the sensor reading.

[0027] Convert the high-resolution image to a uniform pixel matrix format, I = {(x, y, RGB)} x=1,y=1 H,W , where H and W are the height and width of the image, and RGB is the pixel value.

[0028] Convert satellite remote sensing images into a unified raster format, R = {(x, y, Bandk)} x=1,y=1,k=1 H,W,K , where K is the number of bands.

[0029] Among them, in step S3, the convolutional network is combined with the long short-term memory neural network to extract features: the convolutional network is used to extract the spatial features in the satellite remote sensing images and the drone high-resolution images at each time point; the image spatial features at different time points are arranged in chronological order to form a time series, and then the long short-term memory network is used to extract the time features; and then the spatiotemporal feature h is obtained. t .

[0030] Among them, in step S3, a graph neural network combined with a long short-term memory neural network is used to extract features: a graph structure is constructed according to the geographical location and physical relationship of the ground sensors, the nodes represent the ground sensors, and the edges represent the relationship between the ground sensors.

[0031] The mutual influence and spatial dependency between monitoring points are captured through graph neural network.

[0032] Arrange the spatial features of the graph structure at different time points in chronological order to form a time series, and use the long short-term memory network to extract the time features; then obtain the spatiotemporal feature h t '.

[0033] In step S4, the attention mechanism is used to calculate the importance weight of each feature, setting the query vector Q, key vector K and value vector V, Q=W Q -[h t ,h t '],K=W K -[h t ,h t '],V=W V -[h t ,h t '], where W Q , W K and W V is a trainable weight matrix.

[0034] The attention weight A is calculated by weighted fusion through the attention weight A, A=softmax(QK T / d k 1 / 2 ), where d k is the dimension of the key vector.

[0035] The comprehensive spatiotemporal feature data h is obtained by weighting the sum of the attention weights. t fused ,h t fused =AV.

[0036] In step S5, the spatiotemporal feature data h t fused Perform ensemble empirical mode decomposition.

[0037] is the spatiotemporal feature data h t fused Add white noise of different amplitudes to generate multiple different time series.

[0038] The integrated empirical mode decomposition is performed on each time series after adding white noise to obtain several intrinsic mode functions.

[0039] All intrinsic mode functions are averaged to obtain the final intrinsic mode function.

[0040] The entropy value of each intrinsic mode function is calculated, and several intrinsic mode functions with the highest entropy values ​​are selected for reconstruction.

[0041] The selected intrinsic mode functions are reconstructed to form the denoised spatiotemporal features.

[0042] By fusing spatiotemporal features from different sources and performing integrated empirical mode decomposition and reconstruction after feature fusion, noise and unimportant components are removed, the main spatiotemporal features are obtained, and the accuracy of prediction is improved.

[0043] Among them, in step S6, the input is made into the Transformer model for prediction.

[0044] The Transformer model is used to predict the main spatiotemporal features. The Transformer model encoder consists of multiple layers of identical sub-layers, each of which contains a multi-head self-attention mechanism and a feedforward neural network.

[0045] The multi-head self-attention mechanism allows the model to focus on different parts of the input sequence at the same time, decomposing the input vector into three parts: query vector Q, key vector K, and value vector V, calculating the attention weights, and weighted summing the value vector V.

[0046] The feedforward neural network is used to perform nonlinear transformation on the output of self-attention, which includes two linear layers with a ReLU activation function inserted in the middle.

[0047] Before and after each sub-layer, the Transformer model uses layer normalization and residual connections to accelerate convergence and improve model stability.

[0048] In the prediction task, the main spatiotemporal features are used as the input of the Transformer model encoder. The last layer output of the encoder will contain rich spatiotemporal feature information, which is directly input into a linear layer and finally connected to the output layer for prediction. The output layer uses a linear activation function.

[0049] During the training phase, the Transformer model learns by minimizing the loss function between the predicted values ​​and the true slope displacement, and the model parameters are updated using the gradient descent method or its variants.

[0050] The training data uses real historical comprehensive data.

[0051] Using an end-to-end training approach, the entire model is treated as a whole, and all components from input to output are optimized in the same training process.

[0052] In step S6, a phased training method is used to first independently train the convolutional network combined with the long short-term memory neural network and the graph neural network combined with the long short-term memory neural network, freeze their parameters, and then only train the Transformer model.

[0053] According to one aspect of an embodiment of the present application, a highway slope displacement prediction system is provided, and the system includes:

[0054] The data acquisition module is used to deploy a variety of sensors to regularly acquire satellite remote sensing images and drone images of the slope and build a comprehensive data acquisition system for multi-source data.

[0055] The data preprocessing module uses data cleaning algorithms to identify and process outliers and missing values ​​in the comprehensive data to ensure the quality and reliability of the comprehensive data.

[0056] The feature extraction module uses a convolutional network combined with a long short-term memory neural network and a graph neural network combined with a long short-term memory network to extract spatiotemporal features from satellite remote sensing images, drone images, and sensor data, respectively.

[0057] The feature fusion module combines the spatiotemporal features extracted by the convolutional network with the long short-term memory neural network and the graph neural network with the long short-term memory neural network to form a comprehensive and multi-level spatiotemporal feature data.

[0058] The feature reconstruction module performs integrated empirical mode decomposition on the fused spatiotemporal feature data, decomposes the spatiotemporal feature data into different intrinsic mode functions, and selects the intrinsic mode function with the highest entropy value for reconstruction through the entropy method, removes noise and unimportant components, and obtains the main spatiotemporal features.

[0059] The displacement prediction module inputs the main spatiotemporal features into the Transformer model and uses the powerful spatiotemporal dependency capture capability of the Transformer model to predict the displacement of the highway slope.

[0060] The present application provides a highway slope displacement prediction method, which has the following advantages.

[0061] 1. Through the comprehensive data acquisition system, satellite remote sensing images, drone images and slope geological information are combined, the information advantages of multi-source data are fully utilized, and the richness and accuracy of the data are improved.

[0062] 2. The structure of convolutional network combined with long short-term memory neural network and graph neural network combined with long short-term memory neural network is adopted. It combines the advantages of convolutional neural network in spatial feature extraction and the ability of long short-term memory network in time series processing, and effectively extracts the spatiotemporal characteristics of data. The introduction of graph neural network enables the model to capture the mutual influence and spatial dependency between monitoring points, further improving the accuracy of prediction.

[0063] By calculating the importance weight of each feature through the attention mechanism, the effective fusion of multi-source data features is achieved, forming a comprehensive and multi-level spatiotemporal feature data, retaining the characteristics of each data source, and improving the richness and accuracy of feature representation through complementary effects.

[0064] 3. By integrating empirical mode decomposition and entropy method, the intrinsic mode function with the highest entropy value is selected for reconstruction, which effectively removes noise and unimportant components, retains the most important spatiotemporal features, improves the signal-to-noise ratio of the data, and enhances the robustness of the model to noise.

[0065] 4. The Transformer model is used for prediction, and its powerful ability to capture spatiotemporal dependencies is utilized to achieve accurate prediction of highway slope displacement. The multi-head self-attention mechanism in the Transformer model encoder enables the model to focus on different parts of the input sequence at the same time, improving the model's ability to process complex data. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A flow chart of a highway slope displacement prediction method provided in one embodiment of the present application.

[0067] Figure 2 A structural diagram of a highway slope displacement prediction system provided for one embodiment of the present application.

[0068] Figure 3 A highway slope displacement prediction model architecture diagram provided for one embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0070] Please refer to Figure 1 , which shows a flow chart of a highway slope displacement prediction method provided by an embodiment of the present application.

[0071] S1. Deploy a variety of ground sensors to obtain slope geological information.

[0072] Deploy drones to obtain high-resolution images of the slopes.

[0073] Obtain satellite remote sensing images of the slopes regularly.

[0074] Establish a comprehensive data collection system to obtain comprehensive data.

[0075] S2. Use data cleaning algorithms to identify and process outliers and missing values ​​in the comprehensive data to ensure the quality of the comprehensive data.

[0076] S3. Use convolutional networks combined with long short-term memory neural networks to extract spatiotemporal features from satellite remote sensing images and drone high-resolution images. Use graph neural networks combined with long short-term memory neural networks to extract spatiotemporal features of slope geological information.

[0077] S4. Fuse the spatiotemporal feature 1 with the spatiotemporal feature 2 to obtain comprehensive spatiotemporal feature data.

[0078] S5. Perform integrated empirical mode decomposition on the spatiotemporal feature data, decompose the spatiotemporal feature data into different intrinsic mode functions, use the entropy method to select the intrinsic mode function with the highest entropy value for reconstruction, remove noise and unimportant components, and obtain the main spatiotemporal features.

[0079] S6. Input the main spatiotemporal features into the Transformer model for prediction, and use the powerful spatiotemporal dependency capture capability of the Transformer model to predict highway slope displacement.

[0080] As an optional solution of the present application, the ground sensor includes GPS, accelerometer, and inclinometer.

[0081] The highway slope geological information includes the precise displacement of the slope, the position information of the slope, the vibration and acceleration change of the slope, and the inclination of the slope.

[0082] GPS sensors are used to measure the precise displacement and location information of the slope. They are deployed at different key points of the slope, such as the top, middle, and bottom, covering the entire slope area, and obtaining precise displacement and location information in real time and / or periodically.

[0083] Accelerometers are used to monitor the vibration and acceleration changes of the slope and detect possible landslide precursors. They are arranged at different heights and key points along the slope, such as areas prone to landslides, to obtain vibration and acceleration data regularly.

[0084] Inclinometers are used to monitor changes in slope inclination and provide information on slope stability. They are placed in areas prone to landslides and obtain slope inclination regularly (e.g., every hour).

[0085] The slope geological information is transmitted to the data center in real time using a wireless network.

[0086] UAV high-resolution images provide visual monitoring data. Regularly (such as weekly or monthly) use drones to fly to cover the entire slope area and take high-resolution images. Satellite remote sensing images obtain satellite remote sensing images of a large range of slope areas for macro monitoring. Use remote sensing image processing technology to pre-process satellite remote sensing images. Use wireless networks (such as LoRa, ZigBee, Wi-Fi) to transmit slope geological information to the data center in real time. After the drone flight, the image data is transmitted to the data center by wireless or wired means. The latest satellite remote sensing images are downloaded from the satellite data platform regularly. A data storage center is established to store all data from ground sensors, drones, and satellites. Data is backed up regularly to ensure data security and reliability.

[0087] Regularly obtain satellite remote sensing images of the slope and regularly download the latest satellite remote sensing images from the satellite data platform; and pre-process the satellite remote sensing images.

[0088] As an optional solution of the present application, in S2, the integrated data is identified and processed using a data cleaning algorithm.

[0089] Identify outliers in synthetic data using the machine learning model Isolation Forest, IF(x)=2 -E（h（x）） / c（n） , where h(x) is the path length of sample x in the tree, E(h(x)) is the expected value of the path length, c(n) is an adjustment constant for the size of the dataset, and interpolation is used to handle detected outliers and missing values.

[0090] Interpolation method, x i '=(x i-1 +x i+1 ) / 2, where x i ' is the value after filling, x i-1 and x i+1 is the data point before and after the outlier. For missing values, linear interpolation is performed using the data points before and after. i '=x i-1 +(x i+1 -x i-1 ) / (t i+1 -t i-1 )×(t i -t i-1 ), where t i is the time point, x i-1 and x i+1 are the data points before and after.

[0091] Convert sensor data into a unified time series format, Dsensor = {(t i ,s i )} i=1 N , where ti is the timestamp, s i is the sensor reading.

[0092] Convert the high-resolution image to a uniform pixel matrix format, I = {(x, y, RGB)} x=1,y=1 H,W , where H and W are the height and width of the image, and RGB is the pixel value.

[0093] Convert satellite remote sensing images into a unified raster format, R = {(x, y, Bandk)} x=1,y=1,k=1 H,W,K , where K is the number of bands.

[0094] As an optional solution of this application, the convolutional network in S3 is combined with the long short-term memory neural network to extract features: the convolutional network is used to extract the spatial features in the satellite remote sensing images and the drone high-resolution images at each time point; the image spatial features at different time points are arranged in chronological order to form a time series, and then the long short-term memory network is used to extract the time features; and then the spatiotemporal features are obtained. t .

[0095] Use convolutional networks to extract spatial features from satellite remote sensing images and drone high-resolution images at each time point. The input layer inputs the preprocessed satellite remote sensing images or drone high-resolution images. Convolutional layer: Conv(x)=W∗x+b, where W is the convolution kernel, x is the input image, b is the bias, ∗ represents the convolution operation, and pooling layer, Pool(x)=max i,j∈k x i,j , where k is the pooling window size and x i,j is the element within the pooling window.

[0096] The spatial features of images at different time points are arranged in chronological order to form a time series, and then the long short-term memory network is used to extract the time features. The LSTM unit has an input gate, a forget gate, an output gate and a cell state. The input gate, i t =σ(W i -[h t-1 ,x t ]+b i ), forget gate, f t =σ(W f -[h t-1 ,x t ]+b f ), cell state update, C~ t =tanh(W C -[h t-1 ,x t ]+b C), C t =f t ⊙C t-1 +i t ⊙C~ t , output gate, o t =σ(W o -[h t-1 ,x t ]+b o ), hidden state, h t =o t ⊙tanh(C t ), where x t is the input of the current time step, h t-1 is the hidden state of the previous time step, C t is the cell state at the current time step, σ is the sigmoid activation function, tanh is the hyperbolic tangent function, W i ,W f ,W C ,W o and b i ,b f ,b C ,b o is the weight and bias of the model, ⊙ represents the element product, and the spatiotemporal features of satellite remote sensing images or drone high-resolution images are extracted through the convolutional network combined with the structure of the long short-term memory neural network. t .

[0097] S3 uses graph neural networks combined with long short-term memory neural networks to extract features: a graph structure is constructed based on the geographical location and physical relationship of ground sensors, with nodes representing ground sensors and edges representing the relationship between ground sensors.

[0098] The mutual influence and spatial dependency between monitoring points are captured through graph neural network.

[0099] Input layer input constructed graph structure, graph convolution layer, h i （l+1） =σ(∑ j∈N（i） 1 / c ij W （l） h j （l） +b （l） ), where h i （l） is the feature of node i in the lth layer, N(i) is the set of neighbor nodes of node i, W(l) is the weight matrix of the lth layer, b （l） is the bias, c ij is the normalization coefficient and σ is the activation function.

[0100] Arrange the spatial features of the graph structure at different time points in chronological order to form a time series, and use the long short-term memory network to extract the time features; then obtain the spatiotemporal feature h t '.

[0101] In step S4, the attention mechanism is used to calculate the importance weight of each feature, set the query vector Q, key vector K and value vector V, and perform weighted fusion through the attention weight A, Q=W Q -[h t ,h t '],K=W K -[h t ,h t '],V=W V -[h t ,h t '], where W Q , W K and W V is a trainable weight matrix, calculating the attention weight A, A=softmax(QK T / d k 1 / 2 ), where d k is the dimension of the key vector, the summed value vector weighted by the attention weights, h t fused =AV.

[0102] In step S5, the spatiotemporal feature data h t fused Perform ensemble empirical mode decomposition.

[0103] is the fused spatiotemporal feature data h t fused Add white noise of different amplitudes to generate multiple different time series. t fused,j =h t fused +ε j , where ε j is the jth white noise sequence. After adding white noise to each time series, we perform integrated empirical mode decomposition to obtain several intrinsic mode functions, h t fused =∑ i=1 n IMF i +r n , among which, IMF i is the i-th intrinsic mode function, r n is the residual component, and all intrinsic mode functions are averaged to obtain the final intrinsic mode function, IMF i mean=1 / m∑ j=1 m IMF i j , where m is the number of times white noise is added, the entropy value of each IMF is calculated, and several intrinsic mode functions with the highest entropy values ​​are selected for reconstruction. H (IMF i )=-∑ k p k logp k , where p k is the probability distribution of the kth mode, and the selected intrinsic mode function is reconstructed to form the denoised spatiotemporal features, h t reconstructed =∑ i∈selected IMF i mean ,By fusing the spatiotemporal features from different sources and performing integrated empirical mode decomposition and reconstruction after feature fusion,,noise and unimportant components are removed, the most important spatiotemporal features are retained,,and the prediction accuracy is improved.

[0104] In step S6, the Transformer model is used to predict the displacement of the highway slope and the main spatiotemporal features are predicted. The Transformer model encoder consists of multiple identical sub-layers, each of which contains two sub-layers, a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism allows the model to focus on different parts of the input sequence at the same time. input Decompose it into three parts: query vector Q, key vector K and value vector V, calculate the attention weight, and weighted sum value vector V. The calculation formula of multi-head self-attention is MultiHead(Q,K,V)=Concat(head1,...,head h )W O , where head i =Attention(QW i Q ,KW i K ,VW i V ), W i Q ,W i K ,W i V and W O is a learnable weight matrix, h is the number of heads, and the attention of each head is calculated as Attention(Q,K,V)=softmax(QK T / d k 1 / 2), where d k is the dimension of the key vector. The feedforward neural network is used to perform nonlinear transformation on the output of self-attention. It includes two linear layers with a ReLU activation function inserted in the middle. Before and after each sublayer, the Transformer model uses layer normalization and residual connections to accelerate convergence and improve the stability of the model. In the prediction task, the main spatiotemporal feature X input =h t reconstructed As the input of the Transformer model encoder, the last layer output of the encoder will contain rich spatiotemporal feature information, which is directly input into a linear layer (LinearLayer) and finally connected to the output layer for prediction. The output layer uses a linear activation function, y predicted =W out z+b out , where y predicted is the predicted slope displacement value, z is the output of the last fully connected layer, and W out and b out They are the weight matrix and bias term of the output layer respectively. During the training stage, the model learns by minimizing the loss function (mean square error MSE) between the predicted value and the actual slope displacement, and uses the gradient descent method or its variant (Adam optimizer) to update the model parameters. The training data uses real historical comprehensive data. An end-to-end training method can be used, and the entire model is regarded as a whole. All components from input to output are optimized in the same training process. A phased training method can also be used. First, independently train the convolutional network combined with the long short-term memory neural network and the graph neural network combined with the long short-term memory neural network, freeze their parameters, and then only train the Transformer model.

[0105] Please refer to Figure 2 , which shows a structural diagram of a highway slope displacement prediction system provided by an embodiment of the present application.

[0106] The data acquisition module is used to deploy a variety of sensors to regularly acquire satellite remote sensing images and drone images of the slope and build a comprehensive data acquisition system for multi-source data.

[0107] The data preprocessing module uses data cleaning algorithms to identify and process outliers and missing values ​​in the comprehensive data to ensure the quality and reliability of the comprehensive data.

[0108] The feature extraction module uses a convolutional network combined with a long short-term memory neural network and a graph neural network combined with a long short-term memory network to extract spatiotemporal features from satellite remote sensing images, drone images, and sensor data, respectively.

[0109] The feature fusion module combines the spatiotemporal features extracted by the convolutional network with the long short-term memory neural network and the graph neural network with the long short-term memory neural network to form a comprehensive and multi-level spatiotemporal feature data.

[0110] The feature reconstruction module performs integrated empirical mode decomposition on the fused spatiotemporal feature data, decomposes the spatiotemporal feature data into different intrinsic mode functions, and selects the intrinsic mode function with the highest entropy value for reconstruction through the entropy method, removes noise and unimportant components, and obtains the main spatiotemporal features.

[0111] The displacement prediction module inputs the main spatiotemporal features into the Transformer model and uses the powerful spatiotemporal dependency capture capability of the Transformer model to predict the displacement of the highway slope.

Claims

1. A highway slope displacement prediction method, characterized in that: The method comprises: S1. Deploy a variety of ground sensors to obtain slope geological information; Deploy drones to obtain high-resolution images of the slopes; Regularly obtain satellite remote sensing images of the slopes; Establish a comprehensive data collection system to obtain comprehensive data; S2. Use data cleaning algorithms to identify and process outliers and missing values ​​in the comprehensive data to ensure the quality of the comprehensive data; S3. Use convolutional networks combined with long short-term memory neural networks to extract spatiotemporal features from satellite remote sensing images and drone high-resolution images. Use graph neural networks combined with long short-term memory neural networks to extract spatiotemporal features of slope geological information. In S3, convolutional networks are combined with long short-term memory neural networks to extract features: convolutional networks are used to extract spatial features from satellite remote sensing images and drone high-resolution images at each time point; spatial features of images at different time points are arranged in chronological order to form a time series, and then long short-term memory networks are used to extract temporal features; and then the spatiotemporal feature ht is obtained; In S3, graph neural networks combined with long short-term memory neural networks are used to extract features: a graph structure is constructed based on the geographical location and physical relationship of ground sensors, with nodes representing ground sensors and edges representing the relationship between ground sensors; Capture the mutual influence and spatial dependency between monitoring points through graph neural network; Arrange the graph structure spatial features at different time points in chronological order to form a time series, and use the long short-term memory network to extract the time features; then obtain the spatiotemporal feature ht'; S4, fusing the spatiotemporal feature 1 with the spatiotemporal feature 2 to obtain comprehensive spatiotemporal feature data; In S4, the attention mechanism is used to calculate the importance weight of each feature, setting the query vector Q, key vector K and value vector V, Q = WQ-[ht,ht'], K = WK-[ht,ht'], V = WV-[ht,ht'], where WQ, WK and WV are trainable weight matrices; The attention weight A is calculated by weighted fusion through the attention weight A, A=softmax(Q-KT / dk1 / 2), where dk is the dimension of the key vector; The comprehensive spatiotemporal feature data htfused is obtained by weighting the sum value vector of the attention weights, where htfused=AV; S5. Perform integrated empirical mode decomposition on the spatiotemporal feature data, decompose the spatiotemporal feature data into different intrinsic mode functions, select the intrinsic mode function with the highest entropy value for reconstruction using the entropy method, remove noise and unimportant components, and obtain the main spatiotemporal features; In S5, the spatiotemporal feature data htfused is integrated with empirical mode decomposition: Add white noise of different amplitudes to the spatiotemporal feature data htfused to generate multiple different time series; Perform integrated empirical mode decomposition on each time series after adding white noise to obtain several intrinsic mode functions; All intrinsic mode functions are averaged to obtain the final intrinsic mode function; Calculate the entropy value of each intrinsic mode function, and select several intrinsic mode functions with the highest entropy values ​​for reconstruction; Reconstruct the selected intrinsic mode function to form the denoised spatiotemporal features; By fusing spatiotemporal features from different sources and performing integrated empirical mode decomposition and reconstruction after feature fusion, noise and unimportant components are removed to obtain the main spatiotemporal features and improve the accuracy of prediction. S6. Input the main spatiotemporal features into the Transformer model for prediction, and use the spatiotemporal dependency capture capability of the Transformer model to predict highway slope displacement.

2. A highway slope displacement prediction method according to claim 1, characterized in that: The ground sensors include GPS, accelerometer, and inclinometer; The slope geological information includes the precise displacement of the slope, the position information of the slope, the vibration and acceleration change of the slope, and the inclination of the slope; GPS sensors are used to measure the precise displacement and location information of the slope. They are deployed at the top, middle, and bottom of the slope to cover the entire slope area and obtain precise displacement and location information in real time and / or periodically. Accelerometers are used to monitor the vibration and acceleration changes of the slope and detect possible landslide precursors. They are arranged at different heights and key points along the slope to obtain vibration and acceleration data at high frequency. Inclinometers are used to monitor the change in slope inclination and provide information on slope stability. They are placed in areas prone to landslides to obtain slope inclination regularly. The slope geological information is transmitted to the data center in real time using a wireless network.

3. A highway slope displacement prediction method according to claim 1, characterized in that: Regularly obtain satellite remote sensing images of the slope; regularly download the latest satellite remote sensing images from the satellite data platform; and pre-process the satellite remote sensing images.

4. A highway slope displacement prediction method according to claim 1, characterized in that: In S2, the data cleaning algorithm is used to identify and process the comprehensive data: Identify outliers in the synthetic data using the machine learning model Isolation Forest and handle detected outliers and missing values ​​using interpolation methods; Convert sensor data into a unified time series format, Dsensor = {(t i ,s i )} i=1 N , where t i is the timestamp, s i is the sensor reading, Convert the high-resolution image to a uniform pixel matrix format, I = {(x, y, RGB)} x=1,y=1 H,W , where H and W are the height and width of the image, RGB is the pixel value, Convert satellite remote sensing images into a unified raster format, R = {(x, y, Bandk)} x=1,y=1,k=1 H,W,K , where K is the number of bands.

5. A highway slope displacement prediction method according to claim 1, characterized in that: Input the Transformer model in S6 for prediction: The Transformer model is used to predict the main spatiotemporal features. The Transformer model encoder consists of multiple identical sub-layers, each of which contains a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism allows the model to focus on different parts of the input sequence at the same time, decomposing the input vector into three parts: query vector Q, key vector K, and value vector V, calculating the attention weights, and weighted summing the value vector V; The feedforward neural network is used to perform nonlinear transformation on the output of self-attention, which includes two linear layers with a ReLU activation function inserted in between; Before and after each sub-layer, the Transformer model uses layer normalization and residual connections to accelerate convergence and improve model stability; In the prediction task, the main spatiotemporal features are used as the input of the Transformer model encoder. The last layer output of the encoder will contain rich spatiotemporal feature information, which is directly input into a linear layer and finally connected to the output layer for prediction. The output layer uses a linear activation function. During the training phase, the Transformer model learns by minimizing the loss function between the predicted value and the actual slope displacement, and updates the model parameters using the gradient descent method or its variants; The training data uses real historical comprehensive data; Using an end-to-end training approach, the entire model is treated as a whole, and all components from input to output are optimized in the same training process.

6. A highway slope displacement prediction method according to claim 1, characterized in that: Use a staged training approach to first independently train the convolutional network combined with long short-term memory neural network and the graph neural network combined with long short-term memory neural network, freeze their parameters, and then train only the Transformer model.

7. A highway slope displacement prediction system, used to implement a highway slope displacement prediction method according to any one of claims 1 to 6, characterized in that: The system comprises: The data acquisition module is used to deploy a variety of sensors to regularly acquire satellite remote sensing images and drone images of the slopes and build a comprehensive data acquisition system for multi-source data; The data preprocessing module uses data cleaning algorithms to identify and process outliers and missing values ​​in the comprehensive data to ensure the quality and reliability of the comprehensive data; The feature extraction module uses a convolutional network combined with a long short-term memory neural network and a graph neural network combined with a long short-term memory network to extract spatiotemporal features from satellite remote sensing images, drone images, and sensor data, respectively; The feature fusion module combines the spatiotemporal features extracted by the convolutional network combined with the long short-term memory neural network and the graph neural network combined with the long short-term memory neural network to form a comprehensive and multi-level spatiotemporal feature data; The feature reconstruction module performs integrated empirical mode decomposition on the fused spatiotemporal feature data, decomposes the spatiotemporal feature data into different intrinsic mode functions, and selects the intrinsic mode function with the highest entropy value for reconstruction through the entropy method, removes noise and unimportant components, and obtains the main spatiotemporal features; The displacement prediction module inputs the main spatiotemporal features into the Transformer model and uses the spatiotemporal dependency capture capability of the Transformer model to predict the displacement of the highway slope.

Images