Deep learning-based steep slope bridge stability evaluation method and system

Abstract

The invention discloses a deep learning-based abrupt slope bridge stability evaluation method and system. The method comprises the following steps of S1, obtaining bridge area DEM data, bridge displacement data and bridge stress data; s2, constructing a bridge stability evaluation network, and extracting bridge region DEM features based on a convolutional neural network; s3, extracting internal structure features of the bridge based on the recurrent neural network; s4, fusing the DEM features of the bridge region and the features of the internal structure of the bridge based on an attention mechanism; and S5, setting input, output and optimization targets of the network, and optimizing parameters of the bridge stability evaluation network based on an adaptive learning rate optimization algorithm. According to the scheme, the problem of insufficient bridge stability evaluation accuracy is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge monitoring, and in particular to a steep slope bridge stability evaluation method and system based on deep learning. Background Art

[0002] Bridges are the throat of traffic, and the safety of bridge structures involves major property safety and even life safety. With the advancement of bridge service time and the increasing traffic volume, local damage to bridge structures is inevitable, so it is of great significance to monitor the health of bridges.

[0003] However, traditional deep learning-based bridge stability monitoring technology only evaluates the stability of the bridge by analyzing the internal structural parameters of the bridge, without taking into account the characteristics of the complex terrain of steep slopes in mountainous areas, and it is difficult to achieve good results in mountainous steep slope scenarios.

[0004] In the steep slope scene in mountainous areas, the terrain conditions are characterized by large ground height differences, frequent changes, and steep slopes. Therefore, the steep slope bridges in mountainous areas have the characteristics of high adjacent piers and frequent road-bridge interactions. Compared with traditional bridge monitoring scenes, bridge monitoring in steep slope scenes in mountainous areas needs to fully consider the interaction between the detailed structures of the bridge and the terrain. Summary of the invention

[0005] In view of this, the present invention provides a steep slope bridge stability evaluation method and system based on deep learning, aiming to solve the problem that the accuracy of bridge stability monitoring is greatly affected by the terrain in mountainous steep slope scenes. Traditional data fusion methods can only achieve the fusion of various sensor data, but cannot achieve the fusion of multimodal heterogeneous data of sensor data and DEM data. The present invention performs feature representation on the DEM data of the bridge area and performs multimodal fusion with the internal structure data of the bridge, thereby improving the accuracy of bridge stability identification.

[0006] A deep learning-based steep slope bridge stability evaluation method comprises the following steps:

[0007] S1: Obtain bridge area DEM data, bridge displacement data and bridge stress data;

[0008] S2: Construct a bridge stability evaluation network and extract the DEM features of the bridge area based on a convolutional neural network;

[0009] S3: Extracting internal structural features of bridges based on recurrent neural networks;

[0010] S4: Fusion of bridge area DEM features and bridge internal structure features based on attention mechanism;

[0011] S5: Set the network input, output and optimization target, and optimize the network parameters based on the adaptive learning rate optimization algorithm;

[0012] Furthermore, the acquisition of bridge area DEM data, bridge displacement data and bridge stress data in step S1 includes:

[0013] S11: Obtain bridge displacement data and bridge stress data through sensors, and obtain remote sensing image data of the bridge area through satellites;

[0014] S12: Perform image correction, geometric correction, and atmospheric correction on the remote sensing image data of the bridge area, and align the image with the ground control points to generate a stereo image;

[0015] S13: Using stereo images and stereo matching algorithms, DEM data of the bridge area is extracted. DEM (Digital Elevation Model) is the abbreviation of digital elevation model, which is a data structure representing terrain elevation. Through the stereo matching algorithm, the disparity value of each pixel can be extracted from the stereo image, and then the disparity value is converted into an elevation value according to the camera parameters and geometric relationship, so as to obtain the DEM data of the bridge area.

[0016] Furthermore, the step S2 extracts the DEM features of the bridge area based on a convolutional neural network, including:

[0017] S21: Use convolutional neural network, GRU network and attention mechanism to build a bridge stability evaluation network;

[0018] S22: Use two sets of convolutional neural networks composed of convolutional layers, activation functions and pooling layers to convolve the bridge area DEM data, where the structure of the first set of convolutional neural networks is:

[0019] The convolutional layer of the first set of convolutional neural networks uses a convolution kernel of size 3×3, stride 1, and padding 1 to convolve the input image. The number of output channels is 32, and the convolution is calculated as follows:

[0020]

[0021] Among them, O i,j,k is the (i, j)th pixel value of the kth channel of the output feature map after the convolution operation, i and j are the row and column indices of the matrix after convolution, k is the channel index, and x u,v is the (u, v)th element of the original matrix, c i-u,j-vis the (iu, jv)th element of the convolution kernel, and u and v are the indices of the rows and columns of the original matrix; in the first group of convolutional neural networks, a convolution kernel of 3x3, a step size of 1, and a padding of 1 is selected to keep the size of the image unchanged while capturing high-frequency information such as edges and corners corresponding to changes in image details;

[0022] The number of channels determines the number of output feature maps of the convolution layer, that is, the types of features that the convolution layer can extract. The more channels there are, the richer the features that the convolution layer can extract, but it also increases the amount of calculation and memory consumption. Therefore, the number of channels is selected to be 32, which is a moderate size, which can ensure that the convolution layer can extract the spatial and elevation information of the bridge area, while avoiding excessive calculation and memory consumption;

[0023] The activation function of the first group of convolutional neural networks uses the ReLU function to perform a nonlinear transformation on the output of the convolutional layer. The calculation method is:

[0024] A i,j,k =max(0,O i,j,k )

[0025] Among them, A i,j,k is the output value of the activation function; since ReLU only needs to determine whether the input is greater than 0, and does not need to perform complex calculations, the calculation speed is fast. In addition, ReLU can also effectively solve the problem of gradient disappearance, and by allowing more 0 values ​​in the network output, enhance the sparsity of the network, reduce network parameters and calculation amount, and improve the generalization ability of the network;

[0026] The pooling layer of the first set of convolutional neural networks uses maximum pooling to downsample the output of the activation function. The size of the pooling kernel is 2×2, the step size is 2, and the number of output channels is 32. The calculation method is:

[0027]

[0028] Among them, P i,j,k is the output value of the pooling layer, m and n are counting indexes. After pooling is completed, the output value of the pooling layer is input into the second group of convolutional neural networks. Maximum pooling can divide the input feature map into several small areas, and then select the largest value in each small area as the output, which is conducive to focusing on the spatial and elevation change characteristics of each part of the bridge area. At the same time, the translation invariance of the feature map is increased, that is, when the input satellite image of the bridge area undergoes a small change in shooting angle, the output of the maximum pooling will not change significantly, thereby improving the robustness of the network.

[0029] The structure of the second group of convolutional neural networks is:

[0030] The convolutional layer of the second convolutional neural network uses a convolution kernel of size 5×5, stride 1, and padding 2 to convolve the output of the pooling layer of the first convolutional neural network, with an output channel number of 64;

[0031] The activation function and pooling layer settings of the second group of convolutional neural networks are the same as those of the first group of convolutional neural networks; the convolutional layer, activation function, and pooling layer calculation methods of the second group of convolutional neural networks are the same as those of the first group of convolutional neural networks;

[0032] In the second set of convolutional neural networks, a convolution kernel of 5x5, stride 1, and padding 2 is selected to increase the receptive field, that is, the range of pixels covered by the convolution kernel, while capturing low-frequency information such as texture and shape corresponding to smooth changes in the image;

[0033] At the same time, since the second set of convolutional neural networks needs to extract higher-level DEM features of the bridge area and is based on the first set of convolutional neural networks, it is necessary to increase the number of output channels to ensure that the convolution layer has sufficient feature extraction capabilities; and the first set of convolutional neural networks has been pooled to reduce the size of the feature map, so the second set of convolutional neural networks has 64 output channels, which will not bring too much calculation and memory consumption;

[0034] S23: Use the fully connected layer and activation function to compress the output of the second set of convolutional layers into a one-dimensional vector as a representation of the DEM features of the bridge area. The calculation method of the fully connected layer is:

[0035]

[0036] F=[F k＝1 ,F k＝2 ,…,F k＝k ]

[0037] Among them, F k is the pixel value of the kth channel of the output feature map of the fully connected layer, W i,j,k is the weight value of the fully connected layer, b i,j,k is the bias term of the fully connected layer, N is the length of the vector after flattening the pooling layer, F is the concatenation result of the pixel values ​​of each channel of the fully connected layer, [] is the concatenation operation, and the fully connected layer compresses the output feature map into a one-dimensional vector, realizing the dimensionality reduction and abstraction of the DEM features of the bridge area, and then combines the features of different channels through the concatenation operation, thereby enhancing the feature discrimination, paving the way for the subsequent feature fusion based on the Attention mechanism.

[0038] Furthermore, the step S3 extracts the internal structural features of the bridge based on a recurrent neural network, including:

[0039] S31: The bridge displacement data and the bridge stress data are concatenated into a two-dimensional matrix, where each row represents the data of a time step and each column represents the data of a measuring point, as the input of the recurrent neural network;

[0040] S32: Use the GRU network to perform sequence modeling on the input vector, and use the output hidden state vector as the representation of the internal structural features of the bridge. The calculation method of GRU is:

[0041] Renew t =sigmoid(W z ·[hidden t-1 ,input t ])

[0042] Reset t =sigmoid(W r ·[hidden t-1 ,inpu t ])

[0043]

[0044] Among them, Renew t is the update gate of the current time step, t is the current time step, sigmoid() is the sigmoid function, W z is the weight matrix, hidden t-1 is the hidden state of the previous moment, input t is the input of the current time step, Reset t is the reset gate for the current time step, W r is the weight matrix, is the candidate hidden state of the current time step, tanh() is the tanh function, W is the weight matrix, ⊙ is the element multiplication, hidden t is the hidden state of the current time step; GRU dynamically adjusts the flow of information by updating gates and resetting gates, transforms the time correlation of bridge displacement data and bridge stress data into a hidden state vector, and adaptively learns the complexity and diversity of the internal structure of the bridge, thereby capturing the long-term dependency and short-term changes of the input sequence, while avoiding the problem of gradient disappearance or explosion;

[0045] Furthermore, the S4 step integrates the DEM features of the bridge area and the internal structural features of the bridge based on the attention mechanism, including:

[0046] S41: splicing the DEM features of the bridge area and the internal structure features of the bridge to form a two-dimensional matrix;

[0047] S42: Use the fully connected layer and softmax function to calculate the attention weight for each row of the matrix, indicating the importance of the features in that row. The attention weight is calculated as follows:

[0048] a t =softmax(W a [hidden t ,F]+b a )

[0049] Among them, a t is the attention weight vector, softmax() is the softmax function, W a is the weight matrix, b a is the bias vector;

[0050] S43: Use another fully connected layer and sigmoid function to transform each column of the matrix to obtain a new feature matrix, and then perform weighted summation on the matrix according to the attention weight to obtain a one-dimensional vector as a fusion representation of the DEM features of the bridge area and the internal structure features of the bridge. The calculation method involved is:

[0051]

[0052] Among them, I is the fusion representation vector, and T is the total step length of the time step; the Attention mechanism can adaptively weight the input features, highlight important features, suppress irrelevant features, and integrate the different dimensions of the bridge area DEM features and the internal structure features of the bridge, so as to obtain a more comprehensive and accurate bridge feature representation to reflect the overall state and local changes of the bridge, and provide richer and more reliable information for subsequent bridge health assessment;

[0053] Furthermore, the step S5 sets the input, output and optimization target of the network, and optimizes the network parameters based on an adaptive learning rate optimization algorithm, including:

[0054] S51: The fusion representation of the bridge area DEM features and the internal structural features of the bridge is used as the input of the network, and the bridge stability rating is used as the output of the network;

[0055] S52: Select the cross entropy loss function L CNN As the loss function of CNN, the smooth loss function L GRU As the loss function of GRU, it is calculated as:

[0056]

[0057] Among them, H is the number of samples, D is the number of categories of bridge stability rating, and y hdis the true label of the d-th category of the h-th sample, p hd is the predicted probability of the d-th category of the h-th sample, is the predicted value of the hth sample at the tth time step, y ht is the true value of the h-th sample at the t-th time step, σ is a smoothing function, and is calculated as:

[0058]

[0059] Among them, f is the input of the smoothing function; the smoothing function can use a quadratic function to approximate the original function near the inflection point, reduce the sensitivity of the GRU loss function to outliers, and improve the iterative stability of the model;

[0060] S53: Take the weighted sum of the CNN loss function and the GRU loss function as the optimization target to obtain the total loss function of the network. The final loss function is:

[0061] L total =L CNN +L GRU

[0062] S54: Optimize the parameters of the bridge stability evaluation network based on the adaptive learning rate optimization algorithm. The calculation method is:

[0063] q p =β1·q p-1 +(1-β1)·g p

[0064]

[0065] Among them, q p is the first-order moment estimate, p is the number of iterations, q p-1 is the first-order moment estimate of the previous iteration step, β1 and β2 are the attenuation coefficients, g p is the gradient of the current iteration, s p is the second-order moment estimate, s p-1 is the second-order moment estimate of the previous iteration, and is the deviation correction of the first-order moment and the second-order moment, η is the learning rate, θ p is the parameter of the bridge stability evaluation network, θ p+1 is the parameter of the bridge stability evaluation network after iteration, ∈ is a constant.

[0066] The present invention also provides a steep slope bridge stability evaluation system based on deep learning, comprising:

[0067] Data acquisition module: obtain bridge area DEM data, bridge displacement data and bridge stress data;

[0068] DEM feature extraction module: extracts DEM features of bridge area based on convolutional neural network;

[0069] Structural feature extraction module: extracts the internal structural features of the bridge based on recurrent neural network;

[0070] Feature fusion module: Fusion of bridge area DEM features and bridge internal structure features based on attention mechanism;

[0071] Network training module: Set the network input, output and optimization target, and optimize the network parameters based on the adaptive learning rate optimization algorithm.

[0072] Compared with the prior art, the advantages of the present invention are:

[0073] (1) In terms of the overall solution, the present invention integrates the DEM data of the bridge area and the internal structure data of the bridge into multimodal heterogeneous data, comprehensively considers the influence of the terrain and the internal structure characteristics of the bridge, and realizes the evaluation of the bridge stability.

[0074] (2) In terms of the algorithm model, the present invention first uses CNN to extract the DEM features of the bridge area and converts the image data into vector data to complete the structural unification of heterogeneous data; secondly, GRU is used to extract the internal structural features of the bridge, and finally the attention mechanism is used to judge the importance of the features and perform adaptive fusion to achieve the fusion of multi-dimensional heterogeneous data. While ensuring a low number of parameters, the accuracy and efficiency of bridge stability evaluation are improved.

[0075] (3) In terms of network parameter optimization, the present invention adopts a new loss function, combined with the cross entropy loss function and the smoothing loss function, to optimize each part of the bridge stability evaluation network respectively, thereby improving the model training effect and fitting effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A schematic diagram of a process flow of a steep slope bridge stability evaluation method based on deep learning provided by the present invention;

[0077] Figure 2 A schematic diagram of the algorithm flow of a steep slope bridge stability evaluation method based on deep learning provided by the present invention. DETAILED DESCRIPTION

[0078] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0079] Embodiment 1: A method for evaluating the stability of a steep slope bridge based on deep learning, such as Figure 1 As shown, the following steps are included:

[0080] S1: Obtain bridge area DEM data, bridge displacement data and bridge stress data;

[0081] S11: Obtain bridge displacement data and bridge stress data through sensors, and obtain remote sensing image data of the bridge area through satellites;

[0082] S12: Perform image correction, geometric correction, and atmospheric correction on the remote sensing image data of the bridge area, and align the image with the ground control points to generate a stereo image;

[0083] S13: Using stereo images and stereo matching algorithms, DEM data of the bridge area is extracted. DEM (Digital Elevation Model) is the abbreviation of digital elevation model, which is a data structure representing terrain elevation. Through the stereo matching algorithm, the disparity value of each pixel can be extracted from the stereo image, and then the disparity value is converted into an elevation value according to the camera parameters and geometric relationship, so as to obtain the DEM data of the bridge area.

[0084] S2: Construct a bridge stability evaluation network and extract the DEM features of the bridge area based on a convolutional neural network;

[0085] S21: Use convolutional neural network, GRU network and attention mechanism to build a bridge stability evaluation network;

[0086] S22: Use two sets of convolutional neural networks composed of convolutional layers, activation functions and pooling layers to convolve the bridge area DEM data, where the structure of the first set of convolutional neural networks is:

[0087] The convolutional layer of the first set of convolutional neural networks uses a convolution kernel of size 3×3, stride 1, and padding 1 to convolve the input image. The number of output channels is 32, and the convolution is calculated as follows:

[0088]

[0089] Among them, O i,j,k is the (i, j)th pixel value of the kth channel of the output feature map after the convolution operation, i and j are the row and column indices of the matrix after convolution, k is the channel index, and x u,v is the (u, v)th element of the original matrix, c i-u,j-v is the (iu, jv)th element of the convolution kernel, and u and v are the indices of the rows and columns of the original matrix; in the first group of convolutional neural networks, a convolution kernel of 3x3, a step size of 1, and a padding of 1 is selected to keep the size of the image unchanged while capturing high-frequency information such as edges and corners corresponding to changes in image details;

[0090] The number of channels determines the number of output feature maps of the convolution layer, that is, the types of features that the convolution layer can extract. The more channels there are, the richer the features that the convolution layer can extract, but it also increases the amount of calculation and memory consumption. Therefore, the number of channels is selected to be 32, which is a moderate size, which can ensure that the convolution layer can extract the spatial and elevation information of the bridge area, while avoiding excessive calculation and memory consumption;

[0091] The activation function of the first group of convolutional neural networks uses the ReLU function to perform a nonlinear transformation on the output of the convolutional layer. The calculation method is:

[0092] A i,j,k =max(0,O i,j,k )

[0093] Among them, A i,j,k is the output value of the activation function; since ReLU only needs to determine whether the input is greater than 0, and does not need to perform complex calculations, the calculation speed is fast. In addition, ReLU can also effectively solve the problem of gradient disappearance, and by allowing more 0 values ​​in the network output, enhance the sparsity of the network, reduce network parameters and calculation amount, and improve the generalization ability of the network;

[0094] The pooling layer of the first set of convolutional neural networks uses maximum pooling to downsample the output of the activation function. The size of the pooling kernel is 2×2, the step size is 2, and the number of output channels is 32. The calculation method is:

[0095]

[0096] Among them, P i,j,k is the output value of the pooling layer, m and n are counting indexes. After pooling is completed, the output value of the pooling layer is input into the second group of convolutional neural networks. Maximum pooling can divide the input feature map into several small areas, and then select the largest value in each small area as the output, which is conducive to focusing on the spatial and elevation change characteristics of each part of the bridge area. At the same time, the translation invariance of the feature map is increased, that is, when the input satellite image of the bridge area undergoes a small change in shooting angle, the output of the maximum pooling will not change significantly, thereby improving the robustness of the network.

[0097] The structure of the second group of convolutional neural networks is:

[0098] The convolutional layer of the second convolutional neural network uses a convolution kernel of size 5×5, stride 1, and padding 2 to convolve the output of the pooling layer of the first convolutional neural network, with an output channel number of 64;

[0099] The activation function and pooling layer settings of the second group of convolutional neural networks are the same as those of the first group of convolutional neural networks; the convolutional layer, activation function, and pooling layer calculation methods of the second group of convolutional neural networks are the same as those of the first group of convolutional neural networks;

[0100] In the second set of convolutional neural networks, a convolution kernel of 5x5, stride 1, and padding 2 is selected to increase the receptive field, that is, the range of pixels covered by the convolution kernel, while capturing low-frequency information such as texture and shape corresponding to smooth changes in the image;

[0101] At the same time, since the second set of convolutional neural networks needs to extract higher-level DEM features of the bridge area and is based on the first set of convolutional neural networks, it is necessary to increase the number of output channels to ensure that the convolution layer has sufficient feature extraction capabilities; and the first set of convolutional neural networks has been pooled to reduce the size of the feature map, so the second set of convolutional neural networks has 64 output channels, which will not bring too much calculation and memory consumption;

[0102] S23: Use the fully connected layer and activation function to compress the output of the second set of convolutional layers into a one-dimensional vector as a representation of the DEM features of the bridge area. The calculation method of the fully connected layer is:

[0103]

[0104] F=[F k＝1 ,F k＝2 ,…,F k＝k ]

[0105] Among them, F k is the pixel value of the kth channel of the output feature map of the fully connected layer, W i,j,k is the weight value of the fully connected layer, b i,j,k is the bias term of the fully connected layer, N is the length of the vector after flattening the pooling layer, F is the concatenation result of the pixel values ​​of each channel of the fully connected layer, [] is the concatenation operation, and the fully connected layer compresses the output feature map into a one-dimensional vector, realizing the dimensionality reduction and abstraction of the DEM features of the bridge area, and then combines the features of different channels through the concatenation operation, thereby enhancing the feature discrimination, paving the way for the subsequent feature fusion based on the Attention mechanism.

[0106] For example, assuming that the length and width of the DEM data are 256 respectively, after the first convolution layer, the size of the output feature map is 256x256x32, and the dimension is 3; after the first pooling layer, the size of the output feature map is 128x128x32, and the dimension is 3; after the second convolution layer, the size of the output feature map is 128x128x64, and the dimension is 3; after the second pooling layer, the size of the output feature map is 64x64x64, and the dimension is 3; after the fully connected layer, the size of the output feature map is 1x1x4096, and the dimension is 3, which represents the DEM features of the bridge area.

[0107] S3: Extracting internal structural features of bridges based on recurrent neural networks;

[0108] S31: The bridge displacement data and the bridge stress data are concatenated into a two-dimensional matrix, where each row represents the data of a time step and each column represents the data of a measuring point, as the input of the recurrent neural network;

[0109] Assume that there are X measuring points on the bridge, and each measuring point has two data, displacement and stress. Then the data of each time step is a 2X-dimensional vector. By concatenating the data of T time steps, we get a T×2X matrix as the input of the recurrent neural network.

[0110] For example, assuming that the bridge has 4 measuring points, and the displacement and stress data of each measuring point are shown in the following table, then the input of GRU at each time step is:

[0111] Time step Displacement of measuring point 1 Stress at measuring point 1 Displacement of measuring point 2 Stress at measuring point 2 Displacement of measuring point 3 Stress at measuring point 3 Measuring point 4 displacement Stress at measuring point 4 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 2 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 3 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

[0112] S32: Use the GRU network to perform sequence modeling on the input vector, and use the output hidden state vector as the representation of the internal structural features of the bridge. The calculation method of GRU is:

[0113] Renew t =sigmoid(W z ·[hidden t-1 ,input t ])

[0114] Reset t =sigmoid(W r ·[hidden t-1 ,input t ])

[0115]

[0116] Among them, Renew tis the update gate of the current time step, t is the current time step, sigmoid() is the sigmoid function, W z is the weight matrix, hidden t-1 is the hidden state of the previous moment, input t is the input of the current time step, Reset t is the reset gate for the current time step, W r is the weight matrix, is the candidate hidden state of the current time step, tanh() is the tanh function, W is the weight matrix, ⊙ is the element multiplication, hidden t is the hidden state of the current time step; GRU dynamically adjusts the flow of information by updating gates and resetting gates, transforms the time correlation of bridge displacement data and bridge stress data into a hidden state vector, and adaptively learns the complexity and diversity of the internal structure of the bridge, thereby capturing the long-term dependency and short-term changes of the input sequence, while avoiding the problem of gradient disappearance or explosion;

[0117] S4: Fusion of bridge area DEM features and bridge internal structure features based on attention mechanism, such as Figure 2 As shown;

[0118] S41: splicing the DEM features of the bridge area and the internal structure features of the bridge to form a two-dimensional matrix;

[0119] S42: Use the fully connected layer and softmax function to calculate the attention weight for each row of the matrix, indicating the importance of the features in that row. The attention weight is calculated as follows:

[0120] a t =softmax(W a [hidden t ,F]+b a )

[0121] Among them, a t is the attention weight vector, softmax() is the softmax function, W a is the weight matrix, b a is the bias vector;

[0122] S43: Use another fully connected layer and sigmoid function to transform each column of the matrix to obtain a new feature matrix, and then perform weighted summation on the matrix according to the attention weight to obtain a one-dimensional vector as a fusion representation of the DEM features of the bridge area and the internal structure features of the bridge. The calculation method involved is:

[0123]

[0124] Among them, I is the fusion representation vector, and T is the total step length of the time step; the Attention mechanism can adaptively weight the input features, highlight important features, suppress irrelevant features, and integrate the different dimensions of the bridge area DEM features and the internal structure features of the bridge, so as to obtain a more comprehensive and accurate bridge feature representation to reflect the overall state and local changes of the bridge, and provide richer and more reliable information for subsequent bridge health assessment;

[0125] For example, there are two sets of DEM data with a length and width of 256, which represent different terrain features of the bridge area. After the convolutional neural network and the fully connected layer, two one-dimensional vectors with a length of 4096 are obtained as the representation of the DEM features of the bridge area, denoted as F. (1) and F (2) Similarly, after the two sets of bridge displacement data and bridge stress data pass through GRU, two one-dimensional vectors with a length of 1024 will be generated, which represent the characteristics of the internal structure of the bridge, etc., recorded as H (1) and H (2) ;

[0126] The DEM features of the bridge area and the internal structure features of the bridge are spliced ​​to form two two-dimensional matrices of size 5120x2, denoted as M (1) and M (2) ;

[0127] After attention calculation, two sets of attention weights are obtained, namely a (1) =[0.38,0.62], a (2) =[0.17,0.83]; it can be seen that a (2) The first element of a (1) The first element of is small, which means that for the second set of data, the DEM features have less influence; while the internal structural features of the bridge have a greater influence and are more important for the health assessment of the bridge;

[0128] The classification results of bridge health assessment are:

[0129] I<0.7: bridge health;

[0130] 0.7<=I<0.8: The bridge is slightly damaged;

[0131] 0.8<=I<0.85: The bridge is moderately damaged;

[0132] 0.85<=I<0.9: The bridge is seriously damaged;

[0133] 0.9<=I<1: The bridge damage is unacceptable;

[0134] S5: Set the network input, output and optimization target, and optimize the network parameters based on the adaptive learning rate optimization algorithm;

[0135] S51: The fusion representation of the bridge area DEM features and the internal structural features of the bridge is used as the input of the network, and the bridge stability rating is used as the output of the network;

[0136] S52: Select the cross entropy loss function L CNN As the loss function of CNN, the smooth loss function L GRU As the loss function of GRU, it is calculated as:

[0137]

[0138] Among them, H is the number of samples, D is the number of categories of bridge stability rating, and y hd is the true label of the d-th category of the h-th sample, p hd is the predicted probability of the d-th category of the h-th sample, is the predicted value of the hth sample at the tth time step, y ht is the true value of the h-th sample at the t-th time step, σ is a smoothing function, and is calculated as:

[0139]

[0140] Among them, f is the input of the smoothing function; the smoothing function can use a quadratic function to approximate the original function near the inflection point, reduce the sensitivity of the GRU loss function to outliers, and improve the iterative stability of the model;

[0141] S53: Take the weighted sum of the CNN loss function and the GRU loss function as the optimization target to obtain the total loss function of the network. The final loss function is:

[0142] L total =L CNN +L GRU

[0143] S54: Optimize the parameters of the bridge stability evaluation network based on the adaptive learning rate optimization algorithm. The calculation method is:

[0144] q p =β1·q p-1 +(1-β1)·g p

[0145]

[0146] Among them, q p is the first-order moment estimate, p is the number of iterations, q p-1is the first-order moment estimate of the previous iteration step, β1 and β2 are the attenuation coefficients, g p is the gradient of the current iteration, s p is the second-order moment estimate, s p-1 is the second-order moment estimate of the previous iteration, and is the deviation correction of the first-order moment and the second-order moment, η is the learning rate, θ p is the parameter of the bridge stability evaluation network, θ p+1 is the parameter of the bridge stability evaluation network after iteration, ∈ is a constant.

[0147] Embodiment 2: The present invention also discloses a steep slope bridge stability evaluation system based on deep learning, comprising:

[0148] Data acquisition module: obtain bridge area DEM data, bridge displacement data and bridge stress data;

[0149] DEM feature extraction module: extracts DEM features of bridge area based on convolutional neural network;

[0150] Structural feature extraction module: extracts the internal structural features of the bridge based on recurrent neural network;

[0151] Feature fusion module: Fusion of bridge area DEM features and bridge internal structure features based on attention mechanism;

[0152] Network training module: Set the network input, output and optimization target, and optimize the network parameters based on the adaptive learning rate optimization algorithm.

[0153] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0155] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A steep slope bridge stability evaluation method based on deep learning, characterized in that: The following steps are involved: S1: Obtain bridge area DEM data, bridge displacement data and bridge stress data; S2: Construct a bridge stability evaluation network and extract the DEM features of the bridge area based on a convolutional neural network; S3: Extracting internal structural features of bridges based on recurrent neural networks; S4: Fusion of bridge area DEM features and bridge internal structure features based on attention mechanism; S5: Set the network input, output and optimization target, and optimize the network parameters based on the adaptive learning rate optimization algorithm.

2. The steep slope bridge stability evaluation method based on deep learning according to claim 1 is characterized in that: The step S1 comprises the following steps: S11: Obtain bridge displacement data and bridge stress data through sensors, and obtain remote sensing image data of the bridge area through satellites; S12: Perform image correction, geometric correction, and atmospheric correction on the remote sensing image data of the bridge area, and align the image with the ground control points to generate a stereo image; S13: Using stereo images and stereo matching algorithm, DEM data of the bridge area is extracted.

3. The steep slope bridge stability evaluation method based on deep learning according to claim 2 is characterized in that: The step S2 comprises the following steps: S21: Use convolutional neural network, GRU network and attention mechanism to build a bridge stability evaluation network; S22: Use two sets of convolutional neural networks composed of convolutional layers, activation functions and pooling layers to convolve the bridge area DEM data, where the structure of the first set of convolutional neural networks is: The convolutional layer of the first set of convolutional neural networks uses a convolution kernel of size 3×3, stride 1, and padding 1 to convolve the input image. The number of output channels is 32, and the convolution is calculated as follows: Among them, O i,j,k is the (i, j)th pixel value of the kth channel of the output feature map after the convolution operation, i and j are the row and column indices of the matrix after convolution, k is the channel index, and x u,v is the (u, v)th element of the original matrix, c i-u,j-v is the (iu, jv)th element of the convolution kernel, where u and v are the indices of the rows and columns of the original matrix; The activation function of the first group of convolutional neural networks uses the ReLU function to perform a nonlinear transformation on the output of the convolutional layer. The calculation method is: A i,j,k =max(0,O i,j,k ) Among them, A i,j,k is the output value of the activation function; The pooling layer of the first set of convolutional neural networks uses maximum pooling to downsample the output of the activation function. The size of the pooling kernel is 2×2, the step size is 2, and the number of output channels is 32. The calculation method is: Among them, P i,j,k is the output value of the pooling layer, m and n are counting indexes. After pooling, the output value of the pooling layer is input into the second set of convolutional neural networks; The structure of the second group of convolutional neural networks is: The convolutional layer of the second convolutional neural network uses a convolution kernel of size 5×5, stride 1, and padding 2 to convolve the output of the pooling layer of the first convolutional neural network, with an output channel number of 64; The activation function and pooling layer settings of the second group of convolutional neural networks are the same as those of the first group of convolutional neural networks; the convolutional layer, activation function, and pooling layer calculation methods of the second group of convolutional neural networks are the same as those of the first group of convolutional neural networks; S23: Use the fully connected layer and activation function to compress the output of the second set of convolutional layers into a one-dimensional vector as a representation of the DEM features of the bridge area. The calculation method of the fully connected layer is: F=[F k＝1 ,F k＝2 ,…,F k＝k ] Among them, F k is the pixel value of the kth channel of the output feature map of the fully connected layer, W i,j,k is the weight value of the fully connected layer, b i,j,k is the bias term of the fully connected layer, N is the length of the vector after flattening the pooling layer, F is the concatenation result of the pixel values ​​of each channel of the fully connected layer, and [] is the concatenation operation.

4. The steep slope bridge stability evaluation method based on deep learning according to claim 3 is characterized in that: The step S3 comprises the following steps: S31: The bridge displacement data and the bridge stress data are concatenated into a two-dimensional matrix, where each row represents the data of a time step and each column represents the data of a measuring point, as the input vector of the recurrent neural network; S32: Use the GRU network to perform sequence modeling on the input vector, and use the output hidden state vector as the representation of the internal structural features of the bridge. The calculation method of GRU is: Renew t =sigmoid(W z ·[hidden t-1 ,input t ]) Reset t =sigmoid(W r ·[hidden t-1 ,inpu t ]) Among them, Renew t is the update gate of the current time step, t is the current time step, sigmoid() is the sigmoid function, W z is the weight matrix, hidden t-1 is the hidden state of the previous moment, input t is the input of the current time step, Reset t is the reset gate for the current time step, W r is the weight matrix, is the candidate hidden state of the current time step, tanh() is the tanh function, W is the weight matrix, ⊙ is the element multiplication, hidden t is the hidden state at the current time step.

5. The steep slope bridge stability evaluation method based on deep learning according to claim 4 is characterized in that: The step S4 comprises the following steps: S41: splicing the DEM features of the bridge area and the internal structural features of the bridge to form a two-dimensional matrix; S42: Use the fully connected layer and softmax function to calculate the attention weight for each row of the matrix. The attention weight is calculated as follows: a t =softmax(W a [hidden t ,F]+b a ) Among them, a t is the attention weight vector, softmax() is the softmax function, W a is the weight matrix, b a is the bias vector; S43: Use another fully connected layer and sigmoid function to transform each column of the matrix to obtain a new feature matrix, and then perform weighted summation on the matrix according to the attention weight to obtain a one-dimensional vector as a fusion representation of the DEM features of the bridge area and the internal structure features of the bridge. The calculation method involved is: Among them, I is the fusion representation vector and T is the total step size of the time step.

6. The steep slope bridge stability evaluation method based on deep learning according to claim 5 is characterized in that: The step S5 comprises the following steps: S51: The fusion representation of the bridge area DEM features and the internal structural features of the bridge is used as the input of the network, and the bridge stability rating is used as the output of the network; S52: Select the cross entropy loss function L CNN As the loss function of CNN, the smooth loss function L GRU As the loss function of GRU, it is calculated as: Among them, H is the number of samples, D is the number of categories of bridge stability rating, and y hd is the true label of the d-th category of the h-th sample, p hd is the predicted probability of the d-th category of the h-th sample, is the predicted value of the hth sample at the tth time step, y ht is the true value of the h-th sample at the t-th time step, σ is a smoothing function, and is calculated as: Where f is the input of the smoothing function; S53: Take the weighted sum of the CNN loss function and the GRU loss function as the optimization target to obtain the total loss function of the network. The final loss function is: L total =L CNN +L GRU S54: Optimize the parameters of the bridge stability evaluation network based on the adaptive learning rate optimization algorithm. The calculation method is: q p =β1·q p-1 +(1-β1)·g p Among them, q p is the first-order moment estimate, p is the number of iterations, q p-1 is the first-order moment estimate of the previous iteration step, β1 and β2 are the attenuation coefficients, g p is the gradient of the current iteration, s p is the second-order moment estimate, s p-1 is the second-order moment estimate of the previous iteration, and is the deviation correction of the first-order moment and the second-order moment, η is the learning rate, θ p is the parameter of the bridge stability evaluation network, θ p+1 is the parameter of the bridge stability evaluation network after iteration, ∈ is a constant.

7. A steep slope bridge stability evaluation system based on deep learning, characterized in that: include: Data acquisition module: obtain bridge area DEM data, bridge displacement data and bridge stress data; DEM feature extraction module: extracts DEM features of bridge area based on convolutional neural network; Structural feature extraction module: extracts the internal structural features of the bridge based on recurrent neural network; Feature fusion module: Fusion of bridge area DEM features and bridge internal structure features based on attention mechanism; Network training module: set the network input, output and optimization target, and optimize the network parameters based on the adaptive learning rate optimization algorithm; To realize the steep slope bridge stability evaluation method based on deep learning as described in any of claims 1-6.

Images