Bridge state recognition method and device, equipment and medium

Abstract

The invention discloses a bridge state recognition method, device and equipment and a medium, and the method comprises the steps: obtaining the observation data of a bridge, and the observation data comprise bridge parameter data and bridge image data; inputting the observation data into an improved convolutional neural network-long and short term memory network model to obtain a bridge state prediction value; and determining the state of the bridge based on the bridge state predicted value, and fully observing the state change of the bridge data by using an improved convolutional neural network-long and short term memory network model, thereby improving the accuracy and reliability of bridge state recognition.

Description

Technical Field

[0001] The present disclosure generally relates to the field of bridge monitoring technology, and in particular to a bridge status identification method, device, equipment and medium. Background Art

[0002] Bridges are extremely important infrastructure and core hubs in the field of transportation, and they play an important role in promoting my country's economic and social development. In related technologies, sensor equipment is usually used to monitor bridge parameters, such as temperature and humidity, pressure, vibration, load, and inclination, in order to monitor the status of the bridge. However, direct massive sensor data cannot intuitively reflect the status of the bridge building. Summary of the invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide a bridge state recognition method that can utilize an improved convolutional neural network-long short-term memory network model to fully observe the state changes of bridge data and improve the accuracy and reliability of bridge state recognition.

[0004] In a first aspect, an embodiment of the present application provides a bridge state identification method, comprising:

[0005] Acquire observation data of the bridge, wherein the observation data includes bridge parameter data and bridge image data;

[0006] Inputting the observed data into an improved convolutional neural network-long short-term memory network model to obtain a predicted value of the bridge state;

[0007] Based on the bridge state prediction value, the state of the bridge is determined.

[0008] In some embodiments, the improved convolutional neural network-long short-term memory network model includes:

[0009] A convolutional neural network unit is used to extract features from the bridge image data to obtain bridge image features;

[0010] A long short-term memory network unit is used to extract features from the bridge parameter data to obtain bridge parameter features;

[0011] A feature fusion output unit is used to fuse the bridge image feature and the bridge parameter feature and output the bridge state prediction value.

[0012] In some embodiments, the convolutional neural network-long short-term memory network model is improved by using an attention mechanism to obtain the improved convolutional neural network-long short-term memory network model;

[0013] Wherein, a first attention subunit is set after the convolution layer and the pooling layer of the convolutional neural network unit;

[0014] A second attention subunit is arranged after at least one long short-term memory layer of the long short-term memory network unit.

[0015] In some embodiments, the improved convolutional neural network-long short-term memory network model adopts a regularized cross-entropy loss function during training.

[0016] In some embodiments, the loss function uses the following formula:

[0017]

[0018] Among them, L total is the loss function value, η is the loss function correction coefficient, θ i is the i-th parameter of the improved convolutional neural network-long short-term memory network model.

[0019] In some embodiments, determining the state of the bridge based on the bridge state prediction value includes:

[0020] When a prediction residual between the predicted bridge state value and the actual bridge state value is greater than a state warning threshold, determining that the bridge state is abnormal;

[0021] Wherein, the state warning threshold is related to the prediction residual.

[0022] In a second aspect, an embodiment of the present application provides a bridge state identification device, including:

[0023] An acquisition module, used for acquiring observation data of the bridge, wherein the observation data includes bridge parameter data and bridge image data;

[0024] A prediction module, used for inputting the observed data into an improved convolutional neural network-long short-term memory network model to obtain a predicted value of the bridge state;

[0025] A determination module is used to determine the state of the bridge based on the bridge state prediction value.

[0026] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the embodiment of the present application when executing the program.

[0027] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the embodiment of the present application.

[0028] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, it implements the method described in the embodiment of the present application.

[0029] The bridge state identification method, device, equipment and medium proposed in the embodiments of the present application can use the improved convolutional neural network-long short-term memory network model to predict the acquired bridge observation data, obtain the bridge state prediction value, and then determine the state of the bridge based on the bridge state prediction value, thereby effectively improving the accuracy and reliability of the bridge state prediction.

[0030] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0032] Figure 1 A schematic diagram of a flow chart of a bridge state identification method provided by an embodiment of the present application is shown;

[0033] Figure 2 A schematic diagram of the structure of an improved convolutional neural network-long short-term memory network model provided in one embodiment of the present application is shown;

[0034] Figure 3 A schematic diagram of the structure of a bridge state identification device provided by an embodiment of the present application is shown;

[0035] Figure 4 A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing an embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0037] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0038] In order to further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation instruction steps shown in the following embodiments or drawings, more or less operation instruction steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present application. The method may be executed in the order of the method shown in the embodiment or drawings or in parallel during the actual processing process or when the device is executed.

[0039] Please refer to Figure 1 , Figure 1 FIG. 1 is a flow chart of a bridge state identification method provided by an embodiment of the present application. Figure 1 As shown, the method includes:

[0040] Step 101, obtaining observation data of the bridge.

[0041] The observation data of the bridge includes bridge parameter data and bridge image data.

[0042] Optionally, the bridge parameter data is the sensor data collected by the sensor, including but not limited to the strain value of the detection point, the displacement value of the vehicle, the position value of the bridge support, the vehicle acceleration, the bridge vibration acceleration, the temperature and other continuous data reflecting the state parameters of the bridge or the train (train) running on the bridge, that is, time series data. The bridge image data is the bridge surface image collected by the image acquisition device, which is used to collect cracks, corrosion and other surface damage on the bridge surface.

[0043] It should also be understood that the change in bridge state is usually caused by the bridge being affected by external factors, such as strain values ​​including but not limited to vehicle-induced strain values ​​and temperature-induced strain values. In the embodiment of the present application, the overall bridge state is analyzed without distinguishing the reasons for the change in bridge state.

[0044] In a feasible embodiment, since the bridge parameter data comes from both the bridge itself and the trains (trains) running on the bridge, the bridge parameter data have different dimensions and dimensional units, which will affect the accuracy of the bridge status analysis. Therefore, the embodiment of the present application performs standardized processing on the bridge parameter data to obtain bridge parameter data that can be used for data analysis.

[0045] Specifically, in the example of the present application, the multi-dimensional bridge parameter data is standardized by making them have zero mean and unit variance. For example, the following formula can be used:

[0046]

[0047] Among them, Xs ' eq is the bridge parameter data standardized in a certain dimension, X seq is the original bridge parameter data of a certain dimension, μ seq is the mean value of bridge parameter data in a certain dimension, σ seq is the standard deviation of bridge parameter data in a certain dimension.

[0048] Furthermore, based on the same reason, different sensors collect bridge parameter data at different frequencies. Therefore, the embodiment of the present application further performs time alignment on the standardized bridge parameter data by interpolation or resampling to ensure that the bridge parameter data are on the same time scale, that is, to ensure that the bridge parameter data from the bridge itself or from the vehicles (trains) traveling on the bridge are on the same time scale.

[0049] In another feasible embodiment, the bridge image data also has differences. The images collected by different image acquisition devices may have differences due to factors such as shooting angles and characteristics of observation positions. Based on this, the present application uses the following formula to standardize the bridge image data:

[0050]

[0051] Among them, X i ' mg is the standardized bridge image data, X img is the original bridge image data, μ img is the mean of the bridge image data, σ img is the standard deviation of the bridge image data.

[0052] Therefore, the embodiment of the present application can effectively improve the accuracy and reliability of the bridge parameter data and bridge image data used for bridge status identification by standardizing the bridge parameter data and bridge image data respectively, and provide a reliable data basis and guarantee for the subsequent use of the bridge parameter data and bridge image data for bridge status identification.

[0053] Step 102, input the observed data into the improved convolutional neural network-long short-term memory network model to obtain the bridge state prediction value.

[0054] The bridge state prediction value is an attribute parameter that can characterize the bridge state. Preferably, the attribute parameter is an attribute parameter that can be actually collected and observed.

[0055] In an optional embodiment, the bridge state prediction value is the bridge strain value, that is, the bridge state prediction value is the predicted value of the bridge strain value at the detection point. It should be understood that multiple detection points can be preset in advance in the bridge, and the bridge state prediction value corresponding to each detection point is predicted based on the observation data corresponding to the detection point. In another specific embodiment, the detection point corresponding to the observation data and the detection point corresponding to the bridge state prediction value are two detection points with an associated relationship. For example, the detection point corresponding to the observation data is a detection point of a longitudinal beam of a bridge deck, and the detection point corresponding to the bridge state prediction value is a detection point corresponding to the symmetrical position of the longitudinal beam monitoring point of the bridge.

[0056] Furthermore, the improved convolutional neural network-long short-term memory network model includes a convolutional neural network unit, a long short-term memory network unit and a feature fusion output unit. The convolutional neural network unit is used to extract features from bridge image data to obtain bridge image features, the long short-term memory network unit is used to extract features from bridge parameter data to obtain bridge parameter features, and the feature fusion output unit is used to fuse bridge image features and bridge parameter features and output a bridge state prediction value.

[0057] It should also be noted that the improved convolutional neural network-long short-term memory network model is obtained by improving the convolutional neural network-long short-term memory network model using the attention mechanism.

[0058] Among them, a first attention subunit is set after the convolution layer and the pooling layer of the convolutional neural network unit, and a second attention subunit is set after at least one long short-term memory layer of the long short-term memory network unit.

[0059] In a specific embodiment, Figure 2 As shown, the convolutional neural network unit includes a convolution layer, a pooling layer and a first attention sub-unit. The convolution layer is used to perform a convolution operation on the bridge image data to extract the bridge image features in the bridge image data to obtain a bridge feature image; the pooling layer is used to reduce the size of the bridge feature image by maximum pooling or average pooling to retain the main features; the first attention sub-unit is used to enhance the feature expression of the bridge feature image after pooling.

[0060] Among them, the convolutional layer in the convolutional neural network unit can be abstracted as:

[0061] Z = Conv(X i ' mg ,W)+b

[0062] Among them, Z is the bridge feature image output by the convolutional layer, X i ' mg is the standardized bridge image data, W and B are the weight and bias of the convolution kernel in the convolution layer.

[0063] Optionally, the bridge feature image output by the convolutional neural network unit is a bridge feature image processed by an activation function. Specifically, the following function can be used for activation:

[0064] A=max(0,Z)

[0065] Among them, A is the activated bridge feature image, and Z is the bridge feature image without activation of the convolutional layer.

[0066] In an embodiment of the present application, the first attention sub-unit may be a channel attention mechanism module or a spatial attention mechanism module.

[0067] When the first attention sub-unit is the channel attention mechanism module, the channel attention mechanism module performs global average pooling on each channel of the bridge feature image to obtain the bridge feature channel description vector, and then uses the bridge feature channel description vector to determine the channel weight corresponding to each channel. Finally, the channel weight is used to recalibrate the bridge feature image to obtain the attention-enhanced bridge feature image.

[0068] Specifically, the following formula can be used to perform global average pooling on the bridge feature image:

[0069]

[0070] Among them, Zc is the global average pooling result of the bridge feature image of the cth channel, A c (i, j) is the bridge feature image corresponding to the c-th feature channel; H and W are the height and width of the bridge feature image corresponding to the c-th feature channel.

[0071] Furthermore, the embodiment of the present application uses two fully connected layers to generate the channel weight corresponding to each channel. Specifically, the following formula is used:

[0072] s c =σ(W2·δ(W1·z c ))

[0073] Among them, s c is the channel weight value corresponding to the c-th feature channel, Zc is the global average pooling result of the bridge feature image of the c-th feature channel, W1 and W2 are the weight matrices corresponding to the two fully connected layers, σ is the sigmoid activation function, and δ is the ReLU activation function.

[0074] When the first attention subunit is a channel attention mechanism module, the bridge feature image is recalibrated by multiplying the channel weight corresponding to the channel with the bridge feature image corresponding to the channel, which is specifically expressed as follows:

[0075]

[0076] in, is the bridge feature image after attention enhancement, A c is the bridge feature image corresponding to the cth feature channel, s c is the channel weight value corresponding to the cth feature channel.

[0077] When the first attention sub-unit is the spatial attention mechanism module, the bridge feature image is average pooled and maximum pooled in the channel dimension respectively to obtain the average projection image and the maximum projection image of the bridge feature image in the channel dimension. Then, the average projection image and the maximum projection image are used to generate the spatial attention feature image corresponding to the bridge feature image, and the spatial attention feature image is used to recalibrate the bridge feature image.

[0078] Specifically, the average projection image and the maximum projection image can be obtained respectively using the following formulas:

[0079] F avg =AvgPool(A)

[0080] F max =MaxPool(A)

[0081] Among them, A is the (activated) bridge feature image, Favg is the average projection image of the bridge feature image in the channel dimension, and Fmax is the maximum projection image of the bridge feature image in the channel dimension.

[0082] And, the spatial attention feature image corresponding to the bridge feature image is generated using the following formula:

[0083] M s =σ(Conv2D([F avg ,F max ]))

[0084] Among them, Ms is the spatial attention feature image corresponding to the bridge feature image, Conv2D is a two-dimensional convolution operation, σ is the sigmoid activation function, Favg is the average projection image of the bridge feature image in the channel dimension, and Fmax is the maximum projection image of the bridge feature image in the channel dimension.

[0085] When the first attention subunit is a channel attention mechanism module, the bridge feature image is recalibrated by multiplying the bridge feature image pixel by pixel using the spatial attention feature image corresponding to the bridge feature image, which is specifically expressed as follows:

[0086]

[0087] in, is the attention-enhanced bridge feature image, A is the (activated) bridge feature image, and Ms is the spatial attention feature image corresponding to the bridge feature image.

[0088] It should also be noted that in the embodiment of the present application, in order to enable the bridge feature image after attention enhancement to be fused with the bridge parameter feature, the bridge feature image after attention enhancement is further flattened to convert it into a one-dimensional feature F CNN .

[0089] On the other hand, Figure 2 As shown, the long short-term memory network unit includes at least one long short-term memory layer and a second attention subunit arranged after the long short-term memory layer. The long short-term memory layer is used to determine the hidden state sequence of the bridge parameter data at the time step, and the second attention subunit is used to determine the attention weight corresponding to each hidden state sequence based on the attention evaluation score of the hidden state sequence, and then determine the bridge parameter feature corresponding to the bridge parameter data based on the attention weight.

[0090] Among them, the (normalized) bridge parameter data is input into the long short-term memory layer to obtain the hidden state sequence {h1,h2,...,h T}. Then use the following formula to calculate the attention evaluation score corresponding to each hidden state sequence:

[0091] e t =v T tanh(Wh t +b)

[0092] Among them, e t is the attention evaluation score corresponding to the t-th hidden state sequence, T is the total number of time steps, and h t is the hidden state sequence, v, W and b are the learnable parameters in the attention mechanism.

[0093] Then the attention evaluation scores corresponding to each hidden state sequence are normalized:

[0094]

[0095] Among them, α t is the normalized attention evaluation score corresponding to each hidden state sequence, e t is the attention evaluation score corresponding to each hidden state sequence before normalization.

[0096] Then, the bridge parameter features corresponding to the bridge parameter data are determined based on the attention weights as follows:

[0097]

[0098] Among them, F LSTM To determine the bridge parameter features corresponding to the bridge parameter data based on the attention weight, α t is the normalized attention evaluation score corresponding to each hidden state sequence, h t is the hidden state sequence.

[0099] Furthermore, the feature fusion output unit includes a feature concatenation layer, a fully connected layer and an output layer. The feature concatenation layer is used to concatenate the bridge image features and the bridge parameter features to obtain the bridge comprehensive features, the fully connected layer is used to perform further nonlinear transformation on the bridge comprehensive features, and the output layer is used to output the bridge state prediction value using an activation function.

[0100] In the embodiment of the present application, when the convolutional neural network-long short-term memory network model is trained to be able to directly judge the state of the bridge (such as normal state or abnormal state), it can be determined that the convolutional neural network-long short-term memory network model is used for a binary classification task, and the output layer uses a sigmoid activation function to output a bridge state prediction value. When the convolutional neural network-long short-term memory network model is trained to output a certain attribute parameter (such as a strain value) for bridge state judgment, it can be determined that the convolutional neural network-long short-term memory network model is used for a multi-classification task, and the output layer uses an oftmax activation function to output a bridge state prediction value.

[0101] In the embodiment of the present application, since the convolutional neural network-long short-term memory network model can be applied to different classification tasks, it is also necessary to select different loss functions for training according to different classification tasks during the training process. Specifically, the following loss function is used for the binary classification task:

[0102]

[0103] Among them, L is the loss function value, m is the total number of samples, and y (i) is the true label of the i-th sample, is the predicted value of the i-th sample.

[0104] The following loss function is used for multi-classification tasks:

[0105]

[0106] Among them, L is the loss function value, m is the total number of samples, k is the total number of categories, is the true label of the i-th sample, is the predicted value of the i-th sample.

[0107] In a feasible embodiment, the present application uses the Adam algorithm to train the model. Specifically, in each iteration, the loss function L is calculated based on the input bridge observation data and the output bridge state prediction value, and then the gradient corresponding to the loss function is calculated for each trainable parameter in the model, and finally the Adam optimization algorithm is used to update the parameter according to the gradient corresponding to the loss function, and so on, until the model converges or reaches a preset number of training times.

[0108] Specifically, the gradient corresponding to the loss function is calculated using the following formula:

[0109]

[0110] Among them, g t is the gradient of the i-th parameter of the i-th improved convolutional neural network-long short-term memory network model at the t-th iteration, θ i is the i-th parameter of the i-th improved convolutional neural network-long short-term memory network model, L t is the loss function calculated for the tth iteration.

[0111] Based on the Adam algorithm, the first-order moment estimate and the second-order moment estimate of the parameter gradient are calculated respectively:

[0112] m t =β1m t-1 +(1-β1)g t

[0113]

[0114] Among them, m t and v t They are parameter gradients g t are the first-order moment estimate and the second-order moment estimate of , β1 and β2 are the exponential decay rates of the first-order moment estimate and the second-order moment estimate, respectively.

[0115] Then, the parameter gradient g t The first-order moment estimate and the second-order moment estimate of are bias-corrected:

[0116]

[0117] in, and are the bias correction of the first-order moment estimate and the bias correction of the second-order moment estimate, respectively, m t and v t They are parameter gradients g t The first and second moment estimates of .

[0118] Finally, the parameters are updated using the bias correction of the first-order moment estimate and the bias correction of the second-order moment estimate:

[0119]

[0120] Among them, θ t is the updated parameter, α is the learning rate, ε is the compensation constant, θ t-1 is the parameter updated in the previous iteration, and They are bias correction for first-order moment estimation and bias correction for second-order moment estimation, respectively.

[0121] It should be understood that the learning rate can be dynamically adjusted according to the training progress using a learning rate decay or cosine annealing strategy, and this application does not make any specific limitations.

[0122] Based on the above training process, the actual loss function of this application is as follows:

[0123]

[0124] Among them, L total is the loss function value, η is the loss function correction coefficient, θ i is the i-th parameter of the improved convolutional neural network-long short-term memory network model.

[0125] Therefore, the embodiment of the present application provides an improved convolutional neural network-long short-term memory network model. Compared with the traditional convolutional neural network-long short-term memory network model used for bridge state recognition, it adds bridge image data and its independent feature extraction mechanism. Compared with the existing method of converting time series data into images and then extracting features, it effectively enriches the data modality used for bridge state recognition and provides reliable data guarantee for improving the accuracy of bridge state recognition. Moreover, the improved convolutional neural network-long short-term memory network model of the present application adds attention mechanism sub-units to the convolutional neural network unit and the long short-term memory network unit respectively, which can effectively pay attention to the problem of inconsistent feature importance of the bridge state in the time dimension, that is, the feature of the abnormal peak period is more important than the feature of the stable operation period, thereby further realizing the improvement of the accuracy and reliability of bridge state recognition in the feature processing stage. Furthermore, when training the improved convolutional neural network-long short-term memory network model, the present application comprehensively considers the parameters of different units and implements training through a composite loss function, which can better compensate for the problems such as inconsistent convergence efficiency caused by the different data types processed by the convolutional neural network units and the long short-term memory network units. Moreover, the use of the Adam optimization algorithm for training can further improve the efficiency of model training.

[0126] Step 103, determining the state of the bridge based on the bridge state prediction value.

[0127] In a feasible embodiment, when the improved convolutional neural network-long short-term memory network model is a binary classification task and can directly output the abnormal bridge status label as the bridge status prediction value, the status of the bridge can be directly determined based on the bridge status prediction value output by the improved convolutional neural network-long short-term memory network model.

[0128] In another feasible embodiment, when the improved convolutional neural network-long short-term memory network model is a multi-classification task and a certain attribute parameter of the bridge (such as strain value) is output as a bridge state prediction value, the state of the bridge is determined based on the bridge state prediction value and the actual bridge state value.

[0129] Specifically, when the prediction residual between the bridge state prediction value and the actual bridge state value is greater than the state warning threshold, it is determined that the bridge state is abnormal.

[0130] Among them, the status warning threshold is related to the prediction residual.

[0131] It should be understood that when the improved convolutional neural network-long short-term memory network model is a multi-classification task, the improved convolutional neural network-long short-term memory network model can be used to characterize the functional relationship between the strain data when the bridge is in good condition. When the bridge is damaged, that is, the state is abnormal, the output of the improved convolutional neural network-long short-term memory network model is still the strain value when the bridge is in good condition, and the measured strain value is the strain value when the bridge structure is damaged. Therefore, the difference between the bridge state prediction value and the actual bridge state value must be large. That is, when the bridge is in good condition, the residual of the bridge state prediction value and the actual bridge state value changes within a certain range, and when the state of the bridge structure changes, the residual of the bridge state prediction value and the actual bridge state value will also increase accordingly. Therefore, it is only necessary to reasonably set the state warning threshold to determine the state of the bridge according to the bridge state prediction value and the actual bridge state value.

[0132] Specifically, the following formula is used to calculate the variance of bridge parameter data:

[0133]

[0134] in, is the variance of bridge parameter data, λ is the statistical weight of bridge parameter data, n m is the sampling length, is the standard deviation of the model prediction residuals.

[0135] Furthermore, the status warning threshold can be expressed as follows:

[0136]

[0137] Among them, μ and Q are correction coefficients.

[0138] Specifically, when the actual bridge status value exceeds P U At (t), it is determined that the bridge status is abnormal.

[0139] Therefore, the bridge state identification method proposed in the embodiment of the present application can use the improved convolutional neural network-long short-term memory network model to predict the acquired bridge observation data, obtain the bridge state prediction value, and then determine the state of the bridge based on the bridge state prediction value, thereby effectively improving the accuracy and reliability of the bridge state prediction.

[0140] It should be noted that although the operations of the method of the present invention are described in a particular order in the drawings, this does not require or imply that the operations must be performed in this particular order or that all illustrated operations must be performed to achieve desired results.

[0141] Figure 3 FIG. 1 is a schematic diagram showing the structure of a bridge state identification device provided by an embodiment of the present application. Figure 3 As shown, the bridge state identification device 10 includes:

[0142] An acquisition module 11 is used to acquire observation data of the bridge, wherein the observation data includes bridge parameter data and bridge image data;

[0143] A prediction module 12, used for inputting the observed data into an improved convolutional neural network-long short-term memory network model to obtain a predicted value of the bridge state;

[0144] The determination module 13 is used to determine the state of the bridge based on the bridge state prediction value.

[0145] In some embodiments, the improved convolutional neural network-long short-term memory network model includes:

[0146] A convolutional neural network unit is used to extract features from the bridge image data to obtain bridge image features;

[0147] A long short-term memory network unit is used to extract features from the bridge parameter data to obtain bridge parameter features;

[0148] A feature fusion output unit is used to fuse the bridge image feature and the bridge parameter feature and output the bridge state prediction value.

[0149] In some embodiments, the convolutional neural network-long short-term memory network model is improved by using an attention mechanism to obtain the improved convolutional neural network-long short-term memory network model;

[0150] Wherein, a first attention subunit is set after the convolution layer and the pooling layer of the convolutional neural network unit;

[0151] A second attention subunit is arranged after at least one long short-term memory layer of the long short-term memory network unit.

[0152] In some embodiments, the improved convolutional neural network-long short-term memory network model adopts a regularized cross-entropy loss function during training.

[0153] In some embodiments, the loss function uses the following formula:

[0154]

[0155] Among them, L total is the loss function value, L is the loss function before correction, η is the loss function correction coefficient, θ i is the i-th parameter of the improved convolutional neural network-long short-term memory network model.

[0156] In some embodiments, the determination module 13 is further configured to:

[0157] When a prediction residual between the predicted bridge state value and the actual bridge state value is greater than a state warning threshold, determining that the bridge state is abnormal;

[0158] Wherein, the state warning threshold is related to the prediction residual.

[0159] It should be understood that the modules or modules described in the bridge state identification device 10 are similar to those described in the reference Figure 1 The various steps in the described method correspond to each other. Therefore, the operations and features described above for the method are also applicable to the bridge state identification device 10 and the modules contained therein, and will not be repeated here. The bridge state identification device 10 can be pre-implemented in a browser or other security application of an electronic device, or can be loaded into a browser or its security application of an electronic device by downloading or the like. The corresponding modules in the bridge state identification device 10 can cooperate with the modules in the electronic device to implement the solution of the embodiment of the present application.

[0160] For the several modules or units mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0161] Reference below Figure 4 , Figure 4 A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing an embodiment of the present application is shown.

[0162] like Figure 4 As shown, the computer system includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 into the random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation instructions of the system are also stored. The CPU 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0163] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.

[0164] In particular, according to an embodiment of the present application, the above reference flow chart Figure 2 The described process can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above-mentioned functions defined in the system of the present application are executed.

[0165] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium such as a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0166] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operating instructions of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the aforementioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operating instruction, or can be implemented with a combination of dedicated hardware and computer instructions.

[0167] The units or modules involved in the embodiments described in the present application may be implemented by software or by hardware. The described units or modules may also be arranged in a processor, for example, they may be described as: a processor includes an acquisition module, a prediction module and a determination module. Among them, the names of these units or modules do not constitute a limitation on the units or modules themselves under certain circumstances. For example, the acquisition module may also be described as "obtaining observation data of a bridge, wherein the observation data includes bridge parameter data and bridge image data".

[0168] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the bridge state identification method described in the present application.

[0169] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A bridge state identification method, characterized in that: include: Acquire observation data of the bridge, wherein the observation data includes bridge parameter data and bridge image data; Inputting the observed data into an improved convolutional neural network-long short-term memory network model to obtain a predicted value of the bridge state; Based on the bridge state prediction value, the state of the bridge is determined.

2. The bridge state identification method according to claim 1, characterized in that: The improved convolutional neural network-long short-term memory network model includes: A convolutional neural network unit is used to extract features from the bridge image data to obtain bridge image features; A long short-term memory network unit is used to extract features from the bridge parameter data to obtain bridge parameter features; A feature fusion output unit is used to fuse the bridge image feature and the bridge parameter feature and output the bridge state prediction value.

3. The bridge state identification method according to claim 2, characterized in that: The convolutional neural network-long short-term memory network model is improved by using the attention mechanism to obtain the improved convolutional neural network-long short-term memory network model; Wherein, a first attention subunit is set after the convolution layer and the pooling layer of the convolutional neural network unit; A second attention subunit is arranged after at least one long short-term memory layer of the long short-term memory network unit.

4. The bridge state identification method according to claim 2, characterized in that: The improved convolutional neural network-long short-term memory network model adopts a regularized cross-entropy loss function during the training process.

5. The bridge state identification method according to claim 4, characterized in that: The loss function uses the following formula: Among them, L total is the loss function value, L is the loss function before correction, η is the loss function correction coefficient, θ i is the i-th parameter of the improved convolutional neural network-long short-term memory network model.

6. The bridge state identification method according to claim 1, characterized in that: The determining the state of the bridge based on the bridge state prediction value includes: When a prediction residual between the predicted bridge state value and the actual bridge state value is greater than a state warning threshold, determining that the bridge state is abnormal; Wherein, the state warning threshold is related to the prediction residual.

7. A bridge state recognition device, characterized in that: include: An acquisition module, used for acquiring observation data of the bridge, wherein the observation data includes bridge parameter data and bridge image data; A prediction module, used for inputting the observed data into an improved convolutional neural network-long short-term memory network model to obtain a predicted value of the bridge state; A determination module is used to determine the state of the bridge based on the bridge state prediction value.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the bridge state identification method as described in any one of claims 1-6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the bridge state identification method as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the bridge state identification method according to any one of claims 1 to 6 is implemented.

Images