A road and bridge detection method, system and storage medium

Abstract

The present application discloses a method, system and storage medium for road and bridge detection, relating to the technical field of construction engineering. By using finite element simulation and non-destructive testing technologies to obtain stress distribution images of roads and bridges, and combining with a multi-modal convolutional neural network (CNN-LSTM) fusion model for feature extraction and stress analysis, real-time monitoring and prediction of the stress distribution of roads and bridges are realized. The present application predicts roads and bridges by fusing different types of images, which helps to quickly detect potential problems, reduce the workload of maintenance personnel, and enable managers to better understand the health status of road and bridge structures.

Description

Technical Field

[0001] The present invention relates to the technical field of construction engineering, and particularly relates to a method, a system and a storage medium for road and bridge detection. Background Art

[0002] Road and bridges are important components of transportation infrastructure. They are usually used to cross terrain obstacles such as rivers, railways, roads, and canyons to provide passageways for vehicles and pedestrians. The stress of a road and bridge refers to the internal stress distribution in the bridge structure due to loads and other external forces. When designing and analyzing road and bridges, engineers need to consider bending stress, shear stress, compressive stress, tensile stress, shear deformation, and torque;

[0003] The prior art document with the publication number CN117521472A provides a method, a system and a storage medium for road and bridge detection, which combines finite element simulation and cellular automaton simulation to provide highly accurate stress analysis, enabling managers to better understand the health status of road and bridge structures. At the same time, through predictive maintenance, managers can plan maintenance work more specifically, reduce maintenance costs, and extend the lifespan of road and bridges.

[0004] However, this method requires the combination of finite element simulation and cellular automaton simulation, with a large amount of data processing. Cellular automata rely on simplified local rules rather than strict physical principles, and may not be able to accurately simulate the stress distribution in complex continuum mechanics. Stress calculation usually involves complex linear and non-linear mechanical behaviors, and it is difficult for cellular automata to accurately reproduce these behaviors. Especially in the case of high-precision requirements to accurately simulate the stress concentration area, the spatial resolution of CA must be high enough, which will lead to a sharp increase in the amount of calculation. In order to simulate the real stress distribution, complex mechanical models need to be transformed into appropriate cellular update rules, which requires a large amount of experimental verification and experience accumulation, and it is difficult to meet the requirements of fast and real-time road and bridge detection.

[0005] In view of this, we propose a method, a system and a storage medium for road and bridge detection. Summary of the Invention

[0006] The object of the present invention is to provide a method, a system and a storage medium for road and bridge detection, and the technical solutions include the following aspects.

[0007] According to one aspect of the embodiments of the present application, a method for road and bridge detection is provided, and the method includes:

[0008] S1. Obtain stress distribution images of road and bridges through finite element simulation or obtain different types of road and bridge detection images by applying a variety of non-destructive testing techniques;

[0009] S2. Denoise, enhance, correct, etc. various simulation or detection images to improve the image quality and provide clear image data for subsequent feature extraction and stress analysis.

[0010] S3. Construct a convolutional long short-term memory network (CNN-LSTM) fusion model. The model fuses multi-modal features and outputs the stress distribution image of the bridge in the future time period.

[0011] S4. Embed the trained model into the bridge structural health monitoring system to realize the prediction of bridge stress and stress distribution.

[0012] According to one aspect of the embodiments of the present application, another method for detecting road bridges is provided. The method includes:

[0013] S1'. Apply a variety of non-destructive testing technologies to obtain different types of road bridge detection images, and perform preprocessing operations such as noise removal, image enhancement, and geometric correction on various detection images. These operations can improve the image quality and provide high-quality image data for subsequent analysis and processing.

[0014] S2'. Construct a convolutional long short-term memory network (CNN-LSTM) fusion model. The model receives different types of road bridge detection images and outputs future predictions of the corresponding type of detection images.

[0015] S3'. For different types of bridge detection images, adopt different calculation methods to accurately solve the stress values and stress distributions for different types of images.

[0016] According to one aspect of the embodiments of the present application, a road bridge detection system is provided. The system includes:

[0017] The data acquisition module is responsible for collecting road bridge detection images generated by a variety of non-destructive testing technologies (such as ultrasonic testing, magneto-elastic imaging, ground penetrating radar, etc.). These images contain the stress distribution information of the bridge at different times and different positions and are the basis for subsequent analysis.

[0018] The preprocessing module performs preprocessing operations such as noise removal, image enhancement, and geometric correction on the collected images to improve the image quality and provide high-quality image data for subsequent analysis and processing.

[0019] The model training and prediction module is the core part of the system, which includes the process of building and training a CNN-LSTM model. This module first trains the model using historical image data to learn the changing patterns and potential periodicity of the bridge stress distribution. Then, the model can receive new image data and output predicted images of the bridge stress distribution for a period of time in the future. These predicted images can reflect the stress conditions at different positions of the bridge, providing strong data support for bridge maintenance and safety assessment.

[0020] The result display module is responsible for presenting the results predicted by the model to the user in an intuitive way. This can include presenting the predicted stress distribution images in various forms such as color contour maps, color cloud maps, and three-dimensional rendering maps, which is convenient for users to understand and analyze. At the same time, the system can also provide data analysis tools to help users deeply explore the information in the predicted results and provide support for decision-making.

[0021] According to an aspect of the embodiments of the present application, there is also provided a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned road and bridge detection method are implemented.

[0022] The technical solutions provided by the embodiments of the present application can bring the following beneficial effects:

[0023] The road and bridge detection method, system, and storage medium provided by the present invention utilize a multi-modal convolutional neural network and a convolutional long short-term memory network fusion model, combined with non-destructive testing technology, to achieve real-time monitoring and prediction of the stress distribution of road and bridges, improve the detection accuracy and efficiency, reduce the professional skill requirements for operators, and at the same time reduce the calculation time and application costs, having broad application prospects and practical value. Brief Description of the Drawings

[0024] Figure 1 It is a flowchart of a road and bridge detection method provided by an embodiment of the present application;

[0025] Figure 2 It is a flowchart of another road and bridge detection method provided by an embodiment of the present application;

[0026] Figure 3 It is a structural diagram of a road and bridge detection system provided by an embodiment of the present application;

[0027] Figure 4 It is a structural diagram of a convolutional long short-term memory network provided by an embodiment of the present application;

[0028] Figure 5 It is another structural diagram of a convolutional long short-term memory network provided by an embodiment of the present application; Detailed Embodiments

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following will further describe the embodiments of this application in detail with reference to the accompanying drawings.

[0030] Please refer to Figure 1 , which shows a flowchart of a road and bridge detection method provided by an embodiment of this application;

[0031] S1. Obtain stress distribution images of road and bridges simulated by finite element or detection images of different types of road and bridges using a variety of non-destructive testing techniques;

[0032] S2. Process various simulated or detected images through denoising, enhancement, correction, etc. to improve the image quality and provide clear image data for subsequent feature extraction and stress analysis.

[0033] S3. Construct a convolutional long short-term memory network (CNN-LSTM) fusion model that fuses multi-modal features and outputs the stress distribution image of the bridge within a future time period;

[0034] S4. Embed the trained model into the bridge structural health monitoring system to achieve the prediction of bridge stress and stress distribution.

[0035] In an embodiment of this application, a stress distribution map of a road and bridge is generated by finite element simulation. Specifically:

[0036] According to the design drawings of the actual bridge, create an accurate three-dimensional geometric model or two-dimensional plane model, which can import CAD drawings or directly construct in finite element software;

[0037] Define the fixed constraints, load application points, moving boundaries, etc. of the bridge. For example, rigid constraints (zero displacement) are usually set at the supports, etc.;

[0038] Assign appropriate material properties to different parts of the model, such as elastic modulus, Poisson's ratio, density, etc.;

[0039] Use tetrahedral, hexahedral, or triangular elements to reasonably mesh the model to ensure that there are sufficiently fine meshes in the stress concentration areas to accurately capture stress changes;

[0040] Define static or dynamic load conditions, such as vehicle load, wind load, temperature load, etc., and arrange them reasonably according to the actual stress situation;

[0041] Set the solver parameters, including linear and nonlinear analysis options, convergence criteria, etc., and then run the solver to calculate the responses such as displacement, stress, and strain of the model;

[0042] Generate color contour maps, color cloud maps, and three-dimensional rendering maps of the stress distribution of the road and bridge respectively;

[0043] Among them, based on the calculated stress distribution data, the software automatically draws a color contour map of the equivalent stress distribution, where different colors represent different stress levels, and the interval between contour lines reflects the stress change gradient. The stress values are mapped onto a color matrix to form a continuous color distribution. The darker the color, the higher the stress value, thus visually demonstrating the continuous change of stress across the entire structure. In the three-dimensional view of the stereoscopic rendering, the stress is mapped onto the model surface, and high-stress areas are displayed in bright or dull colors (depending on the color mapping scheme), and the visual effect is enhanced through light source and shadow effects, which helps to observe the stress concentration parts inside and on the surface of the bridge structure.

[0044] Among them, the convolutional long short-term memory network (CNN-LSTM) model constructed in step S3 generates stress distribution images of road bridges. The multi-modal convolutional neural network (MCNN) is used to extract different types of image features. MCNN consists of multiple parallel convolutional neural networks (CNNs), and each CNN is responsible for processing one type of image data. The features are extracted layer by layer through a series of convolutional layers, pooling layers, and normalization layers, which can be expressed as: f i = CNN(I), where I represents the input of the simulation image, and f i is the feature vector of the simulation image extracted by the convolutional neural network.

[0045] To integrate features of different modalities, a self-attention mechanism is introduced to calculate the relationship weights between features of each modality and fuse these features. The attention weight matrix between modalities is calculated to emphasize the feature parts that are more important for the final decision. For each feature (f i ), the similarity score (s ij ) with all other features is calculated. The formula can be expressed as: s ij = softmax(f i T W Q (W K f j T ))), where (W Q ) and (W K ) are learnable weight matrices for query and key conversion. The calculated weights (s ij ) are used to weight the features to obtain the fused feature, F 融合 = ∑ j s ij f j .

[0046] After serializing the fused features, they are input into the LSTM network in timestamp order. The LSTM will capture the changing patterns of the time series, and the output of the LSTM will be sent to a decoder network. The decoder uses a deconvolution network (such as transposed convolution) to reconstruct the image. The decoder maps the abstract features captured by the LSTM back to the original spatial dimensions layer by layer to generate an image prediction of the bridge stress distribution in the future time period. The decoder contains several deconvolution layers that gradually increase the spatial resolution of the feature map while maintaining the consistency of time and spatial information.

[0047] During the training process of the entire system, the actual stress distribution measurement data is used as the supervision signal, and the model parameters are optimized by minimizing the difference (mean square error) between the predicted image and the real image. For each pixel point (i, j), the mean square loss function can be defined as L MSE (y true ,y pred ) = 1 / (m × n) ∑ i＝1 m ∑ j＝1 n (y true [i, j] - y pred [i, j]) 2 , where m and n are the width and height of the image respectively, y true [i, j] is the pixel intensity of the real stress distribution image at the coordinate (i, j), and y pred [i, j] is the pixel intensity of the model-predicted stress distribution image at the same coordinate (i, j). With such a design, the CNN-LSTM model can not only process multi-modal data but also effectively predict the future stress distribution of the bridge, providing strong data support for bridge maintenance and safety assessment.

[0048] Please refer to Figure 4, in an embodiment of the present application, color contour maps, color cloud maps, and three-dimensional rendering maps of the stress distribution of road bridges are generated through finite element simulation, and three parallel CNN branches are used to process different types of images respectively. For example, for the color contour map branch, local stress pattern features are extracted through several convolutional layers, and then the amount of calculation is reduced and the generalization ability is enhanced through the pooling layer and the normalization layer to obtain the corresponding feature vector fi1. Similarly, the other two modalities also extract features through their respective CNN branches to obtain fi2 and fi3. These three feature vectors are combined, and the self-attention mechanism is used to dynamically adjust the importance of features of different modalities. By calculating the attention scores between each feature vector, we can obtain a fused feature vector F_fused, which comprehensively reflects the key information from different visual sources. The fused feature vector is input into the LSTM network in chronological order. The LSTM can capture the changing trend of the bridge stress distribution and potential periodic patterns evolving over time. The output sequence of the LSTM is fed into the decoder network, and the decoder generates a set of predicted bridge stress distribution images, which accurately depict the stress conditions of the bridge at different positions in the future for a period of time.

[0049] In another embodiment of the present application, detection images are regularly generated using ultrasonic detection technology, magneto-elastic imaging (MEI), ground penetrating radar detection technology, etc., and the constructed convolutional long short-term memory network (CNN-LSTM) is used to predict the stress distribution of the detection images. Among them, ultrasonic detection technology is used to obtain information such as the sound velocity and attenuation inside the bridge components and convert it into a B-Scan image. The magneto-elastic imaging (MEI) technology is used to obtain the magnetic field changes on the surface or inside the structure and analyze them into stress distribution images. The ground penetrating radar (GPR) technology is used to detect the state of the bridge foundation and underground structures and generate radar images.

[0050] In the embodiment of the present application, the real stress distribution images for training are comprehensively analyzed from different images by an expert group.

[0051] Please refer to Figure 2 , which shows another flowchart of the road bridge detection method provided by an embodiment of the present application;

[0052] S1’: Apply a variety of non-destructive testing technologies to obtain different types of road bridge detection images, and perform preprocessing operations such as noise removal, image enhancement, and geometric correction on various detection images. These operations can improve the quality of the images and provide high-quality image data for subsequent analysis and processing;

[0053] S2’: Construct a convolutional long short-term memory network (CNN-LSTM) fusion model. The model receives different types of road bridge detection images and outputs future predictions of the corresponding type of detection images;

[0054] S3’. For different types of bridge inspection images, different calculation methods are adopted to accurately solve the stress values and stress distributions for different types of images.

[0055] Please refer to Figure 5 , in the convolutional long short-term memory network (CNN-LSTM) model constructed in step S2’, different types of inspection images of road bridges are generated. Several CNN branches are used to extract the features of the same type of inspection images at different time points. After serialization processing, they are input into the LSTM network in the order of time stamps. The LSTM will capture the changing rules of the time series, and the output of the LSTM will be sent to a decoder network. The decoder contains several deconvolution layers, which gradually improve the spatial resolution of the feature map while maintaining the consistency of time and space information.

[0056] In the embodiments of the present application, models are respectively built for the inspection images generated by regularly using ultrasonic detection technology, magneto-elastic imaging (MEI), ground penetrating radar detection technology, etc., and predictions of corresponding types of images are made to facilitate the analysis by the expert group.

[0057] Please refer to Figure 3 , which shows the structural diagram of a road bridge inspection system provided by an embodiment of the present application;

[0058] The system mainly includes a data acquisition module, a preprocessing module, a model training and prediction module, and a result display module.

[0059] The data acquisition module obtains the stress distribution images of road bridges simulated by finite element or the road bridge inspection images generated by collecting a variety of non-destructive testing technologies (such as ultrasonic testing, magneto-elastic imaging, ground penetrating radar, etc.). These images contain the stress distribution information of the bridge at different times and positions and are the basis for subsequent analysis.

[0060] The preprocessing module performs preprocessing operations on the collected images, such as noise removal, image enhancement, geometric correction, etc., to improve the image quality and provide high-quality image data for subsequent analysis and processing.

[0061] The model training and prediction module is the core part of the system and includes the process of constructing and training the CNN-LSTM model. This module first trains the model using historical image data to learn the changing rules and potential periodicity of the bridge stress distribution. Then, the model can receive new image data and output the predicted images of the bridge stress distribution in the next period of time. These predicted images can reflect the stress conditions of the bridge at different positions and provide strong data support for bridge maintenance and safety assessment.

[0062] The result display module is responsible for presenting the results predicted by the model to the user in an intuitive manner. This can include presenting the predicted stress distribution images in various forms such as color contour maps, color cloud maps, and stereoscopic renderings, facilitating users' understanding and analysis. At the same time, the system can also provide data analysis tools to help users deeply explore the information in the predicted results and provide support for decision-making.

[0063] Through the above technical solutions, the road and bridge detection method, system, and storage medium provided by the present invention utilize a multi-modal convolutional neural network and a convolutional long short-term memory network fusion model, combined with non-destructive testing techniques, to achieve real-time monitoring and prediction of the stress distribution of road and bridges, improve the detection accuracy and efficiency, reduce the professional skill requirements for operators, and at the same time reduce the calculation time and application costs, having broad application prospects and practical value.

Claims

1. A road bridge detection method, characterized in that: The method comprises: S1. Stress distribution images of roads and bridges simulated by finite element or different types of road and bridge inspection images obtained by applying a variety of non-destructive testing technologies; S2. De-noise, enhance and correct various simulated stress distribution images or detection images to provide clear image data for subsequent feature extraction and stress analysis; S3, construct a convolutional long short-term memory network fusion model, the model integrates multimodal features and outputs the stress distribution image of the bridge in the future time period; S4. Embed the trained model into the bridge structure health monitoring system to predict bridge stress and stress distribution.

2. A road bridge detection method according to claim 1, characterized in that: Image of road bridge stress distribution simulated by finite element method: Create accurate 3D geometric models or 2D plane models based on actual bridge design drawings; Define the fixed constraints, load application points, and moving boundaries of the bridge; Assign appropriate material properties to different parts of the model; Use tetrahedron, hexahedron or triangle elements to properly mesh the model; Define static or dynamic load cases and set solver parameters; Generate color contour maps, color cloud maps, and three-dimensional renderings of stress distribution of roads and bridges respectively.

3. A road bridge detection method according to claim 1, characterized in that: Obtain different types of road and bridge inspection images through non-destructive testing technology: Use ultrasonic testing technology to obtain information on the speed and attenuation of sound inside bridge components and convert it into B-Scan images; Use magnetoelastic imaging technology to obtain magnetic field changes on the surface or inside the structure and analyze them into stress distribution images; Geological radar technology is used to detect the status of bridge foundations and underground structures and generate radar images.

4. A road bridge detection method according to claim 1, characterized in that: Convolutional long short-term memory network fusion model: Use multimodal convolutional neural networks to extract features of different types of images; The multimodal convolutional neural network consists of multiple parallel convolutional neural networks, each of which is responsible for processing one type of image data; Extract features layer by layer, expressed as: f i =CNN(I), where I represents the simulated image input, f i It is the simulated image feature vector extracted by convolutional neural network.

5. A road bridge detection method according to claim 4, characterized in that: Convolutional long short-term memory network fusion model: A self-attention mechanism is introduced to calculate the relationship weights between each modality feature and fuse these features; For each feature (f i ), and calculate its similarity score with all other features (s ij ), the formula can be expressed as: s ij =softmax(f i T W Q (W K f j T )), where (W Q ) and (W K ) is a learnable weight matrix used for query (Query) and key (Key) conversion, using the calculated weight (s ij ) weights the features to obtain the fused features, F 融合 =∑ j s ij f j .

6. A road bridge detection method according to claim 5, characterized in that: Convolutional long short-term memory network fusion model: The fused features are serialized and input into the long short-term memory network in the order of timestamps; The output of the LSTM network is fed into a decoder network, which uses a deconvolutional network to reconstruct the image. During the training process, the actual stress distribution measurement data is used as a supervisory signal to optimize the model parameters by minimizing the difference between the predicted image and the real image; For each pixel (i, j), the mean square loss function can be defined as L MSE (y true ,y pred ) = 1 / (m×n)∑ i=1 m ∑ j=1 n (y true [i,j]−y pred [i,j]) 2 , m and n are the width and height of the image respectively, y true [i,j] is the pixel intensity of the true stress distribution image at coordinate (i,j), y pred [i, j] is the pixel intensity of the model-predicted stress distribution image at the same coordinate (i, j).

7. A road bridge detection method, characterized in that: The method comprises: S1', apply various non-destructive testing technologies to obtain different types of road and bridge inspection images, and perform noise removal, image enhancement, and geometric correction on various inspection images; S2', construct a convolutional long short-term memory network fusion model, the model receives different types of road and bridge detection images, and outputs future predictions of the corresponding types of detection images; S3', for different types of bridge inspection images, different calculation methods are used to accurately solve stress values ​​and stress distributions for different types of images.

8. A road bridge detection system, characterized in that: The system comprises: The data acquisition module is responsible for collecting road and bridge inspection images generated by various nondestructive testing technologies; The preprocessing module removes noise, enhances the image, and performs geometric correction on the collected images to improve the image quality and provide high-quality image data for subsequent analysis and processing; Model training and prediction module, builds and trains convolutional long short-term memory network model, uses historical image data to train the model, and learns the changing law and potential periodicity of bridge stress distribution; the model receives new image data and outputs the predicted image of bridge stress distribution in the future; The result display module is responsible for displaying the model prediction results to the user in an intuitive way.

9. A road bridge detection system according to claim 8, characterized in that: The result display module presents the predicted image in the form of a color contour map, a color cloud map, and a three-dimensional rendering map to facilitate user understanding and analysis.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the road and bridge detection method according to any one of claims 1 to 7 are implemented.

Images