Multistage bridge defect detection method
Through the multi-stage bridge defect detection method, combined with RGB images and lidar BEV projection data, the multi-task encoder and YOLOX-Tiny model are trained to achieve efficient and accurate detection of bridge defects, and solve the problems of low manual detection efficiency and poor comprehensiveness.
Patent Information
- Application Number
- CN202411614597.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-13
- Publication Date
- 2025-02-28
AI Technical Summary
Existing bridge defect detection mainly relies on manual detection, which has problems such as low efficiency and difficulty in covering subtleties and manual errors, affecting bridge production safety and management.
The multi-stage bridge defect detection method is adopted to obtain single-frame RGB images of the bridge and lidar BEV projection fake images, establish an image data set, perform data cleaning and enhancement processing, train a multi-task encoder to extract image features, and input the YOLOX-Tiny model for detection, and combine the UAV POS data to achieve fixed-point amplification.
It has achieved automatic inspections in full coverage, improved the accuracy and comprehensiveness of bridge defect detection, significantly reduced the missed inspection rate, and saved manpower and material resources.
Smart Images
Figure CN119540185A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of bridge defect detection, and in particular to a multi-stage bridge defect detection method. Background Art
[0002] Bridges are an important part of highways. With the substantial increase in traffic volume, the traffic density and vehicle load are getting bigger and bigger. The importance of river-crossing bridges and viaducts in traffic engineering is increasing day by day. Many bridges with a long service life have become aging, and some bridges have even suffered from different degrees of diseases. It can be clearly found that the concrete of the bridges is cracking, peeling, decaying and the steel bars are rusting. If these diseases are not dealt with in time, they will threaten the structural safety of the bridges and may even cause the collapse of the bridges. Therefore, it is very necessary to conduct regular inspection and maintenance of bridges.
[0003] At present, most of the bridge health monitoring in my country still relies on manual inspection and investigation. Although the traditional manual inspection method can find some obvious damage, it is inefficient and difficult to fully cover every detail of the bridge. In addition, manual inspection is also affected by the experience and skill level of the inspectors, and there are certain manual errors. These all affect the safe production and safety management level of the bridge. Without affecting transportation, efficient and rapid inspection and detection of bridge diseases has become a problem that needs to be solved urgently. Summary of the invention
[0004] In order to solve the above problems, the present invention provides a multi-level bridge defect detection method, which greatly saves manpower and material resources and improves the accuracy of bridge defect detection.
[0005] To achieve the above object, the technical solution adopted by the present invention is:
[0006] A multi-stage bridge defect detection method comprises the following steps:
[0007] S1, obtain a single-frame RGB image of the bridge and a laser radar BEV projection pseudo image to establish a bridge image dataset;
[0008] S2, performing data cleaning, noise reduction and enhancement processing on the bridge image dataset to obtain a data set of bridge disease images to be identified;
[0009] S3, train a multi-task encoder to extract image features from single-frame RGB images and LiDAR BEV projected pseudo images;
[0010] S4. Input the image features into the YOLOX-Tiny model to obtain the detection results of different defect classes.
[0011] Furthermore, in step S1, a single-frame RGB image and a laser radar BEV projection pseudo image of the bridge are obtained through a drone based on bilateral filtering weight processing, and each single-frame RGB image and laser radar BEV projection pseudo image carries drone POS data.
[0012] Furthermore, in step S2, after data cleaning, noise reduction and enhancement processing are performed on the bridge image dataset, a number of 256*256 single-frame RGB images and laser radar BEV projection pseudo images are generated to obtain a data set of bridge disease images to be identified.
[0013] Furthermore, the method also includes the steps of constructing a three-dimensional coordinate set of the weak points of the bridge, and realizing the fixed-point amplification of the image data set of the bridge disease to be identified based on the three-dimensional coordinate set of the weak points of the bridge.
[0014] Furthermore, when zooming in on a fixed point, the three-dimensional coordinate set of the bridge corresponding to each single-frame RGB image and the laser radar BEV projection pseudo-image is first obtained based on the drone POS data carried by the single-frame RGB image and the laser radar BEV projection pseudo-image, and then the corresponding point to be zoomed in is searched in the three-dimensional coordinate set of the bridge corresponding to each single-frame RGB image and the laser radar BEV projection pseudo-image based on the three-dimensional coordinate set of the weak point of the bridge, and then lossless zooming of the area circled by the point to be zoomed in is achieved.
[0015] Furthermore, in step S3, a single-frame RGB image is input into an image feature extraction backbone network, and feature extraction is performed through a ResNet-50 network pre-trained by ImageNet to obtain image features. A laser radar BEV projection pseudo image is input into a laser radar BEV feature extraction backbone network, and feature extraction is performed through a MobileNet-v2 network to obtain laser radar BEV features.
[0016] Furthermore, in step S4, firstly, the full fusion of image features and lidar BEV features is achieved based on the PANet (Path Aggregation Network) structure, and then the important information in the fused features is selectively enhanced through the channel attention mechanism, and finally the detection results of different defect classes are output through the BottleneckCSP structure.
[0017] The present invention has the following beneficial effects:
[0018] 1) Through full coverage automatic inspection, bridge defects can be discovered in time, improving the safety of bridge service and providing protection.
[0019] 2) The introduction of lidar BEV features combined with image features can improve the comprehensiveness of bridge defect detection. At the same time, the combined use of ResNet-50 network, MobileNet-v2 network, and YOLOX-Tiny model can significantly improve the accuracy of bridge defect detection.
[0020] 3) Targeted amplification of weak locations of bridges through fixed-point non-destructive amplification can further enhance the features and thus reduce the missed detection rate of bridge defect detection.
[0021] In summary, the present invention greatly saves manpower and material resources and improves the accuracy of bridge defect detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0023] Figure 1 This is a flow chart of a multi-level bridge defect detection method according to Example 1 of the present invention.
[0024] Figure 2 This is a flow chart of a multi-level bridge defect detection method according to Example 2 of the present invention. DETAILED DESCRIPTION
[0025] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0026] Example 1
[0027] like Figure 1 As shown, a multi-level bridge defect detection method comprises the following steps:
[0028] S1. Obtain a single-frame RGB image and a laser radar BEV projection pseudo-image of the bridge to establish a bridge image dataset. Specifically, first, a bridge inspection three-dimensional model is constructed based on the original design drawing of the bridge. The goal is to traverse the bridge inspection three-dimensional model. The coverage range of the single-frame RGB image of the drone and the coverage range of the laser radar BEV projection pseudo-image are used as constraints to obtain the inspection route of the drone. The inspection route contains the coordinate points of the drone image acquisition. When the drone reaches the coordinate points of the drone image acquisition, the corresponding single-frame RGB image and the laser radar BEV projection pseudo-image are automatically acquired. Here, the coverage range of a single-frame RGB image and the coverage range of a laser radar BEV projection pseudo-image are involved. The small range is used as a constraint, and the overlapping area range of the two images is obtained according to the large range. In the subsequent image processing, the overlapping area range is automatically removed; then, according to the inspection route, the single-frame RGB image and the laser radar BEV projection pseudo-image of the bridge are obtained by the drone. After the obtained single-frame RGB image and the laser radar BEV projection pseudo-image of the bridge are processed by bilateral filtering weights to achieve image clarity, it is necessary to mark the corresponding drone POS data on each single-frame RGB image and the laser radar BEV projection pseudo-image in a hyperlink mode. The POS data mainly includes GPS data and IMU data, that is, the external orientation elements in oblique photogrammetry: latitude, longitude, elevation, heading angle (Phi), pitch angle (Omega) and roll angle (Kappa). Therefore, the Exif of each image contains coordinate information. When an abnormality occurs, the coordinate information in the image can be extracted to facilitate the positioning of the abnormality, and it is also convenient for the subsequent selection of fixed-point magnification areas.
[0029] S2. Perform data cleaning, noise reduction and enhancement on the bridge image data set to obtain a data set of bridge disease images to be identified; specifically, perform data cleaning and noise reduction on the bridge image data in the bridge image data set using a matching model, perform data enhancement on the processed bridge image data, and perform random translation, rotation, flipping and mirroring operations on the data-enhanced bridge image data to generate a number of 256*256 single-frame RGB images and laser radar BEV projection pseudo images to obtain a data set of bridge disease images to be identified;
[0030] S3. Train a multi-task encoder to extract image features from a single-frame RGB image and a laser radar BEV projected pseudo image. The network of the multi-task encoder consists of an image feature extraction backbone network and a laser radar BEV feature extraction backbone network. The single-frame RGB image is input into the image feature extraction backbone network, and image features are extracted through a ResNet-50 network pre-trained with ImageNet. The laser radar BEV projected pseudo image is input into the laser radar BEV feature extraction backbone network, and features are extracted through a MobileNet-v2 network to obtain laser radar BEV features.
[0031] S4. Input the image features into the YOLOX-Tiny model to obtain the detection results of different defect classes. Specifically, firstly, the image features and the lidar BEV features are fully integrated based on the PANet (Path Aggregation Network) structure, and then the important information in the fused features is selectively enhanced through the channel attention mechanism. Finally, the detection results of different defect classes are output through the BottleneckCSP structure.
[0032] Example 1
[0033] like Figure 2 As shown, a multi-level bridge defect detection method comprises the following steps:
[0034] S1, obtain a single-frame RGB image of the bridge and a laser radar BEV projection pseudo image to establish a bridge image dataset;
[0035] S2, performing data cleaning, noise reduction and enhancement processing on the bridge image dataset to obtain a data set of bridge disease images to be identified;
[0036] S3. Construct a three-dimensional coordinate set of the weak points of the bridge, and based on the three-dimensional coordinate set of the weak points of the bridge, realize the fixed-point zooming of the image data set of bridge defects to be identified; first, according to the original design drawing of the bridge, search for the historical bridge defect data corresponding to the bridges similar to it. The historical bridge defect data includes honeycomb surface, dislocation, cracks, damaged exposed reinforcement, water seepage and repair, etc., to obtain the weak and defective positions of the bridge, and then find the three-dimensional coordinates corresponding to the weak and defective positions of each bridge in the three-dimensional bridge inspection model constructed based on the original design drawing of the bridge, and obtain the three-dimensional coordinate set of the weak points of the bridge. When zooming in at a fixed point, first obtain the three-dimensional coordinate set of the bridge corresponding to each single-frame RGB image and the laser radar BEV projection pseudo-image based on the drone POS data carried by the single-frame RGB image and the laser radar BEV projection pseudo-image, and then search for the corresponding point to be zoomed in in the three-dimensional coordinate set of the bridge corresponding to each single-frame RGB image and the laser radar BEV projection pseudo-image based on the three-dimensional coordinate set of the weak points of the bridge, and then realize lossless zooming of the area circled by the point to be zoomed in;
[0037] S4, train a multi-task encoder to extract image features from single-frame RGB images and LiDAR BEV projected pseudo images;
[0038] S5. Input the image features into the YOLOX-Tiny model to obtain the detection results of different defect classes.
[0039] The rest of the design is the same as in Example 1.
[0040] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0041] Comparative Example 1
[0042] A traditional YOLOX-Tiny model is trained to obtain a single-frame RGB image of the bridge. After data cleaning, noise reduction and enhancement, the image is input into the traditional YOLOX-Tiny model to obtain the detection results of different defect types.
[0043] A test set was constructed to test the accuracy of Example 1 and Comparative Example 1. It was found that compared with Comparative Example 1, the average accuracy of APS, APM, and APL targets of different sizes in Example 1 were improved by 3.6%, 8.1%, and 9.8%, respectively.
[0044] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.
Claims
1. A multi-level bridge defect detection method, characterized in that: The steps include: S1, obtain a single-frame RGB image of the bridge and a laser radar BEV projection pseudo image to establish a bridge image dataset; S2, performing data cleaning, noise reduction and enhancement processing on the bridge image dataset to obtain a data set of bridge disease images to be identified; S3, train a multi-task encoder to extract image features from single-frame RGB images and LiDAR BEV projected pseudo images; S4. Input the image features into the YOLOX-Tiny model to obtain the detection results of different defect classes.
2. A multi-level bridge defect detection method as claimed in claim 1, characterized in that: In step S1, a single-frame RGB image and a laser radar BEV projection pseudo image of the bridge are obtained through a drone based on bilateral filtering weight processing, and each single-frame RGB image and laser radar BEV projection pseudo image carries drone POS data.
3. A multi-level bridge defect detection method as claimed in claim 1, characterized in that: In step S2, after data cleaning, noise reduction and enhancement processing are performed on the bridge image dataset, a number of 256*256 single-frame RGB images and laser radar BEV projection pseudo images are generated to obtain a data set of bridge disease images to be identified.
4. A multi-level bridge defect detection method as claimed in claim 1, characterized in that: The method also includes the steps of constructing a three-dimensional coordinate set of the weak points of the bridge, and realizing the fixed-point amplification of the image data set of the bridge disease to be identified based on the three-dimensional coordinate set of the weak points of the bridge.
5. A multi-stage bridge defect detection method as claimed in claim 4, characterized in that: When zooming in on a fixed point, firstly, the three-dimensional coordinate set of the bridge corresponding to each single-frame RGB image and the laser radar BEV projection pseudo-image is obtained based on the drone POS data carried by the single-frame RGB image and the laser radar BEV projection pseudo-image. Then, based on the three-dimensional coordinate set of the weak point of the bridge, the corresponding point to be zoomed in is searched in the three-dimensional coordinate set of the bridge corresponding to each single-frame RGB image and the laser radar BEV projection pseudo-image. Then, lossless zooming of the area circled by the point to be zoomed in is achieved.
6. A multi-stage bridge defect detection method as claimed in claim 1, characterized in that: In step S3, a single-frame RGB image is input into the image feature extraction backbone network, and a ResNet-50 network pre-trained by ImageNet is used to extract the image features. The laser radar BEV projection pseudo image is input into the laser radar BEV feature extraction backbone network, and the laser radar BEV features are obtained by extracting the features through the MobileNet-v2 network.
7. A multi-stage bridge defect detection method as claimed in claim 1, characterized in that: In step S4, firstly, the full fusion of image features and lidar BEV features is achieved based on the PANet structure, and then the important information in the fused features is selectively enhanced through the channel attention mechanism, and finally the detection results of different defect classes are output through the BottleneckCSP structure.