A method and system for detecting pavement structure damage based on radar full-profile image features

Abstract

This invention discloses a method and system for detecting pavement structure damage based on radar full-profile map features, comprising: Step 1, acquiring high-density three-dimensional electromagnetic wave data and preprocessing it; Step 2, converting the preprocessed data into a 3D point cloud profile and segmenting it into B-Scan and C-Scan maps, and labeling the structural damage types; Step 3, constructing a dual-input YOLO target detection model including an input terminal, a feature extraction module, a feature fusion module, a multi-scale discrimination module, and a regressor. The input terminal receives the B-Scan and C-Scan maps and transmits them to the feature extraction module to obtain the structural damage feature information corresponding to the B-Scan and C-Scan maps. The feature fusion module stitches the data together to generate fused feature information, which is then transmitted to the multi-scale discrimination module for multi-scale feature fusion and enhancement processing, and finally outputs the prediction result box through the regressor; Step 4, calculating the actual location and size information of the structural damage and a pavement structure damage detail table through a coordinate transformation algorithm.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional ground-penetrating radar pavement structure defect detection technology, specifically involving a pavement structure damage detection method and system based on radar full profile map features. Background Technology

[0002] With the continuous increase in highway traffic volume, road surfaces experience varying degrees of structural damage during long-term service, which not only reduces pavement performance but also significantly shortens its service life, leading to increasingly heavy maintenance tasks. Most apparent defects are actually the result of the diffraction and development of internal structural defects, such as structural cracks, poor interlayer bonding, loosening, and voids—internal damage that is difficult to detect directly and poses a potential threat to road structural safety. Against this backdrop, strengthening the detection and identification of road defects, especially early-stage internal structural damage, has become a crucial link in ensuring pavement quality and driving safety.

[0003] 3D ground-penetrating radar (GPR), as a non-destructive testing technology, boasts advantages such as high precision, high efficiency, continuous operation, and wide coverage. In recent years, it has received widespread attention in the field of road defect detection and is an important technical means to meet the ever-increasing demand for road structure inspection. However, current GPR data interpretation heavily relies on the experience of professionals. Because the analysis of 3D radar detection results often uses single-section grayscale maps, map interpretation is difficult and visualization is poor, leading to low efficiency in manual interpretation, strong subjectivity in judgment, and unreliable analysis results. Summary of the Invention

[0004] The purpose of this invention is to address the difficulties in interpreting 3D ground-penetrating radar (GPR) image features, the low efficiency of manual interpretation, and the strong subjectivity of judgment, by proposing a pavement structure damage detection method based on GPR full-profile image features. This invention also provides a pavement structure damage detection system based on GPR full-profile image features.

[0005] The technical solution adopted to achieve the purpose of this invention is: A method for detecting pavement structure damage based on radar full-profile image features includes the following steps: Step 1: Collect raw signal data of road surface structure damage, obtain high-density three-dimensional electromagnetic wave data and preprocess it; Step 2: Convert the preprocessed high-density three-dimensional electromagnetic wave data in Step 1 into a 3D point cloud profile volume. By segmenting the 3D point cloud profile volume, obtain radar full profile map slices containing B-Scan and C-Scan maps. Label the structural defect types in the B-Scan and C-Scan maps to establish a radar map dataset of pavement structural defects. Step 3: Construct a dual-input YOLO target detection model including an input terminal, a feature extraction module, a feature fusion module, a multi-scale discriminant module, and a regressor. The input terminal includes two independent input ports, used to receive the B-Scan and C-Scan maps labeled with structural damage types in Step 2, respectively. The feature extraction module contains two independent and structurally corresponding backbone network branches, which extract features from the B-Scan and C-Scan maps to obtain corresponding structural damage feature information. The feature fusion module concatenates the feature information output by the two backbone network branches to generate fused feature information. The multi-scale discriminant module performs multi-scale feature fusion and enhancement processing on the fused feature information. The regressor is connected to the predicted result box of structural damage within the output map region based on the multi-scale feature fusion result. Step 4: Based on the prediction result box obtained in Step 3, the actual location and size information of the structural damage are obtained through coordinate transformation algorithm, and a detailed table of pavement structural damage is automatically generated.

[0006] In the above technical solution, in step 1, a three-dimensional ground-penetrating radar device is used to collect original signal data of road surface structure damage. The three-dimensional ground-penetrating radar device adopts a 21-channel three-dimensional array ground-penetrating radar system with SGY data acquisition format. The high-density three-dimensional electromagnetic wave data is obtained by the radar antenna emitting electromagnetic waves along the direction of lane travel and consists of multiple parallel B-scans. The preprocessing process of the high-density three-dimensional electromagnetic wave data is as follows: the high-density three-dimensional electromagnetic wave data is processed sequentially by Reflexw software to remove direct waves, DC drift, signal gain, horizontal interference signal removal and Butterworth bandpass filtering.

[0007] In the above technical solution, in step 2, the preprocessed high-density three-dimensional electromagnetic wave data is converted into point cloud data by using the Open3D module in the Python language to generate a 3D point cloud model; and the different structural disease types in the B-Scan and C-Scan maps are labeled by the professional annotation software Labelimg.

[0008] In the above technical solution, in step 3, the input end performs data augmentation on the B-Scan and C-Scan maps labeled with structural disease types using the Mosaic data augmentation algorithm. The data augmentation includes random stitching, cropping, and adaptive image scaling. The feature extraction module is an improvement on the original backbone network ResNet50, which is a dual_ResNet50 structure with two independent and structurally corresponding backbone networks with residual connections. The feature extraction module consists of five layers, namely Layer 0 to Layer 4. Layer 0 is composed of the Focus module, and Layers 1 to 4 contain the Focus module and the CSP1_X module. The multi-scale discrimination module adopts a structure combining FPN and PAN, and sets a CSP2_X module between the FPN structure and the PAN structure. The FPN structure is used to pass high-level semantic features to the low-level feature layer through upsampling and fuse them with the low-level features. The PAN structure performs bidirectional fusion of low-level localization information and high-level semantic information through downsampling and lateral connection to obtain fused feature information suitable for target detection at different scales. The regressor includes convolutional layers, pooling layers, and fully connected layers. The convolutional and pooling layers are used to extract features from the feature map, and the fully connected layers integrate the feature map into a tensor to generate a predicted bounding box of structural damage within the map region.

[0009] In the above technical solution, the calculation formula for the residual connection of Dual_ResNet50 is as follows: ; In the formula, Input for the l-th residual unit, For activation function, For the residual function, This is the weight matrix; The multi-scale discrimination module is connected to the feature fusion module and contains a three-layer structure. It is used to discriminate the structural damage type of the fused features at large, medium and small scales.

[0010] In the above technical solution, in step 4, the coordinate transformation algorithm maps the coordinates and size of the predicted box in the recognition result from the image pixel coordinate system to the actual physical coordinate system represented by the mileage station and depth range, and obtains the mileage station of the structural damage center point, the center point depth, the actual width of the damage along the mileage direction and the actual height of the damage along the depth direction, respectively. The coordinate transformation algorithm calculation formula is as follows: ; ; ; ; In the formula, The center point mileage marker, Depth at the center point The actual width of the disease. The actual height of the disease. and This image corresponds to the actual starting and ending station numbers. and These are the x and y coordinates of the center point of the predicted bounding box in the image coordinate system. and To predict the width and height of the bounding box in the image coordinate system, and The width and height of the original image. and These represent the actual minimum and maximum depths corresponding to the image.

[0011] A second aspect of the present invention provides a system for implementing the pavement structure damage detection method based on radar full-profile map features, comprising: The data acquisition module is used for acquiring and storing multi-channel data from 3D ground-penetrating radar. The data preprocessing module is used to obtain 3D point cloud profiles of structural damage from the raw data of 3D ground penetrating radar. By segmenting the point cloud data of the 3D point cloud profiles in the horizontal and longitudinal directions, radar map B-Scan and C-Scan slices are obtained. The structural damage detection module is used to automatically identify the structural damage area and type from the acquired B-Scan and C-Scan maps. This module uses a pre-trained YOLO-based dual-input detection network model to detect pavement structural damage. The coordinate transformation module is used to map the coordinates and dimensions of the predicted bounding boxes in the recognition results from the image pixel coordinate system to the actual physical coordinate system represented by mileage station numbers and depth range; The report generation module is used to automatically summarize the test results and generate a detailed report of structural damage and defects in asphalt pavement.

[0012] A third aspect of the present invention is an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the steps in the method for detecting pavement structure damage based on radar full-profile map features.

[0013] In a fourth aspect, a computer-readable storage medium stores computer-executable instructions, which, when executed, are used to implement the steps in the method for detecting pavement structure damage based on radar full-profile map features.

[0014] A fifth aspect of the present invention is a computer program product comprising computer-executable instructions, which, when executed, implement the method for detecting road structure damage based on radar full-profile map features.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. The dual-input YOLO target detection model constructed in this invention innovatively realizes parallel processing of B-Scan and C-Scan data. Through the dual-channel feature extraction and fusion mechanism, it fully explores the feature information under different cross-sectional perspectives, significantly improves the accuracy and robustness of structural damage identification, and solves the problem of misjudgment and missed judgment caused by incomplete information from a single cross-section. 2. The multi-scale discrimination module designed in this invention performs simultaneous analysis and discrimination of fused features at different levels of the feature pyramid, making full use of the rich semantic information of deep features and the high spatial resolution of shallow features, thereby enhancing the model's ability to perceive and recognize features at different scales, effectively improving detection accuracy and reducing the false negative rate. 3. The coordinate transformation algorithm used in this invention achieves accurate mapping from image pixel coordinates to actual physical coordinates, automatically converting the recognition results into mileage station numbers and depth information with clear engineering significance, providing more effective information on the location and size of defects for subsequent maintenance work; 4. This invention establishes a complete automated processing system by integrating the entire process of data acquisition, preprocessing, damage identification, coordinate transformation, and report generation. It transforms the traditional manual experience-dependent map interpretation work into standardized and automated intelligent detection, significantly improving detection efficiency and reducing labor costs. Attached Figure Description

[0016] Figure 1 The diagram shows the flowchart of the asphalt pavement structural damage detection method based on radar full-profile map features according to the present invention.

[0017] Figure 2 The figure shown is a 3D point cloud profile of the structural damage radar map in an embodiment of the present invention.

[0018] Figure 3 The image shows the hyperbolic echo region of structural damage annotated with rectangular boxes using the Labelimg software in an embodiment of the present invention.

[0019] Figure 4 The figure shows the improved dual-input YOLO target detection model framework in an embodiment of the present invention.

[0020] Figure 5 The diagram shows the Focus module structure in an embodiment of the present invention.

[0021] Figure 6 The image shows the Focus slicing operation in an embodiment of the present invention.

[0022] Figure 7 The diagram shows the CSP module structure in an embodiment of the present invention.

[0023] Figure 8 The diagram shown is a structural diagram of FPN+PAN in an embodiment of the present invention.

[0024] Figure 9 The diagram shown is a structural diagram of the multi-scale discrimination module in an embodiment of the present invention.

[0025] Figure 10 The result shows the confusion matrix of the YOLOv8 model.

[0026] Figure 11 The image shows the confusion matrix results of the two-input YOLO object detection model.

[0027] Figure 12 The image shows the recognition results on the YOLOv8 validation set.

[0028] Figure 13 The image shows the validation set recognition results of the dual-input YOLO object detection model. Detailed Implementation

[0029] The present invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0030] Example 1 Reference Figure 1 A method for detecting pavement structure damage based on radar full-profile image features includes the following steps: Step 1: Use a 3D ground-penetrating radar device to collect raw signal data of asphalt pavement structural damage, obtain high-density 3D electromagnetic wave data containing multiple antenna channels, preprocess the 3D electromagnetic wave data to minimize noise interference in the electromagnetic wave data, and improve the data signal-to-noise ratio.

[0031] Step 2 involves converting the preprocessed high-density 3D electromagnetic wave data from Step 1 into a 3D point cloud of structural damage radar atlas. Then, by segmenting the 3D point cloud profile in the horizontal and longitudinal directions, radar full-section atlas slices containing B-Scan and C-Scan maps are obtained. Different structural damage types in the B-Scan and C-Scan maps are labeled using the professional annotation software Labelimg, thus establishing a pavement structural damage radar atlas dataset.

[0032] Step 3, refer to Figure 4A dual-input YOLO target detection model is constructed, comprising an input terminal, a feature extraction module, a feature fusion module, a multi-scale discriminant module, and a regressor. The input terminal includes two independent input ports, used to receive B-Scan and C-Scan maps labeled with structural damage types from step 2, respectively. The feature extraction module, located after the input terminal, contains two independent and structurally corresponding backbone network branches, which extract features from the B-Scan and C-Scan maps to obtain corresponding structural damage feature information. The feature fusion module, connected to the output terminals of the two backbone network branches, concatenates the feature information output by the two backbone network branches to generate fused feature information. The multi-scale discriminant module, connected after the feature fusion module, performs multi-scale feature fusion and enhancement processing on the fused feature information. The regressor, connected after the multi-scale discriminant module, outputs predicted bounding boxes of structural damage within the map region based on the multi-scale feature fusion results. Step 4: Based on the prediction result box obtained in Step 3, the actual location and size information of the structural damage are obtained through coordinate transformation algorithm, and a detailed table of pavement structural damage is automatically generated.

[0033] Furthermore, in step 1, the three-dimensional ground-penetrating radar equipment adopts a 21-channel three-dimensional array ground-penetrating radar system with SGY data acquisition format; the high-density three-dimensional electromagnetic wave data in SGY format is obtained by the radar antenna emitting electromagnetic waves along the direction of travel of the lane and consists of multiple parallel B-scans.

[0034] In step 1, the preprocessing involves sequentially performing direct wave removal, DC drift removal, signal gain, horizontal interference signal removal, and Butterworth bandpass filtering on the high-density three-dimensional electromagnetic wave data using Reflexw software.

[0035] Reference Figure 2 In step 2, the preprocessed high-density three-dimensional electromagnetic wave data is converted into point cloud data by using the Open3D module in the Python language to generate a 3D point cloud model.

[0036] Different structural defect types in B-Scan and C-Scan were labeled using the professional labeling software Labelimg, with reference to... Figure 3 In Labelimg, select rectangular box annotation ( Figure 3The red rectangle indicates a void, and the green rectangle indicates a structural crack. The annotations are in PASCAL VOC format. That is, when training the dual-input detection network model based on the YOLO model, an Annotations folder and a JPEGImages folder are required. These are used to store the XML format label files and the corresponding original annotated images, respectively. The XML files contain information such as the label name and coordinates of the annotation boxes.

[0037] Furthermore, during the model training phase, the input end uses the Mosaic data augmentation algorithm to augment the received B-Scan and C-Scan maps labeled with structural disease types. The data augmentation includes random stitching, cropping, and adaptive image scaling. Random stitching and cropping reduce memory requirements while increasing the number of new samples, effectively solving the problem of overfitting caused by a small number of training samples. Adaptive image scaling fills the original image with the fewest pixels, calculates the scaling ratio, and then scales the image to a fixed size, which helps improve the speed of subsequent network inference.

[0038] The feature extraction module is an improvement on the original backbone network ResNet50, which is divided into two independent and structurally corresponding backbone networks with residual connections, called Dual_ResNet50, to realize dual-channel feature extraction. This module receives B-Scan and C-Scan maps as inputs and completes training in their respective channels to obtain the corresponding structural disease feature information. The formula for calculating the residual connections of Dual_ResNet50 is as follows: ; in, Input for the l-th residual unit, For activation function, For the residual function, This is the weight matrix.

[0039] The feature extraction module consists of five layers, from Layer 0 to Layer 4. Layer 0 comprises a Focus module, while Layers 1 to 4 contain both a Focus module and a CSP1_X module. (Refer to...) Figure 5First, the Focus module performs a slicing operation on the map, transforming it into multi-layer feature map slices. Specifically, it performs nearest-neighbor downsampling on each map, obtaining four complementary slices. These four complementary slices are then stitched together and input into the Focus module's CBS module for further feature extraction. For example, if a 640×640×3 map is input into the Focus module, the slicing operation and stitching result in a 320×320×12 feature map, which is then input into the Focus module's CBS module to obtain... Figure 6 The feature information shown.

[0040] The CSP1_X module is used to extract features and gradually reduce the spatial size so that the regressor can perform object detection, such as... Figure 7 As shown, the CSP1_X module divides the feature information of the feature map extracted by the Focus module into two groups according to the channels for processing: one group is processed by establishing residual connections, and the other group is processed by performing regular convolution operations. The feature information of the two groups of feature maps after processing is merged to obtain the corresponding structural disease feature information. Grouping the feature information of the feature map avoids the repeated calculation of gradient values ​​and improves the overall inference speed of the model. The CSP module with residual connections helps to prevent gradient vanishing in the deep layers of the model, thereby enhancing the feature extraction capability of the model.

[0041] The feature fusion module is connected to the feature extraction module. It splices the output feature information from the two feature extraction modules along the channel dimension to fuse the feature information reflecting internal profile changes in the B-Scan map with the information reflecting planar distribution features in the C-Scan map, thereby generating fused feature information and improving the model's comprehensive ability to distinguish the location and type of structural defects.

[0042] The multi-scale discrimination module is connected to the feature fusion module and contains a three-layer structure (Layer 5, Layer 6, and Layer 7). These layers are used to determine the structural damage type of the fused features at large, medium, and small scales, respectively, and the output consists of the processed results at different scales, such as... Figure 8 As shown.

[0043] The multi-scale discrimination module adopts a structure combining FPN and PAN, and sets a CSP2_X module between the FPN structure and the PAN structure, referring to... Figure 9 The FPN structure is used to pass high-level semantic features to the low-level feature layer through upsampling and fuse them with the low-level features to enhance the semantic information of the shallow features; the PAN structure further fuses low-level localization information with high-level semantic information bidirectionally through downsampling and lateral connections, thereby forming a multi-scale feature representation suitable for target detection at different scales; refer to Figure 7 The CSP2_X is used to further improve the feature fusion effect and enrich the feature expression during the multi-scale feature transfer process.

[0044] The regressor includes convolutional layers, pooling layers, and fully connected layers. The convolutional and pooling layers are used to extract features from the feature map, and the fully connected layers integrate the feature map into a tensor to reduce the influence of feature location on the classification result.

[0045] In this embodiment, the structural information of each layer of the dual-input detection network model based on the YOLO model is shown in Table 1: Table 1 Network structure information for each layer ; Furthermore, in this embodiment, in step 4, the coordinate transformation algorithm maps the coordinates and dimensions of the predicted box in the recognition result from the image pixel coordinate system to the actual physical coordinate system represented by the mileage station and depth range, respectively obtaining the mileage station of the structural damage center point, the center point depth, the actual width of the damage (along the mileage direction), and the actual height of the damage (along the depth direction).

[0046] Furthermore, the coordinate transformation algorithm is calculated using the following formula: ; ; ; ; In the formula, The center point mileage marker, Depth at the center point The actual width of the disease. The actual height of the disease. and This refers to the actual starting and ending mileage station numbers corresponding to the image. and These are the x and y coordinates of the center point of the prediction box in the image coordinate system; and This refers to the width and height of the predicted bounding box in the image coordinate system. and The width and height of the original image; and These represent the actual minimum and maximum depths corresponding to the image.

[0047] Example 2 This embodiment is a specific experiment of Embodiment 1: All network models used in this experiment ran on the PyTorch framework, with Python 3.8 and Torch 1.8.2 as the development language and tool versions, respectively. Considering the high cost of 3D ground-penetrating radar data acquisition and the small dataset size, batch_size=4 and workers=8 were set during model training to ensure that the images in the dataset were sufficiently trained.

[0048] The dataset contains several disease categories, including cracks, poor bonding between the surface layer and base course, poor base course bonding, soil anomalies, irregularities, voids, and expansion joints. Therefore, the labels for the B-Scan dataset are Crack, SID, BID, Soil AB, AB, Void, and Joint, with 7 categories and a dataset size of 2500 images. The labels for the C-Scan dataset are also Crack, SID, BID, Soil AB, AB, Void, and Joint, with 7 categories and a dataset size of 1800 images. The training and validation sets are split in an 8:1:1 ratio.

[0049] To verify the effectiveness and accuracy of the proposed dual-input YOLO target detection model in ground-penetrating radar (GPR) target detection, performance tests were conducted on a self-built GPR dataset, and the results were compared with those of the unmodified YOLOv8 model. Figure 10 , Figure 11 The confusion matrix results show that the dual-input YOLO model achieves an average recognition accuracy of 81.6% for seven types of structural damage, higher than the original YOLOv8 model's 76.7%, with an average detection accuracy improvement of 4.9%. This indicates that the proposed dual-input feature extraction and fusion strategy can effectively improve the recognition capability of multiple types of structural defects in ground-penetrating radar images. In the YOLO target detection algorithm, confidence is used to evaluate the model's confidence in the presence of a target object within the predicted bounding box and the accuracy of the predicted bounding box in pinpointing the target object's location. The recognition performance of the two models on the validation set is as follows: Figure 12 , Figure 13 As shown, the confidence level of this model is higher than that of the original YOLOv8 model in all 7 types of structural damage, indicating that this model has higher reliability in identifying underground diseases.

[0050] Experimental results show that both the model in this application and the original YOLOv8 model can detect underground targets. The model in this application can distinguish between different types of targets more accurately, indicating that the model can perform detection more comprehensively and accurately.

[0051] Example 3 This embodiment provides a pavement structure damage detection system based on radar full-profile map features, including: The data acquisition module is used for acquiring and storing multi-channel data from 3D ground-penetrating radar. The data preprocessing module is used to obtain 3D point cloud profiles of structural damage from the raw data of 3D ground penetrating radar. By segmenting the point cloud data of the 3D point cloud profiles in the horizontal and longitudinal directions, radar map B-Scan and C-Scan slices are obtained. The structural damage detection module is used to automatically identify the structural damage area and type from the acquired B-Scan and C-Scan maps. This module uses a pre-trained dual-input YOLO target detection model to detect pavement structural damage. The coordinate transformation module is used to map the coordinates and dimensions of the predicted bounding boxes in the recognition results from the image pixel coordinate system to the actual physical coordinate system represented by mileage station numbers and depth ranges.

[0052] The report generation module is used to automatically summarize the test results and generate a detailed report of structural damage and defects in asphalt pavement.

[0053] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0054] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0055] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0056] Example 4 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the asphalt pavement structural damage detection method based on radar full-profile map features as described in Embodiment 1 above.

[0057] The computer-readable storage medium in this embodiment can be an internal storage unit of the terminal, such as the terminal's hard disk or memory; the computer-readable storage medium in this embodiment can also be an external storage device of the terminal, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc. equipped on the terminal; furthermore, the computer-readable storage medium can include both the terminal's internal storage unit and external storage devices.

[0058] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0059] Example 5 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the asphalt pavement structural damage detection method based on radar full profile map features as described in Embodiment 1 above.

[0060] Example 6 This embodiment provides a computer program product including computer-executable instructions stored on a machine-readable storage medium (such as a disk, flash memory, or optical disk). When executed by at least one data processing device (such as a microprocessor or digital signal processor), the instructions cause the data processing device to implement the steps in the asphalt pavement structural damage detection method based on radar full-profile map features described in Embodiment 1.

[0061] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0062] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0063] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0065] The above description is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting pavement structure damage based on radar full-profile image features, characterized in that, Includes the following steps: Step 1: Collect raw signal data of road surface structure damage, obtain high-density three-dimensional electromagnetic wave data and preprocess it; Step 2: Convert the preprocessed high-density three-dimensional electromagnetic wave data in Step 1 into a 3D point cloud profile volume. By segmenting the 3D point cloud profile volume, obtain radar full profile map slices containing B-Scan and C-Scan maps. Label the structural defect types in the B-Scan and C-Scan maps to establish a radar map dataset of pavement structural defects. Step 3: Construct a dual-input YOLO target detection model including an input terminal, a feature extraction module, a feature fusion module, a multi-scale discrimination module, and a regressor. The input terminal includes two independent input ports, which are used to receive the B-Scan and C-Scan maps labeled with structural disease types in Step 2, respectively. The feature extraction module contains two independent and structurally corresponding backbone network branches, which extract features from the B-Scan and C-Scan maps respectively to obtain the corresponding structural disease feature information. The feature fusion module concatenates the feature information output by the two backbone network branches to generate fused feature information; the multi-scale discrimination module performs multi-scale feature fusion and enhancement processing on the fused feature information; the regressor is connected to the prediction result box of structural damage in the output map region based on the multi-scale feature fusion result. Step 4: Based on the prediction result box obtained in Step 3, the actual location and size information of the structural damage are obtained through coordinate transformation algorithm, and a detailed table of pavement structural damage is automatically generated.

2. The pavement structure damage detection method based on radar full-profile map features as described in claim 1, characterized in that, In step 1, a three-dimensional ground-penetrating radar device is used to collect original signal data of road surface structure damage. The three-dimensional ground-penetrating radar device adopts a 21-channel three-dimensional array ground-penetrating radar system with SGY data acquisition format. The high-density three-dimensional electromagnetic wave data is obtained by the radar antenna emitting electromagnetic waves along the direction of lane travel and consists of multiple parallel B-scans. The preprocessing process of the high-density three-dimensional electromagnetic wave data is as follows: the high-density three-dimensional electromagnetic wave data is processed sequentially by Reflexw software to remove direct waves, DC drift, signal gain, horizontal interference signal removal and Butterworth bandpass filtering.

3. The pavement structure damage detection method based on radar full-profile map features as described in claim 1, characterized in that, In step 2, the preprocessed high-density three-dimensional electromagnetic wave data is converted into point cloud data using the Open3D module in Python to generate a 3D point cloud model; and the different structural disease types in the B-Scan and C-Scan maps are labeled using the professional annotation software Labelimg.

4. The pavement structure damage detection method based on radar full-profile map features as described in claim 1, characterized in that, In step 3, the input end uses the Mosaic data augmentation algorithm to perform data augmentation on the B-Scan and C-Scan maps labeled with structural disease types. The data augmentation includes random splicing, cropping, and adaptive image scaling. The feature extraction module is an improvement on the original backbone network ResNet50, which is a dual_ResNet50 structure with two independent and structurally corresponding backbone networks with residual connections. The feature extraction module consists of five layers, namely Layer 0 to Layer 4. Layer 0 is composed of the Focus module, and Layers 1 to 4 contain the Focus module and the CSP1_X module. The multi-scale discrimination module adopts a structure combining FPN and PAN, and sets a CSP2_X module between the FPN structure and the PAN structure. The FPN structure is used to pass high-level semantic features to the low-level feature layer through upsampling and fuse them with the low-level features. The PAN structure performs bidirectional fusion of low-level localization information and high-level semantic information through downsampling and lateral connection to obtain fused feature information suitable for target detection at different scales. The regressor includes convolutional layers, pooling layers, and fully connected layers. The convolutional and pooling layers are used to extract features from the feature map, and the fully connected layers integrate the feature map into a tensor to generate a predicted bounding box of structural damage within the map region.

5. The pavement structure damage detection method based on radar full-profile map features as described in claim 4, characterized in that, The formula for calculating the residual connections of Dual_ResNet50 is as follows: ； In the formula, Input for the l-th residual unit, For activation function, For the residual function, This is the weight matrix; The multi-scale discrimination module is connected to the feature fusion module and contains a three-layer structure. It performs structural damage type discrimination on the fused features at three scales.

6. The pavement structure damage detection method based on radar full-profile map features as described in claim 1, characterized in that, In step 4, the coordinate transformation algorithm maps the coordinates and dimensions of the predicted box in the recognition result from the image pixel coordinate system to the actual physical coordinate system represented by the mileage station and depth range, and obtains the mileage station of the structural damage center point, the center point depth, the actual width of the damage along the mileage direction and the actual height of the damage along the depth direction, respectively. The coordinate transformation algorithm calculation formula is as follows: ； ； ； ； In the formula, The center point mileage marker, Depth at the center point The actual width of the disease. The actual height of the disease. and This image corresponds to the actual starting and ending station numbers. and These are the x and y coordinates of the center point of the predicted bounding box in the image coordinate system. and To predict the width and height of the bounding box in the image coordinate system, and The width and height of the original image. and These represent the actual minimum and maximum depths corresponding to the image.

7. A system for implementing the pavement structure damage detection method based on radar full-profile map features as described in claim 1, characterized in that, include: The data acquisition module is used for acquiring and storing multi-channel data from 3D ground-penetrating radar. The data preprocessing module is used to obtain 3D point cloud profiles of structural damage from the raw data of 3D ground penetrating radar. By segmenting the point cloud data of the 3D point cloud profiles in the horizontal and longitudinal directions, radar map B-Scan and C-Scan slices are obtained. The structural damage detection module is used to automatically identify the structural damage area and type from the acquired B-Scan and C-Scan maps. This module uses a pre-trained YOLO-based dual-input detection network model to detect pavement structural damage. The coordinate transformation module is used to map the coordinates and dimensions of the predicted bounding boxes in the recognition results from the image pixel coordinate system to the actual physical coordinate system represented by mileage station numbers and depth range; The report generation module is used to automatically summarize the test results and generate a detailed report of structural damage and defects in asphalt pavement.

8. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the steps in the pavement structure damage detection method based on radar full profile map features as described in claim 1.

9. A computer-readable storage medium, characterized in that, The system stores computer-executable instructions, which, when executed, implement the steps in the pavement structure damage detection method based on radar full-profile map features as described in claim 1.

10. A computer program product, characterized in that, The aforementioned computer program product includes computer-executable instructions that, when executed, implement the steps in the pavement structure damage detection method based on radar full-profile map features as described in claim 1.

Images