Road disease detection method and device based on ground penetrating radar image and medium

Abstract

The invention relates to a road disease detection method and device based on a ground penetrating radar image and a medium, and the method comprises the following steps: collecting underground crack and cavity images through a ground penetrating radar, and constructing original data; preprocessing the original data to obtain a processed data set, and labeling a label file; inputting the processed data set and the label file into a pre-trained improved neural network model to obtain a detection result; the improved neural network model is a network model obtained by improving based on a YOLOv8 network model, the YOLOv8 network model comprises a backbone network, a neck network and a detection head, and the improvement steps are as follows: the backbone network is improved by using a C2f attention module, the neck network is improved by using an asymptotic feature pyramid network, and the detection head is connected with the C2f attention module; the improved neural network model is trained through a transfer learning strategy. Compared with the prior art, the method has the advantages of high detection precision, high speed and the like.

Description

Technical Field

[0001] The invention relates to the field of geological exploration, and in particular to a road disease detection method, device and medium based on ground penetrating radar images. Background Art

[0002] As an important part of urban infrastructure, the safety and reliability of urban roads are of vital importance. Under the coupling of traffic, environmental and other factors, urban roads are prone to hidden diseases such as hidden cracks and voids. Insufficient compaction of the roadbed, excessive exploitation of groundwater, and damage to municipal underground pipelines can also cause water-rich, loose, and hollow roadbeds. Common diseases such as rutting, cracking, and potholes are obvious diseases. These diseases are easy to observe and locate, and are usually collected and identified using road condition data collection vehicles; while diseases represented by voids, looseness, voids, and hidden cracks are hidden diseases, which are hidden, latent, and sudden. If the treatment of hidden diseases is neglected, the scale of hidden diseases will gradually expand and eventually spread to the ground, causing uneven road settlement, collapse, and other diseases, causing a lot of economic losses.

[0003] Conventional detection methods for hidden road defects can be divided into two types: the detection method represented by core sampling is called destructive testing, which can more comprehensively understand the road structure and thus determine the location and type of the defect, but at the same time it will cause certain damage to the road surface structure, and requires more manpower and is inefficient; while the detection methods represented by the drop weight deflectometer, ultrasonic method, high-density electrical method, ground penetrating radar, etc. are called non-destructive testing, which has no destructive effect on the road surface, and has a faster detection speed and lower labor costs. Compared with other non-destructive testing methods, ground penetrating radar has better detection accuracy and higher efficiency, so it is widely used in road defect detection and has achieved good results.

[0004] However, manual interpretation of GPR images requires high professionalism from the staff, and the recognition speed is slow, which makes it difficult to meet the needs of daily maintenance. In recent years, research on deep learning methods has developed rapidly. By deepening the training network and gradually extracting high-dimensional features hidden in the data, the traditional methods have been abandoned, such as strong subjectivity and cumbersome image processing steps, and the recognition accuracy has been greatly improved. YOLOv8 is one of the representative single-stage target detection algorithms. It can achieve high detection accuracy and high detection speed in different natural scenes. However, due to the complexity and particularity of GPR images, the accuracy of the YOLOv8 detection model still has room for improvement. Summary of the invention

[0005] The purpose of the present invention is to provide a road damage detection method, device and medium based on ground penetrating radar images in order to overcome the low precision defect of the above-mentioned prior art.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A road damage detection method based on ground penetrating radar images comprises the following steps:

[0008] Use ground penetrating radar to collect underground crack and cavity images and construct raw data; pre-process the raw data to obtain a processed data set and annotate the label file; input the processed data set and label file into the pre-trained improved neural network model to obtain the detection results;

[0009] The improved neural network model is a network model improved based on the YOLOv8 network model. The YOLOv8 network model includes a backbone network, a neck network and a detection head. The improvement steps include: the backbone network is improved using a C2f attention module, the neck network is improved using an asymptotic feature pyramid network, and the improved neural network model is trained using a transfer learning strategy.

[0010] Furthermore, the original data is constructed as follows:

[0011] S11: Use ground penetrating radar acquisition tools to collect underground crack and cavity images;

[0012] S12: Screening underground cracks and cavities images, and marking characteristic areas of underground cracks and cavities;

[0013] S13: Delete the image content outside the feature area to construct the original data.

[0014] Furthermore, the preprocessing is specifically as follows:

[0015] S21: Taking the ground reflection point of the ground penetrating radar as the starting point, eliminate the time error caused by the gap between the antenna of the ground penetrating radar and the ground in the original data to obtain zero-point correction data;

[0016] S22: using low-pass filtering and high-pass filtering to filter high-frequency components and low-frequency components of the zero-point correction data respectively, to obtain band-pass filtered data;

[0017] S23: using the amplitude value of the original data, subtracting the amplitude mean of all single-channel signal waves, eliminating the background noise of the bandpass filtered data, and obtaining the de-noised data;

[0018] S24: by multiplying each single-channel signal in the denoised data by an amplification coefficient, the energy attenuation of the radar wave when propagating into the underground medium is compensated, thereby obtaining time-varying gain data;

[0019] S25: Divide the time-varying gain data into a training set, a validation set, and a test set, and annotate a label file of the time-varying gain data.

[0020] Furthermore, the C2f attention module is specifically as follows: an attention mechanism is added to the original C2f module of the backbone network, the attention mechanism includes a spatial attention module and a channel attention module, and the spatial attention module and the channel attention module are combined in a serial manner.

[0021] Furthermore, the backbone network is improved using the C2f attention module as follows:

[0022] The improved backbone network includes, starting from the input end, a first convolution module, a second convolution module, a first C2f attention module, a third convolution module, a second C2f attention module, a fourth convolution module, a third C2f attention module, a fifth convolution module, a fourth C2f attention module and an SPPF module.

[0023] Furthermore, the neck network is improved by using the asymptotic feature pyramid network as follows:

[0024] S41: Extract the last layer of features from each feature layer of the backbone, thereby generating a set of features of different scales, denoted as;

[0025] S42: Upsample the features using bilinear interpolation and 1×1 convolution to align the dimensions of each level;

[0026] S43: Use 2×2 convolution with a stride of 2 to achieve 2 times downsampling and 4×4 convolution with a stride of 4 to achieve 4 times downsampling;

[0027] S44: Use the asymptotic feature pyramid network to assign different spatial weights to different levels of features for adaptive feature fusion to obtain a set of multi-scale features;

[0028] S45: Input the multi-scale features into the detection head for prediction to complete the improvement.

[0029] Furthermore, the specific formula of adaptive feature fusion is as follows:

[0030]

[0031] In the formula, Represents the feature vector at position (i, j) from the nth layer to the lth layer, Respectively represent The characteristic parameters of represents the resulting feature vector.

[0032] Furthermore, the specific process of the transfer learning strategy is:

[0033] S51: Use the first public data set as the source domain to train the improved neural network model to obtain the first model;

[0034] S52: using the second public dataset as an auxiliary domain, training the first model to obtain a second model;

[0035] S53: Use the label file and the processed data set as the target domain to train the second model to obtain the pre-trained improved neural network model.

[0036] Furthermore, the pre-trained improved neural network model was tested for performance through precision, recall, average precision, and detection speed.

[0037] In a second aspect of the present invention, a road defect detection device based on ground penetrating radar images includes a memory, a processor, and a program stored in the memory. When the processor executes the program, it implements any of the above road defect detection methods based on ground penetrating radar images.

[0038] A third aspect of the present invention is a storage medium having a program stored thereon, which, when executed, implements any of the above methods for detecting road damage based on ground penetrating radar images.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1) The present invention improves the YOLOv8 neural network model through the C2f attention module in the backbone network and the asymptotic feature pyramid network in the neck network, thereby increasing the attention of the YOLOv8 neural network model to target features, and can adaptively process targets with different scales and aspect ratios, thereby improving the accuracy of target detection.

[0041] 2) The present invention improves the image quality and reduces the impact of noise on disease identification by establishing a real radar image dataset, including underground cracks and cavities, and preprocessing the dataset, which helps to improve the stability and robustness of the target detection algorithm.

[0042] 3) The improved YOLOv8 neural network model of the present invention is trained through transfer learning, which reduces the data demand while improving the model performance, thereby improving the efficiency and accuracy of disease identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Flowchart for detecting hidden hazards on urban roads using the YOLOv8-CBAM-AFPN model.

[0044] Figure 2 This is the effect diagram of ground penetrating radar data preprocessing.

[0045] Figure 3 Schematic diagram of CBAM attention mechanism.

[0046] Figure 4 Schematic diagram of the C2f-CBAM module.

[0047] Figure 5 This is the AFPN structure diagram.

[0048] Figure 6 This is the ASFF process diagram in the AFPN structure.

[0049] Figure 7 Comparison chart of VisTex image set and ground penetrating radar image set.

[0050] Figure 8 This is the flow chart of transfer learning. DETAILED DESCRIPTION

[0051] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0052] Example 1

[0053] The present invention is a method, device and medium for detecting road damage based on ground penetrating radar images, comprising the following steps:

[0054] Use ground penetrating radar to collect underground crack and cavity images and construct raw data; pre-process the raw data to obtain a processed data set and annotate the label file; input the processed data set and label file into the pre-trained improved neural network model to obtain the detection results;

[0055] The improved neural network model is a network model improved based on the YOLOv8 network model. The YOLOv8 network model includes a backbone network, a neck network and a detection head. The improvement steps include: the backbone network is improved using a C2f attention module, the neck network is improved using an asymptotic feature pyramid network, and the improved neural network model is trained using a transfer learning strategy.

[0056] like Figure 1 As shown, the specific steps of the present invention are:

[0057] S1: Use ground penetrating radar to collect underground crack and cavity images and construct raw data.

[0058] S11: Use the ground penetrating radar acquisition tool to collect ground penetrating radar B-Scan profile images of underground cracks and cavities;

[0059] S12: Screening underground cracks and cavities images, and marking characteristic areas of underground cracks and cavities;

[0060] S13: Use image processing software, such as Photoshop, to crop and delete image content outside the feature area to construct original data.

[0061] Specifically, the main basis for selecting the required pictures is:

[0062] 1. The images should be diverse, including underground cracks and cavities of different locations, shapes and sizes;

[0063] 2. The features of underground cracks and cavities in the image should be representative and clearly identifiable.

[0064] S2: Preprocess the original data to obtain the processed data set and annotate the label file;

[0065] S21: Taking the ground reflection point of the ground penetrating radar as the starting point, eliminate the time error caused by the gap between the antenna of the ground penetrating radar and the ground in the original data to obtain zero-point correction data;

[0066] S22: using low-pass filtering and high-pass filtering to filter high-frequency components and low-frequency components of the zero-point correction data respectively, retaining the antenna main frequency, thereby improving the signal-to-noise ratio of the radar wave, and obtaining bandpass filtered data;

[0067] S23: using the amplitude value of the original data, subtracting the amplitude mean value of all single-channel signal waves, eliminating the background noise of the band-pass filtered data, and obtaining the de-noised data;

[0068] S24: by multiplying each single-channel signal in the denoised data by an amplification coefficient, the energy attenuation of the radar wave when propagating into the underground medium is compensated, thereby obtaining time-varying gain data;

[0069] S25: Divide the time-varying gain data into a training set, a validation set, and a test set, and annotate a label file of the time-varying gain data.

[0070] The makesense website was used to create label files for the images in the dataset, and the calibrated png format images were combined with the generated txt format annotation information to establish an underground crack and cavity database. The database was divided into training set, validation set, and test set in a ratio of 0.7:0.15:0.15.

[0071] Specifically, the effect of data preprocessing is as follows: Figure 2 shown.

[0072] S3: Adding the C2f attention (C2f-CBAM) module improves the backbone network’s ability to extract features from data and improves its ability to focus on the target;

[0073] like Figure 3 As shown in the figure, the C2f-CBAM module adds an attention mechanism CBAM to the original C2f module, which is an attention module for feedforward neural networks. By adding this attention module, the features of complex backgrounds are suppressed and the features of defects are highlighted. The CBAM attention mechanism contains two independent sub-modules, namely the spatial attention module and the channel attention module. These two sub-modules are combined in a serial manner to generate attention feature map information in the channel dimension and the spatial dimension in a serialized manner. This method is plug-and-play and can save parameters and computing power.

[0074] Specifically, the spatial attention module M s (F) The relationship between different spatial positions is used to learn the 2D spatial weight map, which is then multiplied by the corresponding spatial position to learn more representative features. The formula is as follows:

[0075]

[0076] In the formula, σ represents the sigmoid activation function, f 7×7 represents a convolution operation with a filter size of 7×7.

[0077] Specifically, the channel attention module M C (F) is generated by using the inter-channel relationship of features, and the formula is as follows:

[0078]

[0079] Where σ represents the sigmoid activation function, W0∈R C / r×C , W1∈R C×C / r Represents the weights of the multilayer perceptron.

[0080] The specific improvement is to replace the C2f module in the backbone network with the C2f-CBAM module. Figure 4 As shown, the improved backbone network includes, starting from the input end, a first convolution module, a second convolution module, a first C2f attention module, a third convolution module, a second C2f attention module, a fourth convolution module, a third C2f attention module, a fifth convolution module, a fourth C2f attention module and an SPPF module.

[0081] S4: AFPN is used in the neck layer of YOLOv8 to strengthen the multi-scale fusion process, which improves the feature expression ability and reduces the parameter cost;

[0082] like Figure 5 , Figure 6As shown in the figure, the asymptotic feature pyramid network (AFPN) is a feature pyramid network module constructed by introducing the adaptive feature fusion (ASFF) module. The specific process is as follows:

[0083] S41: Extract the last layer of features from each feature layer of the backbone, thereby generating a set of features of different scales, denoted as {C3, C4, C5};

[0084] S42: Upsample the features using bilinear interpolation and 1×1 convolution to align the dimensions of each level;

[0085] S43: Use 2×2 convolution with a stride of 2 to achieve 2 times downsampling and 4×4 convolution with a stride of 4 to achieve 4 times downsampling;

[0086] S44: Use the asymptotic feature pyramid network to assign different spatial weights to different levels of features for adaptive feature fusion to obtain a set of multi-scale features {P3, P4, P5}. The specific formula is as follows:

[0087]

[0088] In the formula, Represents the feature vector at position (i, j) from the nth layer to the lth layer, Respectively represent The characteristic parameters of represents the resulting feature vector;

[0089] S45: Input the multi-scale features {P3, P4, P5} into the detection head for prediction to complete the improvement.

[0090] S5: Using the transfer learning strategy, the improved model is trained on the source domain, auxiliary domain, and target domain in turn to obtain pre-training weights, further improving model performance and accelerating model training:

[0091] Transfer learning is an algorithm in machine learning, which refers to the transfer of features learned by the algorithm in a large sample to a new small sample learning task, including instance-based transfer learning, feature-based transfer learning, parameter-based transfer learning, and related knowledge-based transfer learning. The present invention applies feature-based transfer learning, which is mainly used when there is a significant similarity between source domain data or auxiliary domain data and target domain data. In the present invention, the VisTex image set and the ground penetrating radar image have similar texture features, as shown in the comparison figure. Figure 7 As shown in , it can be used as an auxiliary domain. The specific transfer learning process is as follows Figure 8 As shown:

[0092] S51: Use the COCO128 target detection dataset as the source domain to train the improved neural network model, namely the YOLOv8-CBAM-AFPN model, to obtain the first model;

[0093] S52: Use the VisTex dataset as an auxiliary domain to train the first model and obtain the second model;

[0094] S53: Using the label file and the processed data set as the target domain, training the second model to obtain a pre-trained improved neural network model, so that the model has the ability to recognize underground cracks and cavities ground penetrating radar images.

[0095] S6: Set up an ablation experiment and measure the effect of the improved model through evaluation indicators, including precision P, recall R, average precision mAP and detection speed FPS. The specific formula is as follows:

[0096] The precision rate P is defined as the correct detection rate of all detected targets, which can be expressed as:

[0097]

[0098] The recall rate R is defined as the detection accuracy rate among all positive samples, which can be expressed as:

[0099]

[0100] AP is defined as the area enclosed by the precision rate, recall rate PR curve and the coordinate axis, which can be expressed as:

[0101]

[0102] The average precision mAP is defined as the mean of all categories of AP, which can be expressed as:

[0103]

[0104] The detection speed FPS is defined as the number of images detected per second by the YOLOv8-CBAM-AFPN model, where TP is the number of positive samples predicted as positive; FP is the number of negative samples predicted as positive; FN is the number of positive samples predicted as negative; TN is the number of negative samples predicted as negative, and n is the category, which is 2 in this embodiment. The experimental results of this embodiment are shown in Table 1:

[0105] Table 1 Comparison of prediction performance

[0106]

[0107] From the analysis of Table 1, we can see that when the network replaces the C2f module with the C2f-CBAM module, or introduces the AFPN feature fusion network, the precision and recall rates are slightly improved, the mAP value also increases, but the FPS decreases slightly. The reason may be that the introduction of the attention mechanism and the adaptive feature fusion module increases the complexity of the model, and the corresponding performance is also improved, but the reasoning efficiency decreases; after citing transfer learning, all indicators are improved, indicating that transfer learning helps to improve the model training speed and model performance.

[0108] In summary, a ground penetrating radar urban road hidden disease image detection method based on transfer learning and YOLOv8-CBAM-AFPN model can help improve the accuracy and efficiency of disease recognition and is beneficial to road maintenance work.

[0109] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0110] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0111] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A road damage detection method based on ground penetrating radar images, characterized in that: The following steps are involved: Use ground penetrating radar to collect underground cracks and cavities images and construct raw data; Preprocess the original data to obtain a processed data set and annotate the label file; input the processed data set and label file into the pre-trained improved neural network model to obtain the detection results; The improved neural network model is a network model improved based on the YOLOv8 network model, the YOLOv8 network model includes a backbone network, a neck network and a detection head, and the improvement step includes: the backbone network is improved using a C2f attention module, the neck network is improved using an asymptotic feature pyramid network, and the improved neural network model is trained using a transfer learning strategy.

2. The method for detecting road damage based on ground penetrating radar images according to claim 1, characterized in that: The construction of the original data is specifically as follows: S11: Use ground penetrating radar acquisition tools to collect underground crack and cavity images; S12: Screening underground cracks and cavities images, and marking characteristic areas of underground cracks and cavities; S13: Delete the image content outside the feature area to construct the original data.

3. The method for detecting road damage based on ground penetrating radar images according to claim 1, characterized in that: The pre-processing is specifically as follows: S21: Taking the ground reflection point of the ground penetrating radar as the starting point, eliminate the time error caused by the gap between the antenna of the ground penetrating radar and the ground in the original data to obtain zero-point correction data; S22: using low-pass filtering and high-pass filtering to filter high-frequency components and low-frequency components of the zero-point correction data respectively, to obtain band-pass filtered data; S23: using the amplitude value of the original data, subtracting the amplitude mean of all single-channel signal waves, eliminating the background noise of the bandpass filtered data, and obtaining the de-noised data; S24: by multiplying each single-channel signal in the denoised data by an amplification coefficient, the energy attenuation of the radar wave when propagating into the underground medium is compensated, thereby obtaining time-varying gain data; S25: Divide the time-varying gain data into a training set, a validation set, and a test set, and annotate a label file of the time-varying gain data.

4. The method for detecting road damage based on ground penetrating radar images according to claim 1, characterized in that: The C2f attention module is specifically: adding an attention mechanism to the original C2f module of the backbone network, the attention mechanism includes a spatial attention module and a channel attention module, and the spatial attention module and the channel attention module are combined in a serial manner.

5. The method for detecting road damage based on ground penetrating radar images according to claim 4, characterized in that: The backbone network is improved using the C2f attention module as follows: The improved backbone network includes, starting from the input end, a first convolution module, a second convolution module, a first C2f attention module, a third convolution module, a second C2f attention module, a fourth convolution module, a third C2f attention module, a fifth convolution module, a fourth C2f attention module and an SPPF module.

6. The method for detecting road damage based on ground penetrating radar images according to claim 1, characterized in that: The neck network is improved by using asymptotic feature pyramid network as follows: S41: Extract the last layer of features from each feature layer of the backbone, thereby generating a set of features of different scales, denoted as; S42: Upsample the features using bilinear interpolation and 1×1 convolution to align the dimensions of each level; S43: Use 2×2 convolution with a stride of 2 to achieve 2 times downsampling and 4×4 convolution with a stride of 4 to achieve 4 times downsampling; S44: Use the asymptotic feature pyramid network to assign different spatial weights to different levels of features for adaptive feature fusion to obtain a set of multi-scale features; S45: Input the multi-scale features into the detection head for prediction to complete the improvement.

7. The method for detecting road damage based on ground penetrating radar images according to claim 6, characterized in that: The specific formula of the adaptive feature fusion is as follows: In the formula, represents the feature vector at position (i, j) from the nth layer to the lth layer, Respectively represent The characteristic parameters of represents the resulting feature vector.

8. The method for detecting road damage based on ground penetrating radar images according to claim 1, characterized in that: The specific process of the transfer learning strategy is as follows: S51: Use the first public data set as the source domain to train the improved neural network model to obtain the first model; S52: using the second public dataset as an auxiliary domain, training the first model to obtain a second model; S53: Use the label file and the processed data set as the target domain to train the second model to obtain the pre-trained improved neural network model.

9. A road disease detection device based on ground penetrating radar images, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method for detecting road damage based on ground penetrating radar images as described in any one of claims 1 to 8 is implemented.

10. A storage medium having a program stored thereon, characterized in that: When the program is executed, a road damage detection method based on ground penetrating radar images as described in any one of claims 1 to 8 is implemented.

Images