Lightweight crack penetration degree detection method based on ground penetrating radar and Swin Transform optimization

Abstract

The invention discloses a lightweight crack penetration degree detection method based on ground penetrating radar and Swin Transform optimization, and the method comprises the following steps: 1, collecting ground penetrating radar data, and carrying out the preprocessing and feature extraction; step 2, constructing a multi-feature fusion data set; step 3, designing a two-stage through crack detection model based on Swin Transform optimization; 4, a first-stage through crack detection model optimized on the basis of Swin Transformer-YOLOv8 is designed, and a second-stage through crack detection model is designed on the basis of Swin Transformer-YOLOv8 optimization; and step 5, training and predicting the data of the training set and the test set according to the model architecture in the step 3 and the step 4. According to the invention, the accuracy of crack penetration degree detection is improved, and a scientific basis is provided for road maintenance; the detection speed is greatly improved, and the requirement of large-area rapid detection is met; the calculation complexity is reduced, the equipment performance pressure is reduced, and the detection efficiency is improved; the generalization ability of the model is enhanced, and the method can adapt to different road conditions; and the development of an intelligent road detection technology is promoted.

Description

Technical Field

[0001] The present invention relates to the field of ground penetrating radar technology and deep learning target detection, and in particular to a lightweight crack penetration degree detection method based on ground penetrating radar and Swin Transformer optimization. Background Art

[0002] With the rapid development of economy and the acceleration of urbanization, the traffic volume of highways has increased dramatically. Asphalt pavement, as the main urban road material, has attracted more and more attention for its performance and service life. Asphalt pavement is prone to cracks and other diseases under long-term load stress and temperature stress. If these diseases are not detected and repaired in time, they will lead to further damage to the pavement structure and even cause serious safety accidents such as road collapse. Therefore, early identification and evaluation of cracks in asphalt pavement is particularly important.

[0003] Traditional asphalt pavement crack detection methods mainly rely on manual visual inspection, which is inefficient, inaccurate, and highly subjective. With the development of ground penetrating radar (GPR) technology, it has been widely used in road crack detection as a non-destructive testing technology due to its advantages such as simple operation, fast speed, and high resolution. However, the analysis of GPR data requires professional knowledge and experience, and its processing capabilities for nonlinear and non-stationary signals are limited.

[0004] Applying deep learning technology to crack detection in ground penetrating radar images can significantly improve the automation level and accuracy of crack detection.

[0005] In the field of asphalt pavement crack detection, existing technical solutions mainly rely on traditional image processing techniques and emerging deep learning algorithms. Although these methods can provide useful results in some cases, they are usually difficult to adapt to the complexity and diversity of crack morphology, especially when cracks cross or branch. In addition, these methods also show limitations when dealing with nonlinear and non-stationary signals, because they are often based on linear assumptions and cannot effectively capture the dynamic characteristics of the signal. However, existing deep learning methods still have limitations when dealing with nonlinear and non-stationary signals.

[0006] The shortcomings of existing methods in the crack detection process are as follows: (1) The accuracy and robustness of existing methods are insufficient in complex environments, especially when the crack morphology is complex or there are intersections or branches; (2) Traditional ground penetrating radar analysis methods require manual operation and analysis, which limits the real-time and automation of detection; (3) Although some deep learning methods have improved detection accuracy, they have high computational complexity, resulting in slow detection speed and are not suitable for large-scale applications; (4) Existing methods lack the ability to perform time-frequency analysis on ground penetrating radar signals and cannot fully utilize the time-frequency characteristics of the signals to improve detection performance.

[0007] The present invention uses ground penetrating radar equipment and deep learning technology to design two through crack detection models, which are respectively suitable for refined detection in key areas and rapid general detection in a large range. It can not only improve the efficiency and quality of road maintenance and reduce maintenance costs, but also extend the service life of roads by timely discovering and repairing cracks, reduce traffic accidents caused by road collapse, and protect life and property safety. It can also provide a scientific basis for road design and construction, optimize road structure design, and improve road stability and durability; in the fields of environmental monitoring, disaster prevention, etc., the present invention also has broad application prospects. Summary of the invention

[0008] In view of this, the present invention provides a lightweight crack penetration degree detection method based on ground penetrating radar and Swin Transformer optimization.

[0009] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0010] The lightweight crack penetration detection method based on ground penetrating radar and Swin Transformer optimization includes the following steps:

[0011] Step 1: Collect ground penetrating radar data and perform preprocessing and feature extraction

[0012] Use ground penetrating radar equipment to scan the asphalt pavement and collect radar signal data below the pavement; process the raw radar data; extract key information that helps crack detection from the pre-processed data;

[0013] Step 2: construct a multi-feature fusion data set;

[0014] The instantaneous amplitude, instantaneous phase and instantaneous frequency characteristic images of the ground penetrating radar signal are combined to form a multi-feature fusion data set; then the data set is expanded and annotated to complete the construction of the multi-feature fusion data set of the ground penetrating radar through cracks;

[0015] Step 3: Design a two-stage through crack detection model based on Swin Transformer optimization

[0016] (3a) Optimizing the overall framework of the network

[0017] Swin Transformer is introduced into the backbone network, and FPN network is introduced after the backbone network. Combining the advantages of Mask R-CNN, ROI pooling is replaced with ROIAlign to improve the two-stage detection network.

[0018] (3b) FPN multi-scale fusion module design

[0019] Through the top-down feature extraction network, feature maps of different depths and scales are generated. The resolution of the deepest feature map is dynamically adjusted using upsampling. The feature maps of two adjacent layers with different scales are fused. The fused feature maps containing the shallow network position information and the deep network semantic information are used for target detection.

[0020] (3c) ROI pooling optimization

[0021] After calculating the values ​​of the four positions by bilinear interpolation, ROI outputs of the same size are obtained by Maxpooling;

[0022] (3d) Smooth L1 loss function optimization

[0023] The bounding box position loss function uses the Smooth L1 function, which combines the L1 loss function and the L2 loss function;

[0024] Step 4: Design a one-stage through crack detection model based on Swin Transformer-YOLOv8 optimization

[0025] (4a) Fusion Swin Transformer Network

[0026] Fuse YOLOv8 and Swin Transformer, and introduce the Swin Transformer network into the Backbone part of YOLOv8;

[0027] (4b) Weighted bidirectional feature fusion network

[0028] First, the deep features are continuously upsampled and fused with the bottom features. After the upsampling is completed, the bottom features are downsampled and fused with the deep features. Two horizontal connection paths are added between the two feature extraction paths to fuse the feature maps generated at each stage of the backbone network with the feature maps to be tested.

[0029] (4c) Overall architecture of the Swin Transformer-YOLOv8 model

[0030] YOLOv8 integrates Swin Transformer and introduces weighted bidirectional feature pyramid (BiFPN);

[0031] Step 5: Train and predict the training set and test set data according to the model architecture in steps 3 and 4.

[0032] Preferably, in step 1, the ground penetrating radar data includes detailed information of the pavement structure, especially the location and depth of cracks;

[0033] Preprocessing includes zero bias correction, FIR filtering, and signal gain adjustment processing;

[0034] Feature extraction includes amplitude information, frequency information and phase information.

[0035] Preferably, in step 2, the data set is expanded using random cropping, mirror flipping, and contrast enhancement methods, and finally the 8 data sets are labeled to complete the construction of the multi-feature fusion data set of ground penetrating radar through-cracks.

[0036] Preferably, in step 3, the L1 and L2 loss functions are as follows:

[0037]

[0038] Where f(x i ) and y i They represent the predicted value and the corresponding true value of the i-th sample respectively, and n is the number of samples.

[0039] Preferably, in step 3, the formula of Smooth L1 is as follows:

[0040]

[0041] This function is a piecewise function. Between [-1, 1], it is L2 loss, which solves the problem that L1 has a breaking point at 0. Outside the interval [-1, 1], it is L1 loss, which solves the problem of gradient explosion of outliers. Therefore, the gradient can be limited from the following two aspects: first, when the error between the predicted value and the true value is too large, the gradient value will not be too large; second, when the error between the predicted value and the true value is very small, the gradient value is small enough.

[0042] Preferably, the specific method adopted in step 5 is:

[0043] The through-fracture multi-feature dataset constructed in step 2 is used;

[0044] Step 3 uses the SGD optimizer as the basic iterator, with the weight_decay factor being 0.0001 and the momentum factor being 0.9. A single GPU is used for training. The batchsize is set to 4 in both the original network and the improved network, and the corresponding initial learning rates are set to 0.001. During the training process, the model first loads the initialization weight file of the Swin Transformer trained on the public dataset ImageNet, and the total number of training iterations is set to 100.

[0045] In step 4, the SGD optimizer is used as the basic iterator, the initial learning rate is set to 0.001, the weight_decay weight decay factor is 0.0005, and the momentum factor is 0.937; a single GPU is used for training, the batch size is set to 4, and the total number of training iterations is set to 100 rounds considering the comprehensive training speed and the original image size;

[0046] Step 3 and step 4 use accuracy, average precision, mean average precision, and PR curve as evaluation indicators to measure the performance of the model;

[0047] In step three, the original signal, single feature and multi-feature fusion data sets are trained using a two-stage network structure optimized based on SwinTransformer.

[0048] Preferably, the accuracy rate represents the proportion of correctly detected results in all model detection results, and the recall rate represents the proportion of correctly detected results in all real annotation frames, as shown in the following formula:

[0049]

[0050]

[0051] Among them, TP means that the model is correctly identified, that is, the number of cracks correctly detected; FP means that the model is incorrectly identified, that is, the number of cracks falsely detected; FN means that positive samples are judged as negative samples, that is, the number of cracks missed; P and R can both measure the detection effect of the model in some scenarios, but these two indicators are inversely proportional, so AP is needed to comprehensively evaluate the generalization of the model using P and R indicators.

[0052] Preferably, the average precision (AP) value is calculated by the area enclosed by the PR curve and the coordinate axis. The larger the area of ​​the PR curve, the better the model performance. The formula is as follows:

[0053]

[0054] Preferably, the mean average precision (mAP) is the best evaluation indicator to demonstrate the performance of the target detection model. mAP calculates the average value of AP of all categories in the data set. The calculation formula is as follows:

[0055]

[0056] Where n represents all categories in the dataset. mAP mainly includes three evaluation criteria: mAP@0.5, mAP@0.75 and mAP@0.5:0.95. mAP@0.5 and mAP@0.75 refer to the mAP values ​​when IoU=0.5 and IoU=0.75 respectively. mAP@"0.5:0.95" refers to the average value of the mAP value corresponding to each IoU when IoU changes from 0.5 to 0.95 with a step size of 0.05.

[0057] Compared with the prior art, the present invention has achieved the following technical effects:

[0058] (1) The present invention, by combining ground penetrating radar technology and deep learning models, can accurately identify and locate cracks in asphalt pavement, especially the degree of crack penetration. This high-precision detection capability provides reliable data support for road maintenance and repair work. Specifically, the Hilbert-Huang transform and two-dimensional discrete Fourier transform technology adopted by the present invention can analyze crack characteristics from multiple spatial domains and extract more accurate crack feature images. These feature images serve as inputs to the deep learning model, allowing the model to more accurately learn and identify crack characteristics, thereby improving the accuracy of crack penetration detection. The application of this technology enables road managers to formulate more scientific and effective maintenance plans based on the actual conditions of the cracks, thereby extending the service life of the road and reducing maintenance costs.

[0059] (2) The one-stage detection model Swin Transformer-YOLOv8 adopted by the present invention can achieve rapid scanning and analysis of large areas of roads while maintaining high detection accuracy. Specifically, the model significantly improves the reasoning speed of the model through optimized network structure and algorithm, so that the number of images that can be processed per second is greatly increased. This improvement greatly improves the detection efficiency, making it possible to detect longer sections of road within a limited time, providing a strong guarantee for road safety management. For example, in actual applications, the detection speed of the present invention can reach 35.71 frames per second, which means that a large area of ​​road can be covered quickly and potential crack problems can be identified in a timely manner.

[0060] (3) The present invention fully considers computational efficiency when designing the model, and effectively reduces the computational complexity of the model by optimizing the algorithm and model structure. Specifically, the Swin Transformer-YOLOv8 model adopted by the present invention achieves efficient detection at a lower computational cost by combining the efficient feature extraction capability of Swin Transformer and the fast detection capability of YOLOv8. This design enables the detection model to be run quickly even on lower-performance devices without the need for high-end computing resources. This design not only improves the accessibility and practicality of the detection, but also reduces the energy consumption and cost of the equipment, making the detection technology of the present invention easier to deploy and use.

[0061] (4) The present invention adopts a multi-feature fusion technical solution when constructing a data set, which not only enhances the recognition of crack features, but also improves the model's adaptability to different road conditions. Specifically, by fusing features such as the instantaneous amplitude, instantaneous phase, and instantaneous frequency of the ground penetrating radar signal, the present invention constructs a multi-feature fusion data set. This data set construction method enables the model to learn more comprehensive feature information during the training process, thereby improving the model's ability to recognize crack features under different road conditions. The application of this technology enables the model to maintain stable detection performance in a variety of environments, and can accurately identify cracks regardless of whether it is new or old roads or under different climatic conditions, thereby ensuring the reliability of the detection results.

[0062] (5) The successful implementation of the present invention provides a new technical means for the field of road detection and promotes the development of intelligent road detection technology. Specifically, the present invention not only improves the automation level of detection through the application of deep learning technology, but also lays the foundation for the intelligent and unmanned road detection technology in the future. The application of this technology can not only improve the efficiency and accuracy of road detection, but also provide more scientific data support for road maintenance decisions, and has important scientific research value and application prospects. In addition, the technical solution of the present invention also provides reference for other fields, such as non-destructive testing of infrastructure such as bridges and tunnels, further broadening the application scope of intelligent detection technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a two-stage detection network diagram optimized based on Swin Transformer of the present invention;

[0064] Figure 2 This is the principle diagram of ROI Align of the present invention;

[0065] Figure 3 This is the Smooth L1 curve distribution diagram of the present invention;

[0066] Figure 4 A fusion diagram of the backbone network module of the present invention;

[0067] Figure 5 Two FPN module diagrams of the present invention;

[0068] Figure 6 This is the structure diagram of Swin Transformer-YOLOv8 of the present invention;

[0069] Figure 7 This is a graph of the loss function of a single feature data set of the present invention;

[0070] Figure 8 This is a graph of the loss function of the feature fusion data set of the present invention;

[0071] Fig. 9 This is a diagram of the IA+IF one-stage network training process of the present invention;

[0072] Fig.10 It is the PR curve diagram of IA+IF of the present invention;

[0073] Fig.11 This is a graph of experimental results of a single feature data set of the present invention;

[0074] Fig.12 This is a graph of experimental results of multi-feature fusion data set of the present invention;

[0075] Fig.13 This is a comparison chart of the detection effects of YOLOv8 of the present invention and the optimized Swin Transformer-YOLOv8;

[0076] Fig.14 This is a comparison diagram of the detection effects between the models of the present invention; DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] The present invention discloses a lightweight crack penetration degree detection method based on ground penetrating radar and Swin Transformer optimization, comprising the following steps:

[0079] Step 1: Collect ground penetrating radar data and perform preprocessing and feature extraction

[0080] Use ground penetrating radar equipment to scan the asphalt pavement and collect radar signal data below the pavement; process the raw radar data; extract key information that helps crack detection from the pre-processed data;

[0081] GPR data include detailed information on the pavement structure, especially the location and depth of cracks;

[0082] Preprocessing includes zero bias correction, FIR filtering, and signal gain adjustment processing;

[0083] Feature extraction includes amplitude information, frequency information and phase information;

[0084] Step 2: construct a multi-feature fusion data set;

[0085] The instantaneous amplitude, instantaneous phase and instantaneous frequency characteristic images of the ground penetrating radar signal are combined to form a multi-feature fusion data set; the data set is expanded using random cropping, mirror flipping and contrast enhancement methods, and finally 8 data sets are annotated to complete the construction of the multi-feature fusion data set of ground penetrating radar through cracks;

[0086] Step 3: Design a two-stage through crack detection model based on Swin Transformer optimization

[0087] (3a) Optimizing the overall framework of the network

[0088] Swin Transformer is introduced into the backbone network, and FPN network is introduced after the backbone network. Combining the advantages of Mask R-CNN, ROI pooling is replaced with ROIAlign to improve the two-stage detection network. The improved overall network structure is as follows: Figure 1 As shown;

[0089] (3b) FPN multi-scale fusion module design

[0090] Through the top-down feature extraction network, feature maps of different depths and scales are generated. The resolution of the deepest feature map is dynamically adjusted using upsampling. The feature maps of two adjacent layers with different scales are fused. The fused feature maps containing the shallow network position information and the deep network semantic information are used for target detection.

[0091] (3c) ROI pooling optimization

[0092] After calculating the values ​​of the four positions by bilinear interpolation, the ROI output of the same size is obtained by Maxpooling; the ROIAlign calculation principle diagram is shown in Figure 2;

[0093] (3d) Smooth L1 loss function optimization

[0094] The bounding box position loss function uses the Smooth L1 function, which combines the L1 loss function and the L2 loss function;

[0095] The L1 and L2 loss functions are as follows:

[0096]

[0097] Where f(x i ) and y i They represent the predicted value and the corresponding true value of the i-th sample, respectively, and n is the number of samples;

[0098] The Smooth L1 formula is as follows:

[0099]

[0100] This function is a piecewise function. Between [-1, 1], it is L2 loss, which solves the problem that L1 has a breakpoint at 0. Outside the interval [-1, 1], it is L1 loss, which solves the problem of gradient explosion of outliers. Therefore, the gradient can be limited from the following two aspects: first, when the error between the predicted value and the true value is too large, the gradient value will not be too large; second, when the error between the predicted value and the true value is very small, the gradient value is small enough; the curve distribution is as follows Figure 3 As shown;

[0101] Step 4: Design a one-stage through crack detection model based on Swin Transformer-YOLOv8 optimization

[0102] (4a) Fusion SwinTransformer network

[0103] Fuse YOLOv8 and SwinTransformer, and introduce the SwinTransformer network into the Backbone part of YOLOv8; Figure 4 As shown;

[0104] (4b) Weighted bidirectional feature fusion network

[0105] First, the deep features are continuously upsampled and fused with the bottom features. After the upsampling is completed, the bottom features are downsampled and fused with the deep features. Two horizontal connection paths are added between the two feature extraction paths to fuse the feature maps generated at each stage of the backbone network with the feature maps to be tested. Figure 5 As shown;

[0106] (4c) Overall architecture of the SwinTransformer-YOLOv8 model

[0107] YOLOv8 integrates SwinTransformer and introduces weighted bidirectional feature pyramid (BiFPN); Figure 6 As shown;

[0108] Step 5: Train and predict the training set and test set data according to the model architecture in steps 3 and 4:

[0109] The through-fracture multi-feature dataset constructed in step 2 is used;

[0110] In step 3, the SGD optimizer is used as the basic iterator, the weight_decay factor is 0.0001, the momentum factor is 0.9, and a single GPU is used for training. The batch size is set to 4 in both the original network and the improved network, and the corresponding initial learning rate is set to 0.001. During the training process, the model first loads the initialization weight file of the SwinTransformer trained on the public dataset ImageNet, and the total number of training iterations is set to 100.

[0111] In step 4, the SGD optimizer is used as the basic iterator, the initial learning rate is set to 0.001, the weight_decay weight decay factor is 0.0005, and the momentum factor is 0.937; a single GPU is used for training, the batch size is set to 4, and the total number of training iterations is set to 100 rounds considering the comprehensive training speed and the original image size;

[0112] Step 3 and step 4 use accuracy, average precision, mean average precision, and PR curve as evaluation indicators to measure the performance of the model;

[0113] The accuracy rate indicates the proportion of correct detections in all model detection results, and the recall rate indicates the proportion of correct detections in all real annotation boxes, as shown in the following formula:

[0114]

[0115] TP means the model is correctly identified, that is, the number of cracks correctly detected; FP means the model is incorrectly identified, that is, the number of cracks falsely detected; FN means that positive samples are judged as negative samples, that is, the number of cracks missed; P and R can both measure the detection effect of the model in some scenarios, but these two indicators are inversely proportional, so AP is needed to comprehensively evaluate the generalization of the model with P and R indicators;

[0116] The average precision (AP) value is calculated by the area enclosed by the PR curve and the coordinate axis. The larger the area of ​​the PR curve, the better the model performance. The formula is as follows:

[0117]

[0118] The mean average precision (mAP) is the best evaluation indicator to show the performance of the target detection model. mAP calculates the average value of AP of all categories in the data set. The calculation formula is as follows:

[0119]

[0120] Where n represents all categories in the dataset. mAP mainly includes three evaluation criteria: mAP@0.5, mAP@0.75 and mAP@0.5:0.95. mAP@0.5 and mAP@0.75 refer to the mAP values ​​when IoU=0.5 and IoU=0.75 respectively. mAP@"0.5:0.95" refers to the average value of the mAP value corresponding to each IoU when IoU changes from 0.5 to 0.95 with a step size of 0.05.

[0121] In step 3, the original signal, single feature and multi-feature fusion data sets are trained using a two-stage network structure optimized based on SwinTransformer;

[0122] Example 1: Two-stage through-crack detection model based on Swin Transformer optimization

[0123] (1a) Comparison and analysis of experimental results on different datasets

[0124] The original signal, single feature and multi-feature fusion data sets obtained are trained using the two-stage network structure based on Swin Transformer optimization proposed in step 3 of the technical solution. The training model is able to identify and locate through cracks (tc) and non-through cracks (nc), and the impact of different data sets on model performance is verified. The single feature data set of ground penetrating radar cracks includes: instantaneous amplitude (IA), instantaneous phase (IP) and instantaneous frequency (IF), and the original signal (Origin) is added for comparison. The loss function of the training process is as follows: Figure 7 As shown in Figure 2, the loss function gradually stabilizes after 10,000 training iterations, and the model converges in the training of the four data sets.

[0125] There are four feature fusion datasets, namely IA+IF, IA+IF, IF+IP, and IA+IP+IF. In order to clearly compare the performance of the model on the fusion dataset, Figure 8 As shown in Figure 1, in addition to the loss function curve of the feature fusion dataset, it also contains the loss function of the original signal dataset. During the training of the feature fusion dataset, the loss function decreases rapidly in the first 400 iterations and then stabilizes. The IA+IF model has the smallest loss after convergence.

[0126] After the loss function converges in the training process, the best.pt weight file generated last is used to load the weight file into the model, and the various indicators of the model are tested with the test set. The comprehensive and detailed results are shown in Table 1.

[0127] Table 1 Comparison results of different data sets

[0128]

[0129] Compare the model test results of the original signal and the single feature data set:

[0130] The two-stage network optimized based on Swin Transformer has better performance indicators in the original signal test set than the single feature test set, with an mAP accuracy of 0.838. The accuracy index of the IA dataset is slightly lower than that of the original dataset. This is because the IA dataset obtained from the echo signal is essentially a threshold processing of the abnormal area in the original signal. The part with smaller amplitude lacks other judgment information, which increases the difficulty of network feature extraction. IA reflects the rapid change of amplitude intensity at the abnormal position. When the electromagnetic wave encounters a through crack, due to the small dielectric constant of the air, the energy of the absorbed electromagnetic wave is low. Compared with other single feature datasets, it is difficult for the noise energy to cause significant amplitude changes in a short time.

[0131] In the IP dataset, the network's automatic identification ability is poor. It is possible that IP reflects the continuity of adjacent signals. When electromagnetic waves propagate to different media, the phase of the reflected wave will change, and factors such as scattered waves will cause more noise in the dataset. Therefore, the IP test set has the lowest accuracy.

[0132] The accuracy of the IF dataset is slightly higher than that of the IP dataset. Since high-frequency electromagnetic waves are more sensitive to structural layers with different dielectric constants, the differential calculation involving the phase change of the signal makes the energy of the high-frequency noise drop faster on the spectrum, which helps to suppress certain noise components.

[0133] Compare the model test results of the original signal and the multi-feature fusion data set:

[0134] On the fusion feature dataset, IA+IF has the highest average precision. Compared with the original signal dataset with better verification effect, the verification index mAP and R of the IA+IF dataset are improved by 3.7% and 2.7%, respectively. This may be due to the fact that the IA and IF datasets contain less noise and have complementary advantages. The IA dataset contains location and morphological features, and the IF dataset contains boundary texture information of structural stratification and through-cracks.

[0135] The model performed the worst on the three feature fusion datasets. This may be because the three spectral images were directly weighted 1:1:1, which contained more spectral information, but the weighting ratio was not allocated during the weighting process, which also amplified the noise, thus affecting the training and verification results of the model.

[0136] (1b) Comparison and analysis of ablation experimental results within the model

[0137] Table 2. Ablation experiment results of single feature dataset model performance

[0138]

[0139] Table 3. Ablation experimental results of model performance on multi-feature fusion dataset

[0140]

[0141] In (1a), the impact of different data sets on the performance of the model was verified, and the improved module of the designed two-stage network structure based on Swin Transformer optimization was verified. Four improvements were made to the two-stage network structure: deepening the feature extraction network and introducing the Swin Transformer deep learning network; adding the FPN structure and fusing the feature maps from top to bottom; using ROIAlign to replace the original ROI pooling method; and finally, using Smooth L1 as the bounding box loss function during training. The improvements of each module in the model were verified on different data sets, and the experimental results of the mAP@0.5 index are shown in Tables 2 and 3.

[0142] This embodiment designs five groups of ablation experiments, including verification of 8 different data sets. Taking the Origin data set as a reference, the FasterRCNN network has significantly improved the indicators on all data sets after replacing the backbone network with SwinTransformer, and mAP@0.5 has increased by 8%, which shows that Swin Transformer can effectively improve the feature extraction ability of the model; then after adding the FPN structure, the indicators of the model have been significantly improved, and mAP@0.5 has increased by about 2%; on this basis, using the ROIAlign pooling method, the network performance has also been improved by about 2%; finally, the bounding box loss function is changed to Smooth L1, and the accuracy has increased slightly by less than 1%; finally, the two-stage network optimized based on Swin Transformer has improved about 13% on mAP@0.5 compared with the original classic two-stage network Faster R-CNN. Experiments have shown that for each data set, the four improvement strategies introduced in step 3 of the technical solution have improved the accuracy of the model to varying degrees, and the superposition of the improvement effect is more obvious, which effectively improves the accuracy of through crack identification.

[0143] (1c) Comparison and analysis of experimental results between models

[0144] The original signal data set will be used as a comparison benchmark, and the two-stage detection model of through cracks based on Swin Transformer optimization will be compared with the following benchmark networks: Faster R-CNN, SSD, and YOLOv5 proposed in 2021. The reason for the comparison is: Faster R-CNN is a classic two-stage detection network, SSD and YOLOv5 are one-stage networks, and YOLOv5 also uses the FPN structure. Comparison with these networks can highlight the advantages of the two-stage network optimized based on Swin Transformer. In addition, the number of floating point operations (FLOPs) and parameters (parameters) have been added, as well as the model file size (Model Size) after training. FLOPs refers to the number of floating point operations per second of the computer, which can measure the computational complexity of the model. The number of parameters parameters refers to the sum of the parameters of all modules in the model, which can measure the size of the model. For example, a 3×3 convolution layer contains 9 parameters in the convolution kernel and a bias parameter. ModelSize is the space it occupies on the hard disk.

[0145] Table 4 Comparison of detection results between models

[0146]

[0147] As can be seen from Table 4, the road through crack detection model proposed in step 3 of the technical solution is superior to other algorithms in terms of mAP@0.5 indicators. Compared with FasterR-CNN, the accuracy is about 13% higher; compared with YOLOv5 and SSD with one-stage network structure, the mAP@0.5 is about 11% and 15% higher respectively; compared with the classic network, the improved network in step 3 of the technical solution has a significant improvement in detection accuracy, which shows the effectiveness of the improvement strategy introduced in step 3 of the technical solution.

[0148] Example 2: One-stage through-crack detection model based on Swin Transformer-YOLOv8 optimization

[0149] (2a) Comparison and analysis of experimental results on different datasets

[0150] In order to compare the performance of the multi-feature fusion dataset in the one-stage detection network, the original signal and seven other feature datasets were trained and tested. Through different comparative experiments, the role of different datasets in improving model performance was evaluated. Among the eight datasets, the IA+IF training set performed outstandingly, and its indicators during the training process changed steadily and had a good trend, such as Fig. 9 As shown in the figure, both the learning potential and generalization ability are better than the original signal dataset, the single feature dataset and the other three fusion datasets.

[0151] Corresponding to the training process, the PR curve of the dataset also shows high precision and recall, such as Fig.10 As shown in the figure, it is further verified that IA+IF has a good application effect on the designed one-stage detection model of through cracks.

[0152] In addition, in order to provide more comprehensive and detailed data support, the detection accuracy data of more data sets are collated and presented in Tables 5 and 6. These tables record in detail the performance of each data set under different evaluation indicators.

[0153] Table 5 Experimental results of mAP@0.5 (%) on single feature dataset

[0154]

[0155] Table 6 Experimental results of feature fusion dataset mAP@0.5 (%)

[0156]

[0157]

[0158] After a detailed analysis of the test results, it was found that for the one-stage target detection network, the combination of IA+IF in the multi-feature fusion dataset performed particularly well in the eight datasets, with a detection accuracy of 86.2%, slightly higher than the original signal dataset by 2.4%. This result further confirms the positive role of the multi-feature fusion process in enhancing crack features, thereby significantly improving the automatic recognition ability of the model. However, the performance of the IA+IP+IF dataset in the one-stage detection network did not bring about performance improvement, and it was also the worst performing fusion dataset. This shows that in the data fusion process, the impact of noise interference cannot be ignored, and excessive or improper fusion may introduce negative effects, thereby affecting the performance of the model. Therefore, when performing feature fusion, it is necessary to select feature combinations to make full use of the complementarity between features and minimize noise interference.

[0159] (2b) Comparison and analysis of ablation experimental results within the model

[0160] At the same time, in order to verify the improvement brought by the two improvement strategies of YOLOv8 and Swin Transformer fusion and BiFPN module to the one-stage through crack detection network model, three groups of ablation experiments were designed, and the two parts of the model improvement were evaluated and analyzed with detection accuracy as the indicator. The ablation experiment results of detection accuracy are also shown in Tables 5 and 6. Fig.11 , 12 It better reflects the changing trend of detection accuracy.

[0161] From Tables 5, 6 and Fig.11 , 12 It can be seen that when YOLOv8 is not optimized, the mAP@0.5 reaches 71.6%. This shows the effectiveness of YOLOv8 as a basic model in the task of through crack detection. Then, when the SwinTransformer structure is introduced and fused with YOLOv8, it can be observed that the accuracy of the optimized model on all data sets has improved. In particular, on the Origin data set, the accuracy has increased by about 8%. This improvement shows that by fusing a complex network structure such as SwinTransformer, the model's ability to automatically extract features has been enhanced. The introduction of SwinTransformer enhances the model's ability to perceive targets of different scales and deformations in the image, thereby improving detection accuracy.

[0162] Furthermore, based on the Swin Transformer-YOLOv8 model, the BiFPN structure was introduced in the Neck part, which improved the accuracy by about 4%. The effectiveness of the BiFPN structure in improving the model accuracy was verified.

[0163] Fig.13 The detection effects of YOLOv8 and SwinTransformer-YOLOv8 that integrates two improved strategies are compared. It can be seen from the figure that for some cracks with unclear features, the optimized Swin Transformer-YOLOv8 can detect them while the original YOLOv8 cannot. This is because the BiFPN structure introduced in the Neck part performs weighted fusion of feature maps of different layers during feature fusion, so that the network can extract some unclear features of the original image when extracting features.

[0164] (2c) Comparison and analysis of experimental results between models

[0165] Some one-stage and two-stage target detection models are selected to be reproduced on the multi-feature fusion dataset of ground penetrating radar images, such as YOLOv7, Faster R-CNN, and the two-stage network based on SwinTransformer optimization proposed in step 3 of the technical solution. In addition to the indicators proposed above, FPS is also introduced, which measures the model detection speed, that is, the number of images that the model can detect within one second (in frames / s). The comparison results on the through-crack original signal dataset are shown in Table 7.

[0166] Table 7 Comparison of model detection results

[0167]

[0168] After comparing with the two-stage detection model proposed in step 3 of the technical solution and other networks, the YOLOv8 one-stage detection network based on Swin Transformer fusion introduced in step 4 showed excellent performance in many key indicators. Specifically, Model Size, Params, and FLOPs, the three indicators that measure model complexity and computational burden, all ranked first among the compared models, which fully proves the remarkable effectiveness of the Swin Transformer-YOLOv8 optimization network in alleviating device storage pressure and reducing computational complexity.

[0169] From the perspective of detection accuracy, although it is slightly lower than the two-stage detection model optimized by SwinTransformer in step 3 of the technical solution by about 1%, in practical applications, such an accuracy gap is usually acceptable. More importantly, the FPS (frames per second) of the one-stage detection network has been increased from 10.37 frames / s to 35.71 frames / s. Since the ground penetrating radar test image used corresponds to 12cm in the actual road, it can be calculated that the detection speed of the model can reach 4.2m / s, that is, the detection speed is 15.42km / h, which is an increase of about 10km / h, and the detection speed has been significantly improved.

[0170] In addition, although the FPS of the Swin Transformer-YOLOv8 optimized network is about 1 frame / s slower than that of the YOLOv7 network, it has achieved a 2.6% improvement in accuracy. The trade-off between accuracy and speed is of great significance in practical applications, especially in scenarios that have certain requirements for accuracy and real-time performance.

[0171] In summary, the one-stage detection network optimized by Swin Transformer-YOLOv8 maintains high detection accuracy while greatly optimizing the complexity and computational complexity of the model, making it more efficient and stable when handling actual detection tasks.

[0172] (2d) Model applicability analysis

[0173] The two-stage through-crack detection based on Swin Transformer optimization designed in step 3 of the technical solution has good accuracy, but the FPS of the model for detecting and identifying through-crack images is only 10.7 frames per second, that is, only 10 images can be detected per second on average, and the detection rate can only reach 4.6km / h. Therefore, this two-stage through-crack detection based on Swin Transformer optimization has high accuracy but low detection rate, and is suitable for scenarios where key areas are detected.

[0174] The Swin Transformer-YOLOv8 optimized one-stage through crack detection model designed in step 4 of the technical solution has a detection rate of 37 images per second, an FPS increase of 25.34 frames, and a detection speed of 15.42km / h, while the mAP@0.5 is only about 1% lower than the detection model in step 3 of the technical solution. Therefore, the Swin Transformer-YOLOv8 optimized one-stage through crack detection model is suitable for certain general detection scenarios, that is, scenarios with a wide detection range, short detection time, and real-time detection while ensuring a certain detection accuracy.

[0175] The present invention introduces three optimization strategies based on the two-stage detection network of SwinTransformer. First, the FPN structure is combined to make full use of the position information of the underlying feature map to improve the feature utilization of the network; secondly, in order to avoid the loss of some small crack features when ROIpooling uses the nearest neighbor interpolation, the pooling method in the network is improved to ROIAlign using bilinear interpolation to improve the feature extraction ability of small cracks; finally, in order to prevent the gradient explosion problem during the training process, the bounding box loss function uses Smooth L1.

[0176] In addition, the present invention optimizes Swin Transformer-YOLOv8 to improve the detection speed so that it meets the requirements of rapid detection of road cracks. The YOLOv8 backbone network integrates Swin Transformer, and then introduces BiFPN to adjust the contribution of different feature layers. The training test results of the Swin Transformer-YOLOv8 optimized first-stage network on different feature data sets are compared, which further proves that feature fusion can enhance crack features, and the performance is best on IA+IF, reaching 86.2%. Then, the ablation experiment of the model structure proves that the detection accuracy of the optimized Swin Transformer-YOLOv8 network has been effectively improved, reaching 83.9% on the original signal data set. And compared with the network optimized by the two-stage detection network based on Swin Transformer and other networks, it is found that its various indicators meet the design concept. While maintaining a certain degree of accuracy, the detection speed FPS reaches 35.71 frames / s, which lays the foundation for real-time detection.

[0177] The above description is only a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any slight modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A lightweight crack penetration detection method based on ground penetrating radar and Swin Transformer optimization, characterized in that: The steps include: Step 1: Collect ground penetrating radar data and perform preprocessing and feature extraction Use ground penetrating radar equipment to scan the asphalt pavement and collect radar signal data below the pavement; process the raw radar data; extract key information that helps crack detection from the pre-processed data; Step 2: construct a multi-feature fusion data set; The instantaneous amplitude, instantaneous phase and instantaneous frequency characteristic images of the ground penetrating radar signal are combined to form a multi-feature fusion data set; then the data set is expanded and annotated to complete the construction of the multi-feature fusion data set of the ground penetrating radar through cracks; Step 3: Design a two-stage through crack detection model based on Swin Transformer optimization (3a) Optimizing the overall framework of the network Swin Transformer is introduced into the backbone network, and FPN network is introduced after the backbone network. Combining the advantages of Mask R-CNN, ROI pooling is replaced with ROIAlign to improve the two-stage detection network. (3b) FPN multi-scale fusion module design Through the top-down feature extraction network, feature maps of different depths and scales are generated. The resolution of the deepest feature map is dynamically adjusted using upsampling. The feature maps of two adjacent layers with different scales are fused. The fused feature maps containing the shallow network position information and the deep network semantic information are used for target detection. (3c) ROI pooling optimization After calculating the values ​​of the four positions by bilinear interpolation, ROI outputs of the same size are obtained by Maxpooling; (3d) Smooth L1 loss function optimization The bounding box position loss function uses the Smooth L1 function, which combines the L1 loss function and the L2 loss function; Step 4: Design a one-stage through crack detection model based on Swin Transformer-YOLOv8 optimization (4a) Fusion Swin Transformer Network Fuse YOLOv8 and Swin Transformer, and introduce the Swin Transformer network into the Backbone part of YOLOv8; (4b) Weighted bidirectional feature fusion network First, the deep features are continuously upsampled and fused with the bottom features. After the upsampling is completed, the bottom features are downsampled and fused with the deep features. Two horizontal connection paths are added between the two feature extraction paths to fuse the feature maps generated at each stage of the backbone network with the feature maps to be tested. (4c) Overall architecture of the Swin Transformer-YOLOv8 model YOLOv8 integrates Swin Transformer and introduces weighted bidirectional feature pyramid (BiFPN); Step 5: Train and predict the training set and test set data according to the model architecture in steps 3 and 4.

2. The lightweight crack penetration degree detection method based on ground penetrating radar and Swin Transformer optimization according to claim 1 is characterized in that: In step 1, the ground penetrating radar data includes detailed information of the pavement structure, especially the location and depth of cracks; Preprocessing includes zero bias correction, FIR filtering, and signal gain adjustment processing; Feature extraction includes amplitude information, frequency information and phase information.

3. The lightweight crack penetration degree detection method based on ground penetrating radar and SwinTransformer optimization according to claim 1 is characterized in that: In step 2, the data set is expanded using random cropping, mirror flipping, and contrast enhancement methods, and finally the eight data sets are labeled to complete the construction of the multi-feature fusion data set of ground penetrating radar through-cracks.

4. The lightweight crack penetration degree detection method based on ground penetrating radar and SwinTransformer optimization according to claim 1 is characterized in that: In step 3, the L1 and L2 loss functions are as follows: Where f(x i ) and y i They represent the predicted value and the corresponding true value of the i-th sample respectively, and n is the number of samples.

5. The lightweight crack penetration degree detection method based on ground penetrating radar and Swin Transformer optimization according to claim 1 is characterized in that: In step 3, the formula of Smooth L1 is as follows: This function is a piecewise function. Between [-1, 1], it is L2 loss, which solves the problem that L1 has a breaking point at 0. Outside the interval [-1, 1], it is L1 loss, which solves the problem of gradient explosion of outliers. Therefore, the gradient can be limited from the following two aspects: first, when the error between the predicted value and the true value is too large, the gradient value will not be too large; second, when the error between the predicted value and the true value is very small, the gradient value is small enough.

6. The lightweight crack penetration degree detection method based on ground penetrating radar and SwinTransformer optimization according to claim 1 is characterized in that: The specific method adopted in step 5 is: The through-fracture multi-feature dataset constructed in step 2 is used; In step 3, the SGD optimizer is used as the basic iterator, the weight_decay factor is 0.0001, the momentum factor is 0.9, and a single GPU is used for training. The batch size is set to 4 in both the original network and the improved network, and the corresponding initial learning rate is set to 0.

001. During the training process, the model first loads the initialization weight file of the SwinTransformer trained on the public dataset ImageNet, and the total number of training iterations is set to 100. In step 4, the SGD optimizer is used as the basic iterator, the initial learning rate is set to 0.001, the weight_decay weight decay factor is 0.0005, and the momentum factor is 0.937; a single GPU is used for training, the batch size is set to 4, and the total number of training iterations is set to 100 rounds considering the comprehensive training speed and the original image size; Step 3 and step 4 use accuracy, average precision, mean average precision, and PR curve as evaluation indicators to measure the performance of the model; In step three, the original signal, single feature and multi-feature fusion data sets are trained using a two-stage network structure optimized based on SwinTransformer.

7. The lightweight crack penetration degree detection method based on ground penetrating radar and SwinTransformer optimization according to claim 6 is characterized in that: The accuracy rate indicates the proportion of correct detections in all model detection results, and the recall rate indicates the proportion of correct detections in all real annotation frames, as shown in the following formula: Among them, TP means that the model is correctly identified, that is, the number of cracks correctly detected; FP means that the model is incorrectly identified, that is, the number of cracks falsely detected; FN means that positive samples are judged as negative samples, that is, the number of cracks missed; P and R can both measure the detection effect of the model in some scenarios, but these two indicators are inversely proportional, so AP is needed to comprehensively evaluate the generalization of the model using P and R indicators.

8. The lightweight crack penetration degree detection method based on ground penetrating radar and SwinTransformer optimization according to claim 6 is characterized in that: The average precision (AP) value is calculated by the area enclosed by the PR curve and the coordinate axis. The larger the area of ​​the PR curve, the better the model performance. The formula is as follows:

9. The lightweight crack penetration degree detection method based on ground penetrating radar and SwinTransformer optimization according to claim 6 is characterized in that: The mean average precision (mAP) is the best evaluation indicator to demonstrate the performance of the target detection model. mAP calculates the average value of AP of all categories in the data set. The calculation formula is as follows: Where n represents all categories in the dataset, and mAP mainly includes three evaluation criteria: mAP@0.5, mAP@0.75, and mAP@0.5:0.95; mAP@0.5 and mAP@0.75 refer to the mAP values ​​when IoU=0.5 and IoU=0.75 respectively. mAP@"0.5:0.95" refers to the average value of the mAP value corresponding to each IoU when IoU changes from 0.5 to 0.95 with a step size of 0.05.

Images