Road surface defect detection algorithm based on improved YOLOv8

Abstract

The invention belongs to the technical field of deep learning target detection, and provides a road surface defect detection algorithm based on improved YOLOv8. The problems of complex background, small disease targets, difference and similarity among types and the like exist in pavement defect detection tasks, and meanwhile, a road pavement defect detection model with balanced detection precision and lightweight is designed for reducing the limitation of the model on hardware resource conditions. The method comprises the following steps: firstly, introducing a CA attention mechanism into a backbone network of a target detection algorithm YOLOv8 so as to integrate position information into channel attention; secondly, deformable convolution is introduced to adapt to the problem that the shape, the size, the position, the direction and the like of a target object are inconsistent, and the disease feature extraction capacity of the network is improved; wIoU is adopted as a loss function in the prediction frame regression process, and the convergence speed and regression precision of the model are improved. The improved model not only can improve the image detection precision, but also can reduce the limitation of the model on hardware conditions.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning target detection, and proposes a road pavement defect detection algorithm based on improved YOLOv8. In the road defect detection task, there are problems such as complex background, small disease targets, differences and similarities between types, and at the same time, in order to reduce the restrictions of the model on hardware resource conditions, a road pavement defect detection model with balanced detection accuracy and lightweight is designed. Firstly, the CA attention mechanism is introduced into the backbone network of the target detection algorithm YOLOv8, aiming to integrate the position information into the channel attention. Secondly, deformable convolution is introduced to adapt to the inconsistency of the shape, size, position and direction of the target object, and increase the network's ability to extract disease features. WIoU is used as the loss function in the prediction box regression process to improve the convergence speed and regression accuracy of the model. The improved model can not only improve the detection accuracy of the image, but also reduce the restrictions of the model on hardware conditions. Background Art

[0002] With the rapid development of highway construction in my country, the volume of highway transportation has maintained a high-speed increase. Through the use of roads over the years, as well as the impact of severe climate and natural disasters, various diseases will appear on the road surface. Its various hidden dangers will have a great impact on the stability of the road pavement structure, resulting in reduced pavement bearing capacity and durability, which can easily cause vehicle performance degradation, and ultimately affect driving safety. If not repaired in time, the degree of road damage will increase, resulting in higher road maintenance costs in the later stage, causing additional economic losses. Road maintenance is a way to prevent road damage and an important way to increase the service life of roads. Therefore, it is necessary to improve road maintenance management, which can not only reduce the occurrence of traffic accidents, but also reduce unnecessary economic losses. As a prerequisite for highway maintenance, road pavement defect detection has a relatively small impact on highway transportation in the early stage of pavement disease formation. Therefore, effective methods are needed to identify road pavement defects.

[0003] At present, the YOLO series of algorithms based on deep learning have attracted widespread attention due to their high accuracy and robustness. The YOLO algorithm achieves direct regression and classification prediction of the prediction box through its unique network structure design, thus avoiding the step of candidate region extraction in traditional target detection algorithms. Therefore, this algorithm is more suitable for real-time application. Summary of the invention

[0004] The present invention aims to solve the problems of the current YOLOv8 used in road surface defect target detection, such as the difficulty in detection due to dense targets, numerous small targets, occlusion, etc., and the lack of real-time performance due to the complexity of the backbone network. It is intended to study a road surface defect detection algorithm based on improved YOLOv8, and to optimize the structure of the backbone network to make it better. Therefore, a road surface defect detection algorithm based on improved YOLOv8 is proposed, using the improved YOLOv8 algorithm.

[0005] In order to achieve the above object, the technical solution of the present invention is as follows: A road pavement defect detection algorithm based on improved YOLOv8 includes the following steps: Step 1: Introduce the CA attention mechanism into the backbone network of the target detection algorithm YOLOv8, which not only considers channel information but also direction-related position information. It is lightweight and flexible enough to optimize the shortcomings of SE and CBAM attention. Step 2: Introduce deformable convolution into the C2f structure to form the C2f_DCN module, which can better adapt to road damage targets of different scales and shapes, thereby improving the accuracy of road damage targets and strengthening the feature extraction of damages; Step 3: Using WIoU as the loss function in the prediction box regression process can fully consider the angle between the prediction box and the true box, improve the convergence speed and regression accuracy of the model, and make the whole regression process pay more attention to high-quality anchor boxes.

[0006] The specific situation in the step 1 is as follows.

[0007] The open source RDD2022 dataset, which was taken with a car-mounted smartphone, contains four types of defects: longitudinal cracks (D00), transverse cracks (D10), reticular cracks (D20), and potholes (D40). The samples in this dataset are consistent with the actual situation of defect images in real road pavement scenes and have complex background problems. Using this dataset to train the model can better improve the model's effectiveness in detecting road pavement defects in complex environments.

[0008] In this step, after the CA module is inserted into the 2nd, 3rd, and 4th C2f layers of the backbone part of the YOLOv8 model, the CA attention module not only considers the channel information, but also the direction-related position information, and is lightweight and flexible enough to be easily inserted into the backbone feature extraction network of each network. The specific steps of CA attention are: First, in order to avoid the loss of spatial position information, global average pooling is not used, but pooling is performed separately in the width and height directions to generate feature maps of C*H*1 and C*1*W sizes respectively. The formula is as follows: ; ; Then, the generated C*1*W feature map is transformed in dimension, and then concat operation is performed with the C*H*1 feature map. After that, the 1*1 convolution kernel is used for dimensionality reduction, and an activation function is added to generate a feature map of size C / r*(H+W)*1. The formula is as follows: ; Finally, the feature map is divided into sum feature maps of sizes C / r*H*1 and C / r*1*W (after dimension transformation), and then the 1*1 convolution kernel is used to increase the dimension, and an activation function is added to generate the sum of feature maps of sizes C*H*1 and C*1*W. These two feature maps are expanded and multiplied with the input feature map to get the final output. The mathematical formula is as follows: ; ;; .

[0009] The specific situation in the step 2 is as follows: Deformable convolution is introduced into the C2f structure to form the C2f_DCN module. The DCNv2 convolution can dynamically adjust the sampling position of the convolution kernel by learning the offset parameter, thereby capturing more complex deformation information. Deformable convolution introduces offset and control point offset based on the original convolution. The offset is used to adjust the sampling position of the convolution kernel on the input feature map, while the control point offset is used to make more precise adjustments to the sampling position. By learning the offset parameter, the deformable convolution can dynamically adjust the sampling position of the convolution kernel to capture more complex deformation information. The mathematical formula is as follows: ; Given The convolution kernel of sampling positions is and Respectively represent The weights of the positions and the pre-specified offsets. Respectively represent The position is the learned offset, are the learned weights.

[0010] The specific situation in step three is as follows: The WIoU loss calculation method used is as follows: ; , ; in, , Represent the center of the predicted box and the true box respectively, represents the Euclidean distance between two centers, is the diagonal distance between the predicted box and the minimum enclosed area of ​​the true box, is the outlier degree, and is a hyperparameter. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 Schematic diagram of the improved YOLOv8 model used in the present invention; Figure 2 This is the structural diagram of the CA attention mechanism used in the present invention; Figure 3 This is a structural diagram of the C2f_DCN module used in the present invention; DETAILED DESCRIPTION

[0012] For those skilled in the art, some well-known structures and their descriptions in the drawings may be omitted. The technical solution of the present invention is further described below in conjunction with the drawings and implementation methods.

[0013] The present invention aims to solve the problems of complex background, small defect targets, differences and similarities between types, etc. in the current YOLOv8 target detection of road defects, and proposes a road pavement defect detection algorithm based on improved YOLOv8.

[0014] First, the CA attention mechanism is introduced into the backbone network of the target detection algorithm YOLOv8, aiming to integrate the position information into the channel attention. The coordinate attention mechanism can more accurately capture the spatial distribution characteristics in the image by calculating the attention separately in two spatial dimensions (height and width), thereby more comprehensively capturing the dependencies between features. The deformable convolution is introduced into the C2f structure to form the C2f_DCN module, which can better adapt to road disease targets of different scales and shapes, thereby improving the accuracy of road disease targets and strengthening the feature extraction of diseases. At the same time, WIoU is used as the loss function in the prediction box regression process, which can fully consider the angle effect between the prediction box and the real box, and improve the convergence speed and regression accuracy of the model, so that more attention is paid to high-quality anchor boxes in the entire regression process.

[0015] The specific implementation steps are: Step 1.1 Choose to use the open source RDD2022 dataset shot by a car-mounted smartphone; Step 1.2 Add CA module to the original model backbone network; Step 1.3 First, perform global average pooling on the input feature map in the H and W directions respectively. H represents the height and W represents the width. The feature maps in the H and W directions are obtained respectively. The specific formula is as follows: ; ; In the formula, and To perform pooling operations on the H and W directions respectively, and Respectively represent the total number of pixels occupied by its direction; Step 1.4 The feature maps of the H and W directions that have obtained the global receptive field information are spliced ​​together, and then they are sent to the convolution module with a convolution kernel of 1 to reduce their dimension. After batch normalization, they are sent to the Sigmoid activation function to obtain the feature map, as shown in the formula below: ; In the formula, is the batch normalization function, is the sigmoid function, and Output the result for Step 1.3; Step 1.5 Perform convolution on the above feature map f according to the original values ​​of H and W with a convolution kernel of 1×1 to obtain feature maps and respectively, and then use the Sigmoid activation function to obtain the weights and feature maps of H and W respectively. The formula is as follows: ; ; In the formula, and is a 1×1 convolution operation. is the sigmoid function, and This is the output result of Step 1.4; Step 1.6 After the above steps, the weight feature vectors of the input feature map in H and W will be obtained. and , the original input vector By weighting the feature vector and Multiplication weighted calculation to obtain vectors with attention weights in the H and W directions , the formula is as follows: ;; In the formula, is the original input vector, and is the weight feature vector, is the final output vector with attention weights.

[0016] Step 2.1 introduces deformable convolution into the C2f structure to form the C2f_DCN module; Step 2.2 DCNv2 convolution can dynamically adjust the sampling position of the convolution kernel by learning the offset parameter, thereby capturing more complex deformation information. Deformable convolution introduces offset and control point offset based on the original convolution. The offset is used to adjust the sampling position of the convolution kernel on the input feature map, while the control point offset is used to make more detailed adjustments to the sampling position. By learning the offset parameter, the deformable convolution can dynamically adjust the sampling position of the convolution kernel to capture more complex deformation information. The mathematical formula is as follows: ; Given The convolution kernel of sampling positions is and Respectively represent The weights of the positions and the pre-specified offsets. Indicates The position is the learned offset, are the learned weights.

[0017] Step 3.1 WIoU uses a dynamic non-monotonic mechanism to evaluate the quality of anchor boxes, which makes the model pay more attention to anchoring boxes of ordinary quality and improves the model's ability to locate objects. For target detection tasks in drone aerial photography scenes, small objects with a high proportion increase the difficulty of detection. The loss can be dynamically optimized to weight small objects to improve the detection performance of the model; Step 3.2 The WIoU loss calculation method used is as shown in the formula: ; , ; in, , Represent the center of the predicted box and the true box respectively, represents the Euclidean distance between two centers, is the diagonal distance between the predicted box and the minimum enclosed area of ​​the true box, is the outlier degree, and is a hyperparameter.

Claims

1. A road surface defect detection algorithm based on improved YOLOv8, comprising the following steps: Step 1: Improve the target detection algorithm YOLOv8 and introduce the CA attention mechanism into the backbone network of the target detection algorithm YOLOv8; Step 2: Introduce deformable convolution into the C2f structure to form the C2f_DCN module, which can better adapt to road damage targets of different scales and shapes, thereby improving the accuracy of road damage targets and strengthening the feature extraction of damages; Step 3: Using WIoU as the loss function in the prediction box regression process can fully consider the angle influence between the prediction box and the true box, and improve the convergence speed and regression accuracy of the model, so that the entire regression process pays more attention to high-quality anchor boxes.

2. According to the road pavement defect detection algorithm based on improved YOLOv8 according to claim 1, the specific process in Step 1 is as follows: Step 1.1 Choose to use the open source RDD2022 dataset shot by a car-mounted smartphone; Step 1.2 Add CA module to the original model backbone network; Step 1.3 First, perform global average pooling on the input feature map in the H and W directions respectively. H represents the height and W represents the width. The feature maps in the H and W directions are obtained respectively. The specific formula is as follows: ； ； In the formula, and To perform pooling operations on the H and W directions respectively, and Respectively represent the total number of pixels occupied by its direction; Step 1.4 The feature maps of the H and W directions that have obtained the global receptive field information are spliced ​​together, and then they are sent to the convolution module with a convolution kernel of 1 to reduce their dimension. After batch normalization, they are sent to the Sigmoid activation function to obtain the feature map, as shown in the formula below: ；； In the formula, is the batch normalization function, is the sigmoid function, and Output the result for Step 1.3; Step 1.5 Perform convolution on the above feature map f according to the original values ​​of H and W with a convolution kernel of 1×1 to obtain feature maps and respectively, and then use the Sigmoid activation function to obtain the weights and feature maps of H and W respectively. The formula is as follows: ； ； In the formula, and is a 1×1 convolution operation. is the sigmoid function, and This is the output result of Step 1.4; Step 1.6 After the above steps, the weight feature vectors of the input feature map in H and W will be obtained. and , the original input vector By weighting the feature vector and Multiplication weighted calculation to obtain vectors with attention weights in the H and W directions , the formula is as follows: ； In the formula, is the original input vector, and is the weight feature vector, is the final output vector with attention weights. 3.Step 2.1 introduces deformable convolution into the C2f structure to form the C2f_DCN module; Step 2.2 DCNv2 convolution can dynamically adjust the sampling position of the convolution kernel by learning the offset parameter, thereby capturing more complex deformation information. Deformable convolution introduces offset and control point offset based on the original convolution. The offset is used to adjust the sampling position of the convolution kernel on the input feature map, while the control point offset is used to make more detailed adjustments to the sampling position. By learning the offset parameter, the deformable convolution can dynamically adjust the sampling position of the convolution kernel to capture more complex deformation information. The mathematical formula is as follows: ； Given The convolution kernel of sampling positions is and Respectively represent The weights of the positions and the pre-specified offsets. Indicates The position is the learned offset, are the learned weights. 4.Step3.1 WIoU uses a dynamic non-monotonic mechanism to evaluate the quality of anchor boxes, which makes the model pay more attention to anchoring boxes of ordinary quality and improves the model's ability to locate objects. For target detection tasks in drone aerial photography scenes, small objects with a high proportion increase the difficulty of detection. The loss can be dynamically optimized to weight small objects to improve the detection performance of the model; Step 3.2 The WIoU loss calculation method used is as shown in the formula: ； ， ； in, , Represent the center of the predicted box and the true box respectively, represents the Euclidean distance between two centers, is the diagonal distance between the predicted box and the minimum enclosed area of ​​the true box, is the outlier degree, and is a hyperparameter.

Images