Pavement defect detection method of YOLOv8 network based on DRB lightweight module and lightweight detection head

Abstract

The invention provides a pavement defect detection method of a YOLOv8 network based on a DRB lightweight module and a lightweight detection head, and the method comprises the steps: fusing the DRB lightweight module into a C2f module, replacing a convolution structure in a Bottleneck residual module in C2f, forming a C2fDRB module to form a backbone network of YOLOv8, enabling the backbone network structure to better extract feature information, carrying out the parameterization, and carrying out the detection of the pavement defect of the YOLOv8 network. The convolution receptive field is expanded to adapt to different scale features; and then, improving a detection head of YOLOv8, improving a CBS module in the head with relatively high parameter quantity and calculation quantity, and selecting group convolution and depth separable convolution with relatively low parameter quantity and calculation quantity to replace the CBS. Experiments are carried out on an RDD2020 pavement defect data set, compared with an original YOLOv8 model, the YOLOv8 model proposes an improved network C2fDRB structure, the parameter quantity of the network is reduced by 8.1%, the calculated amount is reduced by 8.4%, the accuracy is improved by 2.1%, the parameter quantity of the network of the improved detection head is reduced by 14.1%, and the calculated amount is reduced by 24%. The overall improved model has a relatively obvious improvement effect. The result shows that the improved model provided by the invention effectively reduces the parameter quantity and the calculation quantity, and the accuracy is also improved.

Description

Technical Field

[0001] The present invention relates to the field of computer image detection, and in particular to a road surface defect detection method based on a YOLOv8 network of a DRB lightweight module and a lightweight detection head. Background Art

[0002] As the scale of roads expands, the problem of road damage becomes increasingly prominent. Due to differences in vehicle load and construction life, these roads face varying degrees of damage. Road damage such as potholes and cracks not only affects road traffic efficiency, but also increases the instability of vehicle driving, increases the probability of traffic accidents, and poses a threat to the safety of vehicles and passengers. Object detection algorithms based on deep learning have gradually become the mainstream algorithms in the detection field. These algorithms use deep learning models such as convolutional neural networks (CNNs) to achieve higher detection accuracy and faster detection speed.

[0003] Existing deep learning models can no longer meet the requirements of detection and real-time road damage detection. Therefore, it is urgent to find a suitable model that can improve detection accuracy and reduce model complexity. The present invention proposes a road defect detection method based on a YOLOv8 network with a DRB lightweight module and a lightweight detection head, which is improved on the basis of the original YOLOv8, so that the improved network has higher detection accuracy, less parameters and less calculation. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a road defect detection method based on a YOLOv8 network of a DRB lightweight module and a lightweight detection head in view of the above-mentioned deficiencies in the prior art, comprising the following steps:

[0005] Step 1: Obtain a pavement disease dataset, process the obtained traffic multi-objective table dataset into a 640×640 image, convert it into a file format suitable for YOLO training, extract the types of defects that need to be detected, and divide it into training set, validation set, and test set.

[0006] Step 2: Integrate the DRB lightweight module into the C2f module, replace the convolution structure in the Bottleneck residual module in C2f, and form the C2f_DRB module to form the backbone network of YOLOv8.

[0007] Step 3: Improve the detection head of YOLOv8 by improving the CBS module in the original head with high parameter and computational complexity, and replace CBS with group convolution and depthwise separable convolution with lower parameter and computational complexity.

[0008] Step 4: Put the road defect detection dataset processed in step 1 into the improved YOLOv8 network, set the appropriate number of training times, and wait until the model's loss, mAP and other indicators are stable and convergent.

[0009] Step 5: Apply the weight file of the model trained in step 4 to the YOLOv8 model. Select the validation set in step 1 for validation to see how the improved model compares to the base model.

[0010] Preferably, the pavement disease dataset is obtained, the acquired traffic multi-object table dataset is processed into a 640×640 size picture, converted into a file format suitable for YOLO training, the defect types to be detected are extracted, and divided into training set, validation set, and test set. Specifically: The RDD2020 dataset contains 26,336 road images from India, Japan, and the Czech Republic, involving more than 31,000 road damage instances. We select four types of pavement defects, namely longitudinal cracks, transverse cracks, crocodile cracks, and potholes for detection. The label text of the RDD2020 dataset is extracted through a python script, and the incomplete labels and other types of pavement defects in the dataset are eliminated, and the images and label texts are divided into training set, test set, and validation set in a ratio of 8:1:1.

[0011] Preferably, the DRB lightweight module is integrated into the C2f module, replacing the convolution structure in the Bottleneck residual module in C2f, and forming the C2f_DRB module to form the backbone network of YOLOv8, specifically: the non-expanded large core convolution layer is enhanced by using an expanded small core convolution layer. These expanded convolution layers are equivalent to a non-expanded convolution layer with a larger sparse core from the perspective of parameters, so that the entire module can be equivalently converted into a single large core convolution layer, thereby maintaining the number of learnable parameters and computational efficiency while enhancing the network's ability to capture spatial information. The DRB lightweight module is used to replace the residual structure in C2f of the YOLOv8 network, and a new residual module named C2f_DRB module is created. The C2f_DRB module is used to form the backbone structure of YOLOv8 to extract the feature map of the input image.

[0012] Preferably, use:

[0013] y=Conv(x,w,r)

[0014] Where w represents the weight, r represents the void rate, x represents the input, and y represents the output.

[0015] Preferably, the detection head of YOLOv8 is improved by improving the CBS module in the original head with high parameter and computational complexity, and group convolution and depthwise separable convolution with lower parameter and computational complexity are selected to replace CBS. Specifically, the improved detection head is composed of grouped convolution and depthwise separable convolution. Grouped convolution is performed by grouping the depth of the convolution kernel and performing convolution operations in segments. Then, depthwise separable convolution is performed, and depthwise separable convolution is divided into two operations: depthwise convolution and pointwise convolution. The parameter calculation formula for depthwise convolution is:

[0016] D k *D k *M

[0017] Where D k is the size of the convolution kernel, and M is the number of convolution kernels.

[0018] On the other hand, in the point-by-point convolution operation, after the depthwise convolution, each channel is convolved using a 1*1 kernel. The parameter quantity is:

[0019] M*N

[0020] N is the number of output channels, and M is the number of convolution kernels.

[0021] This integration helps capture inter-channel correlations and enhances model expressiveness. Therefore, the number of parameters of depth-wise separable convolution is:

[0022] D k *D k *M+M*N

[0023] The parameters of the improved detection head are:

[0024] 4*D k *D k *M*N

[0025] The parameters of the improved detection head are:

[0026] 2*D k *D k *M+M*N+2*D k *D k *M / G*N

[0027] Preferably, the road surface defect detection data set processed in step 1 is set with an appropriate number of training times in the improved YOLOv8 network. When the loss of the model, mAP and other indicators tend to be stable and convergent, specifically: 200 epochs are set for the YOLOv8 network, and the val.py function is called to verify on the validation set data after each training epoch, and mAP_0.5, mAP_0.5:0.95, Precision, Recall, training set loss, validation set loss and other indicator values ​​are marked on Tensorboard until the corresponding curve graph is generated after 200 epochs of training, and the convergence effect is judged by observing the trend of the curve graph after the training is completed.

[0028] Preferably, the same traffic multi-target dataset processed in step 1 is experimented on the basis of the improved YOLOv8 to verify the improvement effect: the two improvements of the C2f_DRB residual structure and the redesign of the detection head are integrated into the YOLOv8 network structure, and an ablation experiment is performed on the RDD2020 dataset processed in step 1 to verify the effects of the improvements in different combinations and the effectiveness of the improvements. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a diagram of the improved YOLOv8 network structure in an example of the present invention;

[0030] Figure 2 It is a DRB structure diagram in an example of the present invention;

[0031] Figure 3 Improve the lightweight detection head of YOLOv8 in the example of the present invention; DETAILED DESCRIPTION

[0032] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0033] like Figure 1 As shown, the embodiment of the present invention provides a road defect detection method based on a YOLOv8 network of a DRB lightweight module and a lightweight detection head, and the specific steps are as follows:

[0034] Step 1: Obtain a pavement disease dataset, process the obtained traffic multi-objective table dataset into a 640×640 image, convert it into a file format suitable for YOLO training, extract the types of defects that need to be detected, and divide it into training set, validation set, and test set.

[0035] The RDD2020 dataset contains 26,336 road images from India, Japan, and the Czech Republic, involving more than 31,000 road damage instances. We selected four types of road surface defects for detection: longitudinal cracks, transverse cracks, alligator cracks, and potholes. We extracted the label text of the RDD2020 dataset through a python script, removed incomplete labels and other types of road surface defects in the dataset, and split the images and label text into training set, test set, and validation set in a ratio of 8:1:1.

[0036] Step 2: Integrate the DRB lightweight module into the C2f module, replace the convolution structure in the Bottleneck residual module in C2f, and form the C2f_DRB module to form the backbone network of YOLOv8.

[0037] like Figure 2 As shown in Figure 2, through the structural reparameterization method, we merge BN layers into convolutional layers so that small convolutional kernels can be equivalently merged into large convolutional kernels at inference time. Structural reparameterization is a model optimization technique in deep learning that aims to improve the efficiency of the model at the inference stage while retaining high performance at the training stage. This technique enhances the learning and generalization ability of the model by using complex model structures at the training stage, and then transforms these complex structures into equivalent, simpler structures at the inference stage, thereby reducing computational and memory overhead and improving inference speed. In this work, we note that in addition to small-scale patterns, enhancing the ability of large convolutional kernels to capture sparse patterns (i.e., a pixel on the feature map may be more correlated with its neighboring pixels than some distant pixels) may produce higher quality features. The need to capture such patterns is exactly matched with the mechanism of dilated convolutions - from the perspective of sliding windows, a dilated convolutional layer with a dilation rate of r scans the input channels to capture spatial patterns, where each pixel of interest is r-1 pixels away from its neighbors. Therefore, we use dilated convolutional layers in parallel with large convolutional kernels and add their outputs. The DRB module utilizes a non-extended small core layer and multiple extended small core layers to enhance a non-extended large core transformation layer.

[0038] To eliminate the inference cost of the additional dilated convolutional layer, we propose to convert the entire block into a single non-dilated convolutional layer for inference. Since ignoring the input pixels is equivalent to inserting additional zeros in the convolution kernel, the dilated convolutional layer with a small kernel can be equivalently converted to a non-dilated (i.e., r=1) layer with a sparse large kernel. Let k be the kernel size of the dilated layer. By inserting zeros, the kernel size of the corresponding non-dilated layer is (k-1)*r+1. We further note that this is a good conversion from the previous kernel W∈R k*k To the latter W1∈R ((k-1)*r+1)*((k-1)*r+1)It can be obtained by a transposed convolution with a stride r and a unit kernel I∈R 1*1 To achieve it elegantly, it is a scalar 1. Print out the network structure parameters. According to the parameter results, after the improved network C2f_DRB structure proposed in this study, the network parameters are reduced by 8.1% and the calculation amount is reduced by 8.4%.

[0039] Step 3: Improve the detection head of YOLOv8 by improving the CBS module in the original head with high parameter and computational complexity, and replace CBS with group convolution and depthwise separable convolution with lower parameter and computational complexity.

[0040] like Figure 3 As shown in the figure, the improved detection head consists of grouped convolution and depthwise separable convolution. Grouped convolution is to group the depth of the convolution kernel and perform convolution operations in segments. Then, depthwise separable convolution is performed, which is divided into two operations: depthwise convolution and pointwise convolution. In depthwise convolution, each channel of the input undergoes an independent convolution operation. By assigning only one convolution kernel to each channel, this method effectively reduces the complexity of the model, mitigates the risk of overfitting, and maintains information independence across channels. The parameter calculation formula for depthwise convolution is:

[0041] D k *D k *M

[0042] Where D k is the size of the convolution kernel, and M is the number of convolution kernels.

[0043] On the other hand, in the point-by-point convolution operation, after the depthwise convolution, each channel is convolved using a 1*1 kernel. The parameter quantity is:

[0044] M*N

[0045] N is the number of output channels, and M is the number of convolution kernels.

[0046] This integration helps capture inter-channel correlations and enhances model expressiveness. Therefore, the number of parameters of depth-wise separable convolution is:

[0047] D k *D k *M+M*N

[0048] The parameters of the improved detection head are:

[0049] 4*D k *D k *M*N

[0050] The parameters of the improved detection head are:

[0051] 2*Dk *D k *M+M*N+2*D k *D k *M / G*N

[0052] By printing out the network structure parameters of YOLOv8, it can be concluded that the number of parameters of the improved detection head network is reduced by 14.1% and the amount of calculation is reduced by 24%.

[0053] Step 4: Put the road defect detection dataset processed in step 1 into the improved YOLOv8 network, set the appropriate number of training times, and wait until the model's loss, mAP and other indicators are stable and convergent.

[0054] Set 200 epochs for the YOLOv8 network. After each epoch of training, the val.py function will be called to verify on the validation set data, and the mAP_0.5, mAP_0.5:0.95, Precision, Recall, training set loss, validation set loss and other indicator values ​​will be marked on Tensorboard until the corresponding curve graph is generated after 200 epochs of training.

[0055] Step 5: Apply the weight file of the model trained in step 4 to the YOLOv8 model. Select the validation set in step 1 for validation to see how the improved model compares to the base model.

[0056] After the YOLOv8 training in step 4 is completed, the best training weight file is generated and loaded into the YOLOv8 network. The validation set data set is imported into the YOLOv8 loaded with the training weight file for verification. At this time, the verification results can be seen in the runs / test folder.

[0057] Based on the YOLOv8 model, the present invention incorporates C2f_DRB. After the detection head is improved, the existing road defect detection model solves the problems of uneven recognition accuracy and large number of parameters. The model has a significant improvement. Compared with the original YOLOv8 model, the improved YOLOv8 model finally proposes an improved network C2f_DRB structure. The network parameters are reduced by 8.1%, the calculation amount is reduced by 8.4%, and the accuracy is improved by 2.1%. The parameters of the improved detection head network are reduced by 14.1%, and the calculation amount is reduced by 24%. The improved model as a whole has a more obvious improvement effect.

[0058] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments disclosed in the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments disclosed in the present invention.

Claims

1. A road defect detection method based on a YOLOv8 network of a DRB lightweight module and a lightweight detection head, characterized in that: The steps are as follows: Step 1: Obtain a pavement disease dataset, process the obtained traffic multi-objective table dataset into a 640×640 image, convert it into a file format suitable for YOLO training, extract the types of defects that need to be detected, and divide it into training set, validation set, and test set. Step 2: Integrate the DRB lightweight module into the C2f module, replace the convolution structure in the Bottleneck residual module in C2f, and form the C2f_DRB module to form the backbone network of YOLOv8. Step 3: Improve the detection head of YOLOv8 by improving the CBS module in the original head with high parameter and computational complexity, and replace CBS with group convolution and depthwise separable convolution with lower parameter and computational complexity. Step 4: Put the road defect detection dataset processed in step 1 into the improved YOLOv8 network, set the appropriate number of training times, and wait until the model's loss, mAP and other indicators are stable and convergent. Step 5: Apply the weight file of the model trained in step 4 to the YOLOv8 model. Select the validation set in step 1 for validation to see how the improved model compares to the base model.

2. According to claim 1, a road defect detection method based on a YOLOv8 network of a DRB lightweight module and a lightweight detection head is characterized in that: The pavement disease dataset is obtained, and the obtained traffic multi-object table dataset is processed into a 640×640 image, converted into a file format suitable for YOLO training, and the types of defects that need to be detected are extracted, and divided into training set, validation set, and test set. Specifically, the RDD2020 dataset contains 26,336 road images from India, Japan, and the Czech Republic, involving more than 31,000 road damage instances. We select four types of pavement defects, namely longitudinal cracks, transverse cracks, crocodile cracks, and potholes for detection. The label text of the RDD2020 dataset is extracted through a python script, and incomplete labels and other types of pavement defects in the dataset are eliminated, and the images and label texts are divided into training set, test set, and validation set in a ratio of 8:1:

1.

3. According to claim 1, a road defect detection method based on a YOLOv8 network of a DRB lightweight module and a lightweight detection head is characterized in that: The method of integrating the DRB lightweight module into the C2f module, replacing the convolution structure in the Bottleneck residual module in C2f, and forming the C2f_DRB module to form the backbone network of YOLOv8 is specifically as follows: the non-expanded large core convolution layer is enhanced by using an expanded small core convolution layer. These expanded convolution layers are equivalent to a non-expanded convolution layer with a larger sparse core from the perspective of parameters, so that the entire module can be equivalently converted into a single large core convolution layer, thereby maintaining the number of learnable parameters and computational efficiency while enhancing the network's ability to capture spatial information. The DRB lightweight module is used to replace the residual structure in C2f of the YOLOv8 network, and a new residual module named C2f_DRB module is created. The C2f_DRB module is used to form the backbone structure of YOLOv8 to extract the feature map of the input image. Preferably, use: y=Conv(x,w,r) Where w represents the weight, r represents the void rate, x represents the input, and y represents the output. According to claim 1, a road defect detection method based on a YOLOv8 network of a DRB lightweight module and a lightweight detection head is characterized in that the detection head of YOLOv8 is improved, the CBS module in the original head with higher parameter and computational complexity is improved, and group convolution and depthwise separable convolution with lower parameter and computational complexity are selected to replace CBS. Specifically, the improved detection head consists of grouped convolution and depthwise separable convolution. Grouped convolution is performed by grouping the depth of the convolution kernel and performing convolution operations in segments. Then, depthwise separable convolution is performed, and the depthwise separable convolution is divided into two operations: depthwise convolution and pointwise convolution. The parameter calculation formula for depthwise convolution is: D k *D k *M Where D k is the size of the convolution kernel, and M is the number of convolution kernels. On the other hand, in the point-by-point convolution operation, after the depthwise convolution, each channel is convolved using a 1*1 kernel. The parameter quantity is: M*N N is the number of output channels, and M is the number of convolution kernels. This integration helps capture inter-channel correlations and enhances model expressiveness. Therefore, the number of parameters of depth-wise separable convolution is: D k *D k *M+M*N The parameters of the improved detection head are: 4*D k *D k *M*N。 The parameters of the improved detection head are: 2*D k *D k *M+M*N+2*D k *D k *M / G*N。 4. According to claim 1, a road defect detection method based on a YOLOv8 network of a DRB lightweight module and a lightweight detection head is characterized in that: The road surface defect detection dataset processed in step 1 is trained in the improved YOLOv8 network with an appropriate number of training times. When the model loss, mAP and other indicators are close to stability and convergence, the YOLOv8 network is trained for 200 epochs. After each epoch, the val.py function is called to perform verification on the validation set data, and mAP_0.5, mAP_0.5:0.95, Precision, Recall, training set loss, validation set loss and other indicator values ​​are marked on Tensorboard until the corresponding curve graph is generated after 200 epochs of training. The convergence effect can be judged by observing the trend of the curve graph after training.

5. According to claim 2, a road defect detection method based on a YOLOv8 network of a DRB lightweight module and a lightweight detection head is characterized in that: The road surface defect detection dataset processed in step 1 is trained in the improved YOLOv8 network with an appropriate number of training times. When the model loss, mAP and other indicators are close to stability and convergence, the YOLOv8 network is trained for 200 epochs. After each epoch, the val.py function is called to perform verification on the validation set data, and mAP_0.5, mAP_0.5:0.95, Precision, Recall, training set loss, validation set loss and other indicator values ​​are marked on Tensorboard until the corresponding curve graph is generated after 200 epochs of training. The convergence effect can be judged by observing the trend of the curve graph after training.

Images