Pavement damage detection method based on improved YOLOv8 model

Abstract

The present invention discloses a road surface disease detection method based on an improved YOLOv8 model. First, a road surface disease image data set is obtained and divided into a training set, a validation set and a test set; then an improved YOLOv8 model introducing a deformable convolution and a receptive field attention mechanism is constructed; then the improved YOLOv8 model is trained based on the training set, the validation set and the test set to obtain a trained improved YOLOv8 model; finally, the road surface disease image to be detected is input into the trained improved YOLOv8 model for recognition to obtain a detection result. The present invention introduces a deformable convolution, which can adaptively adjust to adapt to irregular disease shapes and better capture local features; at the same time, a receptive field attention mechanism is added, which can enhance the receptive field of the neural network, adapt to road surface disease detection of different scales and overcome the interference of complex backgrounds.

Description

Technical Field

[0001] The present invention relates to the technical field of information perception and recognition, and in particular to a road surface disease detection method based on an improved YOLOv8 model. Background Art

[0002] my country's highway construction has developed rapidly. As of the end of 2022, the total mileage of highways open to traffic has reached 5.3548 million kilometers. As my country's highway mileage construction continues to improve, the demand for highway maintenance services is also growing. Most of my country's highways are asphalt pavements. Due to their own shortcomings such as large voids, poor temperature stability, and poor aging resistance, they are very prone to cracks, looseness and other diseases. Concrete pavements will cause road diseases due to excessive traffic loads and harsh environments, such as road waterlogging. These pavement diseases affect the appearance of the highway, reduce the comfort of drivers, reduce the carrying capacity and transportation capacity of the highway, and reduce the service life of the highway. Severe pavement diseases may even threaten the driving safety of passenger and freight, cause traffic accidents, and cause serious casualties and economic losses. Therefore, the rapid, accurate and effective detection of pavement diseases is crucial to ensure the safety of road transportation.

[0003] Traditional manual inspection methods rely on inspectors to directly observe road surface defects using human eyes or small auxiliary equipment. This method has great safety hazards, incomplete and inaccurate data collection, and the detection results are easily affected by the subjective influence of the inspectors. The detection efficiency is low. Because the total mileage of roads is long, the labor cost of manual inspection is also high. Closing the road during inspection will cause traffic jams and affect traffic. Vision-based road surface disease detection uses driving recorders, vehicle-mounted cameras, drones and other equipment to collect road images for disease detection. It is low-cost and has the characteristics of non-destructive, high precision and high efficiency. It has been widely used in recent years, but in actual work, the rapid and accurate identification of road surface diseases is still a difficult problem. The one-stage target detection network is superior to the two-stage target detection network in detection speed, so it is widely used in fields with high real-time requirements such as industrial inspection. In terms of one-stage networks, the YOLO series models have always attracted widespread attention, but the latest YOLO series model YOLOv8 is still less studied in road surface detection.

[0004] Therefore, the current technical problem that needs to be solved urgently is how to optimize the YOLOv8 model while leveraging its basic network structure to make it suitable for road disease detection and improve detection speed and accuracy. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a pavement defect detection method based on an improved YOLOv8 model in response to the defects involved in the background technology, which can effectively solve the problems of irregular defect shapes, large differences in defect sizes, weak defect information and complex road background interference, resulting in the original YOLOv8 detection network being unable to fully learn the relevant characteristics of the defects and thus having a low detection accuracy.

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] The pavement disease detection method based on the improved YOLOv8 model includes the following steps:

[0008] Step 1), obtaining a pavement disease image dataset and dividing it into a training set, a validation set, and a test set;

[0009] Step 2), build an improved YOLOv8 model that introduces deformable convolution and receptive field attention mechanism;

[0010] Step 2.1), build the C2f_dcnv2 module;

[0011] The C2f_dcnv2 module includes the first to second Conv modules, the Split module, and the first to third Bottleneck_dcnv2 modules;

[0012] The first to third Bottleneck_dcnv2 modules have the same structure, and all include a third Conv module, a deformable convolution layer DCNv2, a batch normalization layer, and a SiLU activation function layer connected in sequence, wherein the input end of the third Conv module is connected to the external input, and the input end of the third Conv module and the output end of the SiLU activation function layer are added as output;

[0013] The first Conv module, the Split module, the first Bottleneck_dcnv2 module, the second Bottleneck_dcnv2 module, and the third Bottleneck_dcnv2 module are connected in sequence, the input end of the first Conv module is used as the input end of the C2f_dcnv2 module, the output end of the Split module, the output end of the first Bottleneck_dcnv2 module, the output end of the second Bottleneck_dcnv2 module, and the output end of the third Bottleneck_dcnv2 module are spliced ​​according to channels and input to the input end of the second Conv module, and the output end of the second Conv module is used as the output end of the C2f_dcnv2 module;

[0014] Step 2.2), for the head network of the YOLOv8 model, replace all C2f modules with C2f_dcnv2 modules to obtain the YOLOv8 model with deformable convolution;

[0015] Step 2.3), for the backbone network of the YOLOv8 model that introduces deformable convolution, replace all Conv modules except the first Conv module with the receptive field attention convolution module RFAConv to obtain an improved YOLOv8 model;

[0016] Step 3), training the improved YOLOv8 model based on the training set, the validation set and the test set to obtain a trained improved YOLOv8 model;

[0017] Step 4), input the road surface disease image to be detected into the trained improved YOLOv8 model for recognition to obtain the detection result.

[0018] As a further optimization scheme of the pavement disease detection method based on the improved YOLOv8 model of the present invention, the images in the pavement disease image dataset in step 1) are all annotated with pavement disease frame types using labelme annotation software, and the annotation information includes the disease category and coordinates.

[0019] As a further optimization scheme of the pavement damage detection method based on the improved YOLOv8 model of the present invention, the pavement damage image dataset in step 1) is divided into a training set, a validation set and a test set in a ratio of 8:1:1.

[0020] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0021] 1. The present invention performs sample balancing and sample amplification on the data set, which can improve the generalization ability of the model and prevent the model from overfitting;

[0022] 2. The present invention introduces deformable convolution, which can adaptively adjust to irregular disease shapes and better capture local features;

[0023] 3. The present invention incorporates a receptive field attention mechanism, which can enhance the receptive field of the neural network, adapt to road disease detection of different scales, and overcome the interference of complex backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the process of the present invention;

[0025] Figure 2 is a schematic diagram of a pavement disease image in the present invention;

[0026] Figure 3It is a schematic diagram of comparing road images before and after data amplification in the present invention;

[0027] Figure 4 It is a schematic diagram of the structure of C2f_dcnv2 in the present invention;

[0028] Figure 5 It is a schematic diagram of the structure of the deformable convolution DCNv2 in the present invention;

[0029] Figure 6 Schematic diagram of the structure of the receptive field attention convolution module RFAConv in the present invention;

[0030] Figure 7 It is a schematic diagram of the overall network structure of the present invention;

[0031] Figure 8 It is a schematic diagram of the road disease detection results in the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings:

[0033] The present invention can be implemented in many different forms and should not be considered to be limited to the embodiments described herein. On the contrary, these embodiments are provided to make this disclosure thorough and complete, and will fully express the scope of the present invention to those skilled in the art. In the accompanying drawings, components are enlarged for clarity.

[0034] The main application scenarios of the present invention are: under the influence of factors such as extreme environments and traffic load, various damages and defects appear on the road. During the road maintenance process, it is necessary to detect road diseases so as to repair them and ensure the safety and durability of the road. Therefore, the main work of the present invention is to study the road disease detection algorithm based on deep learning, and optimize the detection algorithm to make it more efficient to detect defects of different sizes and shapes in road surface diseases, and have a certain robustness.

[0035] See also Figure 1 , which shows a flow chart of a road surface disease detection method based on an improved YOLOv8 model provided by an embodiment of the present invention, the method comprising the following steps:

[0036] Step 1), obtain the pavement disease image dataset and divide it into training set, validation set and test set.

[0037] In step 1), the pavement damage image dataset may be a public dataset or may be acquired through collection, and it is composed of a preset number of pavement damage images containing annotation information, and is used to improve the training stage of the YOLOv8 target detection model.

[0038] When the pavement disease image dataset is acquired through collection, the following steps are included:

[0039] Step 1.1), collect a preset number of pavement images, screen the pavement images containing pavement diseases, use labelme annotation software to frame-annotate seven types of diseases, namely cracks, crack patches, cracks, crack patches, potholes, pothole patches and manhole covers, and obtain pavement disease images containing annotation information, wherein the annotation information includes the disease category and coordinates, such as Figure 2 As shown;

[0040] Step 1.2), in order to solve the sample imbalance problem in the data set, undersample the categories with more samples to reduce the number of samples; image enhancement is performed on the categories with fewer samples, including random addition of salt and pepper noise, random histogram equalization, random adjustment of image saturation, brightness, contrast and sharpness, etc., to simulate image changes under different lighting and shooting conditions, so as to achieve sample amplification while maintaining sample category information. The enhanced image is as follows: Figure 3 As shown;

[0041] Step 1.3), divide the data set into training set, validation set and test set in a ratio of 8:1:1.

[0042] At this point, a data set for training the pavement damage detection model can be obtained.

[0043] Step 2), build an improved YOLOv8 model that introduces deformable convolution and receptive field attention mechanism.

[0044] After obtaining the data set for training, it is necessary to build a disease detection network. It should be noted that the disease detection network in the present invention is a YOLOv8 network, but the original YOLOv8 model is not very ideal for detecting pavement diseases, and the detection accuracy is low. Therefore, it is necessary to further improve the disease detection network so that it can detect pavement diseases more accurately.

[0045] Step 2.1), build the C2f_dcnv2 module;

[0046] For the head network of the YOLOv8 model, all C2f modules are replaced with C2f_dcnv2 modules; the shape and size of road defects may vary depending on the road condition and maintenance. Deformable convolution can adaptively adjust the shape of the convolution kernel to better capture and identify various irregular defects such as cracks, potholes, etc.

[0047] References Figure 4The structural schematic diagram of the C2f_dcnv2 module shown in the figure, wherein the C2f_dcnv2 module includes the first to second Conv modules, the Split module, and the first to third Bottleneck_dcnv2 modules;

[0048] The first to third Bottleneck_dcnv2 modules have the same structure, and all include a third Conv module, a deformable convolution layer DCNv2, a batch normalization layer, and a SiLU activation function layer connected in sequence, wherein the input end of the third Conv module is connected to the external input, and the input end of the third Conv module and the output end of the SiLU activation function layer are added as output;

[0049] The first Conv module, the Split module, the first Bottleneck_dcnv2 module, the second Bottleneck_dcnv2 module, and the third Bottleneck_dcnv2 module are connected in sequence, the input end of the first Conv module is used as the input end of the C2f_dcnv2 module, the output end of the Split module, the output end of the first Bottleneck_dcnv2 module, the output end of the second Bottleneck_dcnv2 module, and the output end of the third Bottleneck_dcnv2 module are spliced ​​according to channels and input into the input end of the second Conv module, and the output end of the second Conv module is used as the output end of the C2f_dcnv2 module.

[0050] refer to Figure 5 The deformable convolution DCNv2 schematic diagram shown in FIG. 1 is a deformable convolution layer that obtains a set of convolution kernel offset prediction results and weight prediction results through a convolution operation on the input feature map. The size of the offset feature map is similar to the input feature map. Figure 1 The number of channels is 2N, where 2 means that each offset is two values ​​(x, y), and N is the number of pixels in the convolution kernel, that is, the deformable convolution learns a set of offsets for each pixel of the input feature map. This embodiment uses DCNv2, which introduces a modulation mechanism that uses weights to adjust the input feature amplitudes from different spatial positions. The size of the weighted feature map is proportional to the input feature. Figure 1 The number of channels is N;

[0051] Given a convolution kernel with K sampling positions, let w k and p k Denote the weight and pre-specified offset of the kth position respectively. Let x(p) and y(p) denote the features of position p in the input feature map x and the output feature map y respectively. The expression of deformable convolution DCNv2 is shown in formula (1);

[0052]

[0053] In the formula, Δp k and Δm k are the learnable offset and weight coefficient of the kth position respectively. Weight coefficient Δm k In the range [0,1], Δp k is a real number with no range constraints. k and Δm k They are initialized to 0 and 0.5 respectively (the default offset is 0, and the contribution rate of the sampling points cannot be distinguished). The corresponding convolution kernel parameters are initially 0, and the learning rate of the convolution layer is set to 0.1 of the existing layer.

[0054] Step 2.2), for the head network of the YOLOv8 model, replace all C2f modules with C2f_dcnv2 modules to obtain the YOLOv8 model with deformable convolution.

[0055] Step 2.3), for the backbone network of the YOLOv8 model that introduces deformable convolution, all Conv modules except the first Conv module are replaced with the receptive field attention convolution module RFAConv to obtain the improved YOLOv8 model.

[0056] Spatial attention is widely used to improve the performance of convolutional neural networks, but existing spatial attention, such as convolutional block attention module (CBAM) and coordinated attention (CA), cannot effectively solve the parameter sharing problem of large-scale convolution kernels (such as 3×3 convolution) because it does not consider the spatial characteristics of the entire receptive field. Receptive field attention (RFA) uses non-overlapping receptive field sliders to extract features, emphasizing the importance of various features within the receptive field slider, and also pays attention to the spatial characteristics of the receptive field to thoroughly solve the problem of convolution kernel parameter sharing.

[0057] The receptive field attention convolution module RFAConv reference Figure 6 As shown, in order to ensure a small amount of computational overhead and number of parameters, RFAconv uses AvgPool to aggregate the global features of each receptive field feature, then uses 1×1 grouped convolution for information interaction, and finally uses softmax to emphasize the importance of each feature in the receptive field feature. The calculation of RFA is shown in formula (2);

[0058]

[0059] In the formula, g 1×1 represents a grouped convolution of size 1×1, k represents the size of the convolution kernel, k=3 is selected, Norm represents normalization, X represents the input feature map, and F is composed of the attention map A rf And the transformed receptive field spatial feature F rf Multiply them together to get .

[0060] In step 2), the existing YOLOv8 model is improved. The improved YOLOv8 road surface disease detection model refers to Figure 7 As shown. In the head network of the YOLOv8 model, all C2f modules are replaced with C2f_dcnv2 modules. The shape and size of road defects may vary depending on the road surface conditions and maintenance conditions. The deformable convolution can adaptively adjust the shape of the convolution kernel to better capture and identify various irregular defects, such as cracks, potholes, etc. For the backbone network of the YOLOv8 model that introduces deformable convolution, all Conv modules except the first Conv module are replaced with the receptive field attention convolution module RFAConv to enhance the receptive field of the neural network and adapt to the detection of road defects of different scales, thereby obtaining an improved YOLOv8 model.

[0061] Step 3), training the improved YOLOv8 model based on the training set, the validation set and the test set to obtain a trained improved YOLOv8 model.

[0062] Step 3.1), in the training phase, first initialize the model parameters of the improved YOLOv8 target detection model;

[0063] Step 3.2), input the pavement defect image in the pavement disease image dataset into the improved YOLOv8 target detection model for identification, and obtain the detection frame and category information; input the detection frame and category information and the annotation information of the pavement defect image into a preset optimization loss function, and calculate the loss value, wherein the loss function is the bounding box regression loss CIoU Loss and DFL loss, and the category classification cross entropy loss;

[0064] In step 3.3), the gradient descent algorithm is set to SGD, the model parameters of the improved YOLOv8 target detection model are optimized, and the training rounds are set to 650 epochs to obtain the trained improved YOLOv8 target detection model.

[0065] The superiority of the improved pavement disease detection network was evaluated by Precision, Recall and Average Precision. The training results of the improved pavement disease detection network were compared with the original network without improvement. The results show that the improved YOLOv8 network has better accuracy performance in pavement disease detection. The results are shown in Tables 1 and 2, where Table 1 represents the data detected by the original YOLOv8 and Table 2 represents the data of the improved YOLOv8 network.

[0066] Table 1 Detection results of the original YOLOv8 model

[0067]

[0068] Table 2 Improved YOLOv8 model detection results

[0069]

[0070]

[0071] Step 4), inputting the road surface disease image to be detected into the trained improved YOLOv8 model for recognition to obtain the detection result;

[0072] The road surface image to be detected is input into the trained optimized disease detection model, and the output is as follows Figure 8 The pavement defect detection image shown in the figure includes the defect category of each defect, which is marked with a rectangular frame in the defect detection image and the confidence level is indicated.

[0073] In summary, the pavement disease detection method based on the improved YOLOv8 model provided by the embodiment of the present invention first obtains a pavement disease data set. Secondly, the C2f_dcnv2 module is used to replace all C2f modules of the original YOLOv8 model head network to better capture and identify various irregular diseases, such as cracks, potholes, etc., and improve the detection accuracy; further, the receptive field attention convolution module RFAConv is used to replace all Conv modules of the original YOLOv8 model backbone network except the first Conv module, so as to improve the feature extraction ability of the model, enhance the receptive field of the neural network, and adapt to the detection of pavement diseases of different scales; then, the improved model is trained to obtain a model with higher detection accuracy; finally, the trained model is used to detect the pavement image to obtain the final detection effect.

[0074] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0075] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. The pavement damage detection method based on the improved YOLOv8 model is characterized by: The following steps are involved: Step 1), obtain the pavement disease image dataset and divide it into a training set, a validation set and a test set; Step 2) Build an improved YOLOv8 model that introduces deformable convolution and receptive field attention mechanism; Step 2.1), build the C2f_dcnv2 module; The C2f_dcnv2 module includes the first to second Conv modules, the Split module, and the first to third Bottleneck_dcnv2 modules; The first to third Bottleneck_dcnv2 modules have the same structure, and all include a third Conv module, a deformable convolution layer DCNv2, a batch normalization layer, and a SiLU activation function layer connected in sequence, wherein the input end of the third Conv module is connected to the external input, and the input end of the third Conv module and the output end of the SiLU activation function layer are added as output; The first Conv module, the Split module, the first Bottleneck_dcnv2 module, the second Bottleneck_dcnv2 module, and the third Bottleneck_dcnv2 module are connected in sequence, the input end of the first Conv module is used as the input end of the C2f_dcnv2 module, the output end of the Split module, the output end of the first Bottleneck_dcnv2 module, the output end of the second Bottleneck_dcnv2 module, and the output end of the third Bottleneck_dcnv2 module are spliced ​​according to channels and input to the input end of the second Conv module, and the output end of the second Conv module is used as the output end of the C2f_dcnv2 module; The first to third Conv modules have the same structure, and all include a Conv layer, a batch normalization layer, and a SiLU activation function layer connected in sequence; Step 2.2), for the head network of the YOLOv8 model, replace all C2f modules with C2f_dcnv2 modules to obtain the YOLOv8 model with deformable convolution; Step 2.3), for the backbone network of the YOLOv8 model that introduces deformable convolution, replace all Conv modules except the first Conv module with the receptive field attention convolution module RFAConv to obtain the improved YOLOv8 model; Step 3), training the improved YOLOv8 model based on the training set, the validation set and the test set to obtain a trained improved YOLOv8 model; Step 4) Input the road surface disease image to be detected into the trained improved YOLOv8 model for recognition to obtain the detection result.

2. The pavement disease detection method based on the improved YOLOv8 model according to claim 1 is characterized in that: The images in the pavement disease image dataset described in step 1) are all annotated with pavement disease frame types using labelme annotation software, and the annotation information includes the disease category and coordinates.

3. The pavement disease detection method based on the improved YOLOv8 model according to claim 1 is characterized in that: In step 1), the pavement disease image dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

Images