Asphalt pavement defect detection method based on improved YOLO module

Abstract

The invention discloses an asphalt pavement defect detection method based on an improved YOLO module, which comprises the following steps: acquiring asphalt pavement data and preprocessing the data to obtain a data set; a ResNet50 neural network is adopted to replace a backbone network of the YOLOv4 model, and a YOLOv4-Res50 detection model is constructed; determining the number and the size of detection anchor frames in the YOLOv4-Res50 detection model; by taking the data set as input and the asphalt pavement defect result as output, training the YOLOv4-Res50 detection model to obtain a trained YOLOv4-Res50 detection model; and carrying out defect detection on the asphalt pavement through the trained YOLOv4-Res50 detection model. According to the method, an original YOLOv4 backbone network is replaced by a ResNet50 convolutional neural network model, and the number and size design of anchor frames are detected again, so that an improved model YOLOv4-Res50 is formed. Through verification, the improved YOLOv4-Res50 model can obviously improve the asphalt pavement defect detection precision.

Description

Technical Field

[0001] This invention belongs to the field of road engineering and civil engineering technology, and in particular relates to a method for detecting defects in asphalt pavement based on an improved YOLO module. Background Technology

[0002] With the development of computer technology and convolutional neural network technology, more and more artificial intelligence technologies are being applied to the construction and maintenance of practical road engineering and civil engineering. Roads bear the heavy responsibility of transporting people and goods, connecting urban nodes, and promoting regional economic prosperity. However, as roads age, traffic factors such as overloading and environmental factors such as rain, snow, high temperatures, and sunlight inevitably cause damage to roads, especially asphalt roads, affecting driving safety. Therefore, road operation and maintenance, especially defect detection of asphalt pavements, is a crucial link in ensuring road driving safety and passenger comfort. YOLOv4, as a one-stop target detection software, is widely used in the field of asphalt pavement defect detection, which can significantly improve detection efficiency and avoid human detection errors. However, through engineering practice, it has been found that the accuracy achieved by directly using the YOLOv4 model for asphalt pavement defect detection is not high, only 48%. Therefore, how to improve the accuracy of asphalt pavement defect detection has become a hot issue in this field. Summary of the Invention

[0003] Objective: The objective of this invention is to provide an asphalt pavement defect detection method based on an improved YOLO module. Specifically, the original YOLOv4 backbone network is replaced with a ResNet50 convolutional neural network model, and the number and size of the detection anchor frames are redesigned, resulting in the improved YOLOv4-Res50 model. Verification has shown that the improved YOLOv4-Res50 model significantly improves the accuracy of asphalt pavement defect detection.

[0004] Technical solution: The present invention provides a method for detecting defects in asphalt pavement based on an improved YOLO module, comprising the following steps:

[0005] Step 1: Collect asphalt pavement data and preprocess the data to obtain a dataset;

[0006] Step 2: Replace the backbone network of the YOLOv4 model with a ResNet50 neural network to construct a YOLOv4-Res50 detection model;

[0007] Step 3: Determine the number and size of the detection anchor frames in the YOLOv4-Res50 detection model;

[0008] Step 4: Using the dataset from Step 1 as input and the asphalt pavement defect results as output, train the YOLOv4-Res50 detection model to obtain the trained YOLOv4-Res50 detection model.

[0009] Step 5: Detect defects in asphalt pavement using the trained YOLOv4-Res50 detection model.

[0010] Further, step 1 specifically involves: collecting image data of asphalt pavement defects, then renaming and unifying the size of the collected image data, and then labeling the processed images, classifying them into cracks and potholes; dividing the labeled asphalt pavement defect data into training set, validation set and test set according to the proportion; and performing enhancement processing on the image data in the training set, including random translation, random rotation, inversion, and random changes in light and color.

[0011] Further, step 2 specifically involves: the YOLOv4 model includes a backbone network, a neck, and a probe; a ResNet50 network is used to replace the original YOLOv4 backbone network; the neck includes upsampling, downsampling, and pooling layers to fuse features at different scales of the image; finally, the probe is used for localization and judgment, and the final calculation result is output; the calculation result consists of three parts, including the target object position, object class, and probability value.

[0012] Furthermore, step 3 specifically involves:

[0013] Step 3.1: The number and size of the detection anchor frames are determined by calculation using the image database. First, the number of detection anchor frames is determined. The relationship between the number of detection anchor frames and the mean Intersection over Union (IoU) is determined through data fitting and is expressed as:

[0014] Mean_IoU=a·n b (n≤15) (1)

[0015] In equation (1), parameter a = 0.494, the upper and lower limits of the 95% confidence interval are [0.478, 0.51], parameter b = 0.165, the upper and lower limits of the 95% confidence interval are [0.150, 0.180], and n is the number of detection anchor frames;

[0016] Step 3.2: Determine the size of the detection anchor box; First, normalize the annotation box, that is, normalize the width and height of the annotation box to the ratio of the size to the image size; then use k-means clustering analysis to classify the annotation box into k classes, that is, determine the number of detection anchor boxes; calculate according to formula (2) to determine the size of the detection anchor box;

[0017]

[0018] In equation (2), N is the number of annotation boxes.

[0019] Furthermore, step 4 specifically includes the following steps:

[0020] Step 4.1, Model training and validation, using the position loss function as shown in equation (3);

[0021]

[0022] In equation (3), ρ is the distance between the center of the annotation box and the center of the detection anchor box, l is the diagonal length of the smallest rectangle that can simultaneously contain the annotation box and the detection anchor box, and u is the penalty function term;

[0023] Step 4.2: Perform tests on the test set and compare the results with the original YOLOv4 model to verify its accuracy.

[0024] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the present invention.

[0025] The present invention also discloses a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the method of the present invention.

[0026] The present invention also discloses a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method of the present invention.

[0027] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: This invention proposes a replacement technique for the YOLOv4 backbone network. Using an improved detection model, called YOLOv4-Res50, the detection accuracy of the model for asphalt pavement defects can be significantly improved. Specifically, the original YOLOv4 backbone network is replaced with a ResNet50 convolutional neural network model, and the number and size of the detection anchor frames are redesigned, thus forming the improved model YOLOv4-Res50. The YOLOv4 model, as an artificial intelligence model that balances running speed and detection accuracy, has been widely used in various fields. When attempting to use YOLOv4 for asphalt pavement defect detection, this invention found that its detection accuracy was low, reaching only 48%. Therefore, this invention proposes replacing the original YOLOv4 model backbone network with a ResNet50 convolutional neural network and recalculating and determining the number and size of the detection anchor frames. Finally, the improved YOLOv4-Res50 model has been verified to achieve an asphalt pavement defect identification accuracy of 58%, a 10% improvement compared to the original model. This invention is beneficial for promoting the use of the YOLO model in asphalt pavement defect detection, improving the accuracy of asphalt pavement defect detection, and thus ensuring driving safety. Attached Figure Description

[0028] Figure 1 This is a flowchart of the present invention.

[0029] Figure 2 This is a replacement diagram for the backbone network.

[0030] Figure 3 This is a map showing the distribution of defects in asphalt pavement.

[0031] Figure 4 Enhance the image by adding (a) cracks and (b) pits;

[0032] Figure 5 The curve showing the number of anchor frames versus the average intersection-to-union ratio;

[0033] Figure 6 Precision-recall curves for the YOLOv4 model; (a) Precision-recall curves for pit identification under different cross-union ratios (CURRs); (b) Precision-recall curves when the CURR is 0.5.

[0034] Figure 7 Precision-recall curves for the YOLO-Re50s model; (a) Precision-recall curves for pit identification under different cross-union ratios (CURRs); (b) Precision-recall curves at an CURR of 0.5.

[0035] Figure 8 Examples of identification results for two neural network models: (a) YOLOv4 model results; (b) YOLO-Res50 model results. Detailed Implementation

[0036] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0037] The specific implementation process of this invention is as follows: Figure 1 As shown, the first step is to collect asphalt pavement defect image data. The asphalt pavement defect image data used in this invention was mainly collected from certain road sections in Nantong City, including Yuanlin Road, Qingnian Road, and Yanhe Road. The asphalt pavement defects to be identified are mainly divided into two categories: asphalt pavement cracks and pavement potholes. Then, these asphalt pavement defect images undergo preprocessing. After replacing the original YOLOv4 backbone network with a ResNet50 convolutional neural network, the required number and size of detection anchor boxes are determined programmatically. Since the number and size of detection anchor boxes are related to the recognition speed and accuracy of the improved model, this step is indispensable. Finally, the improved YOLOv4-Res50 model and the original YOLOv4 model are tested on the same test set to compare the recognition accuracy between the two models.

[0038] Figure 2This diagram illustrates the replacement of the YOLOv4 model backbone network. The YOLOv4 model structure is roughly divided into three parts: the backbone network, the neckline, and the probe, as shown below. Figure 2 As shown. The backbone network typically uses a convolutional neural network to extract features from the input image at different scales; the neck of the model includes upsampling, downsampling, and pooling layers to fuse features from different scales; finally, the probe performs localization and judgment, outputting the final calculation result. The calculation result consists of three parts: the target object's location, the object's class, and the probability value. ResNet50 is a 50-layer residual neural network, mainly used to alleviate network degradation problems, and is particularly suitable for image feature extraction, thus frequently used in image classification tasks. Here, the ResNet50 network is used to replace the original YOLOv4 backbone network to improve the YOLOv4 model's accuracy in identifying asphalt pavement defect images; the improved model is called the YOLOv4-Res50 model.

[0039] Image data of asphalt pavement defects were mainly collected from several streets in Nantong. The collected images were then renamed and resized to a uniform 1024×1024 pixels. The processed images were then labeled. This invention primarily addresses two types of asphalt pavement defects: cracks and potholes. After labeling, the labeled objects were statistically analyzed, and the results are as follows: Figure 3 As shown. Figure 3 The data shows that there are a total of 1,107 cracks and 691 potholes in the asphalt pavement.

[0040] The labeled asphalt pavement defect data were then divided into training, validation, and test sets in a ratio of 0.75:0.15:0.15. To ensure the robustness of the trained model, the image data in the training set was enhanced, including random translation, rotation, reversal, and random changes in lighting and color. Figure 4 Provide an example of the training set image data after enhancement.

[0041] Figure 4 (a) shows the processing result of a certain crack in the training set. It can be seen that the image has been randomly translated a certain distance along the x-axis and y-axis. Figure 4 (b) is a processed image of a certain pit in the training set, which is also translated a certain distance along the x-axis and y-axis.

[0042] The next crucial step is determining the number and size of the detection anchor boxes, as this affects the recognition speed and accuracy of the improved YOLOv4 model. More detection anchor boxes improve recognition accuracy but also increase recognition time. Conversely, too few detection anchor boxes will negatively impact recognition accuracy. Figure 5As shown. The number of detection anchor frames needs to be determined through calculation based on the specific image database. Figure 5 The relationship between the number of detection anchor frames and the mean intersection-to-union ratio (Mean IoU) is given for the asphalt pavement defect detection problem of the present invention.

[0043] Depend on Figure 5 It can be seen that as the number of detection anchor boxes increases, the average intersection-union ratio also increases. However, the increase in detection anchor boxes also means a decrease in the model's recognition speed, requiring more time for inference. Therefore, according to Figure 5 Ultimately, six detection anchor frames were selected. These six anchor frames were divided into two groups of three. Once the number of anchor frames was determined, their size needed to be specified based on the dimensions of the objects being inspected. Larger anchor frames were used to anchor larger targets, and smaller anchor frames were used to anchor smaller targets. In this invention, the size of the six anchor frames was determined based on the size distribution of road surface cracks and potholes, which was implemented using a programmed method.

[0044] The detection accuracy of the improved YOLOv4-Res50 model was then validated using test set data. First, the original YOLOv4 model was used to identify defects in asphalt pavement images from the test set, yielding the precision (mAP) index and precision-recall curves for the YOLOv4 model. Then, the improved YOLOv4-Res50 model was used to identify defects in the same test set images, yielding the precision (mAP) index and precision-recall curves for the YOLOv4-Res50 model. The accuracy of the YOLOv4-Res50 model was verified by comparing the differences between the two models.

[0045] Figure 6 The identification results using the YOLOv4 model on the test set data are presented. Figure 6 (a) The YOLOv4 model's identification results for pothole defects in asphalt pavements at different cross-union ratio (IoU) levels (0.5, 0.75, 0.9) are presented. It can be seen that the model's identification accuracy gradually decreases as the IoU level increases. To standardize the comparison and facilitate comparison, the index at the IoU = 0.5 level is used. Figure 6 (b) The precision-recall curves of the YOLOv4 model for identifying two types of defects in asphalt pavement are presented. It can be seen that the YOLOv4 model performs better in identifying pothole defects in asphalt pavement. Specifically, the precision (AP) of the YOLOv4 model for identifying cracks is 0.41, the precision (AP) for identifying potholes is 0.55, and the total mAP is 0.48.

[0046] and Figure 6 similar, Figure 7Precision-recall curves for identifying the same set of image data using the improved YOLOv4-Res50 model are presented. Figure 7 (a) The YOLOv4-Res50 model is presented to identify potholes in asphalt pavements at different cross-union ratio (CUN) levels (0.5, 0.75, 0.9). Similarly, the model's identification accuracy gradually decreases as the CUN level increases. Figure 7 (b) Presents the precision-recall curves of the YOLOv4-Res50 model for identifying two types of defects in asphalt pavement at an intersection-over-union (IoU) ratio of 0.5. It can be seen that the YOLOv4-Res50 model outperforms the YOLOv4 model in identifying both types of defects in asphalt pavement. The YOLOv4-Res50 model achieves an accuracy (AP) of 0.52 for cracks and 0.64 for potholes, with an overall mAP of 0.58. The comparison shows that by replacing the backbone network of the original YOLOv4 model with a ResNet50 convolutional neural network, the improved YOLOv4-Res50 model achieves a 10% higher mAP for identifying asphalt pavement defects compared to the original model, basically meeting the actual requirements for detecting asphalt pavement defects.

[0047] Figure 8 Taking a crack in an asphalt pavement in the test set as an example, we demonstrate the different identification effects of the two models. Figure 8 (a) shows the identification results using the YOLOv4 model. The model accurately detected the location of the crack and correctly classified it. However, there are duplicate detection anchor boxes in the lower left corner, which reflects that the accuracy of the model still needs to be improved. Figure 8 (b) The recognition results of the same image using the improved YOLOv4-Res50 model can be seen. It can be found that the YOLOv4-Res50 model can completely frame the crack with only two detection anchor boxes. This shows that the improved YOLOv4-Res50 model is more accurate in detecting target objects than the YOLOv4 model.

[0048] In summary, by replacing the original YOLOv4 backbone network with a ResNet50 convolutional neural network and redesigning the number and size of the detection anchor frames, the improved YOLOv4-Res50 model achieves a 10% higher detection accuracy for the two types of asphalt pavement defects compared to the original model, thus meeting the actual needs of asphalt pavement defect detection.

Claims

1. A method for detecting defects in asphalt pavement based on an improved YOLO module, characterized in that, Includes the following steps: Step 1: Collect asphalt pavement data and preprocess the data to obtain a dataset; Step 2: Replace the backbone network of the YOLOv4 model with a ResNet50 neural network to construct a YOLOv4-Res50 detection model; Step 3: Determine the number and size of the detection anchor frames in the YOLOv4-Res50 detection model; Step 4: Using the dataset from Step 1 as input and the asphalt pavement defect results as output, train the YOLOv4-Res50 detection model to obtain the trained YOLOv4-Res50 detection model. Step 5: Detect defects in asphalt pavement using the trained YOLOv4-Res50 detection model.

2. The asphalt pavement defect detection method based on the improved YOLO module according to claim 1, characterized in that, Step 1 specifically involves: collecting image data of asphalt pavement defects, then renaming and unifying the size of the collected image data, and then labeling the processed images, classifying them into cracks and potholes; dividing the labeled asphalt pavement defect data into training set, validation set and test set according to the proportion; and performing enhancement processing on the image data in the training set, including random translation, random rotation, inversion, and random changes in light and color.

3. The asphalt pavement defect detection method based on the improved YOLO module according to claim 1, characterized in that, Step 2 is as follows: The YOLOv4 model includes a backbone network, a neck, and a probe; a ResNet50 network is used to replace the original YOLOv4 backbone network; the neck includes upsampling, downsampling, and pooling layers to fuse features at different scales of the image; finally, the probe is used for localization and judgment, and the final calculation result is output; the calculation result consists of three parts, including the target object position, object class, and probability value.

4. The asphalt pavement defect detection method based on the improved YOLO module according to claim 1, characterized in that, Step 3 specifically involves: Step 3.1: The number and size of the detection anchor frames are determined by calculation using the image database. First, the number of detection anchor frames is determined. The relationship between the number of detection anchor frames and the mean Intersection over Union (IoU) is determined through data fitting and is expressed as: Mean_IoU=a·n b (n≤15) (1) In equation (1), parameter a = 0.494, the upper and lower limits of the 95% confidence interval are [0.478, 0.51], parameter b = 0.165, the upper and lower limits of the 95% confidence interval are [0.150, 0.180], and n is the number of detection anchor frames; Step 3.2: Determine the size of the detection anchor box; First, normalize the annotation box, that is, normalize the width and height of the annotation box to the ratio of the size to the image size; then use k-means clustering analysis to classify the annotation box into k classes, that is, determine the number of detection anchor boxes; calculate according to formula (2) to determine the size of the detection anchor box; In equation (2), N is the number of annotation boxes.

5. The asphalt pavement defect detection method based on the improved YOLO module according to claim 1, characterized in that, Step 4 specifically includes the following steps: Step 4.1, Model training and validation, using the position loss function as shown in equation (3); In equation (3), ρ is the distance between the center of the annotation box and the center of the detection anchor box, l is the diagonal length of the smallest rectangle that can simultaneously contain the annotation box and the detection anchor box, and u is the penalty function term; Step 4.2: Perform tests on the test set and compare the results with the original YOLOv4 model to verify its accuracy.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 1.

7. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

Images