Intelligent detection method capable of reflecting damage position and shape direction of asphalt pavement
By introducing a rotating detection box into the YOLOv4 neural network, the problem of insufficient accuracy in asphalt pavement damage detection in existing technologies is solved, achieving higher detection accuracy and more accurate identification of damage morphology and direction.
Patent Information
- Application Number
- CN202510851205.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-24
- Publication Date
- 2025-11-07
AI Technical Summary
The existing YOLO series software uses a traditional horizontal detection frame in asphalt pavement damage detection, which cannot accurately reflect the actual direction and shape of pavement damage, resulting in insufficient detection accuracy.
A rotating detection frame is used to annotate asphalt pavement damage, and the YOLOv4 neural network is optimized to output the rotating detection frame parameters, forming a YOLOv4-R model for detecting the location and morphology of asphalt pavement damage.
The detection accuracy has been improved, with the mAP of the rotating detection frame model increasing from 48% to 74%, enabling it to more accurately capture the shape and direction of road surface damage, making it particularly suitable for detecting inclined cracks and potholes.
Smart Images
Figure CN120912840A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of traffic engineering and civil engineering, and particularly relates to an intelligent detection method capable of reflecting the position and shape direction of asphalt pavement damage. BACKGROUND
[0002] With the development of artificial intelligence software and computer vision technology, many artificial intelligence software are used in road engineering and civil engineering, for example, YOLO series model is used to identify and locate the damage of asphalt pavement. The application of this artificial intelligence technology not only saves manual labor, improves the detection efficiency, but also avoids the errors caused by manual detection, and improves the detection accuracy. However, so far, the YOLO series software used in the detection of asphalt pavement damage in the literature and actual application all use traditional horizontal detection boxes, that is, the horizontal line of the detection box is parallel to the x-axis, and the vertical line of the detection box is parallel to the y-axis, which does not conform to the actual situation of asphalt pavement damage. In most cases, the cracks or potholes of the road are not in the horizontal or vertical direction.
[0003] The commonly used target recognition software can be roughly divided into two categories. One is the two-stage detection model represented by R-CNN, which detects the target in two steps, that is, first selects some regions with high possibility, and then analyzes and identifies these regions. The other is the one-stage detection software represented by YOLO series, that is, the image data only needs to pass through the neural network once to identify the target object. Compared with the two-stage detection model, the YOLO model can better balance the speed and accuracy of target recognition, so it is more popular in the industry.
[0004] At present, it is very common to use YOLO software to detect the damage of asphalt pavement, but so far, the YOLO software used in the detection of asphalt pavement damage reported in the literature all use traditional horizontal detection boxes, which does not conform to the actual situation of asphalt pavement damage. SUMMARY
[0005] The purpose of the present application is to provide an intelligent detection method capable of reflecting the position and shape direction of asphalt pavement damage. The use of a rotating detection box makes the detection result more consistent with the actual situation of asphalt pavement damage. The rotating detection box proposed in the present application can not only identify the type and position of asphalt pavement damage, but also detect the shape and direction of pavement damage.
[0006] TECHNICAL SCHEME: The present application provides an intelligent detection method capable of reflecting the position and shape direction of asphalt pavement damage, comprising the following steps:
[0007] Step 1, collecting asphalt pavement data and preprocessing the data;
[0008] Step 2, the pre-processed data is labeled for asphalt damage by rotating the bounding box, and a labeled data set is obtained;
[0009] Step 3, the YOLOv4 neural network is optimized to obtain a YOLOv4-R model, the labeled data set is input, and the detection box is output, the YOLOv4-R model is trained, and a trained YOLOv4-R model is obtained;
[0010] Step 4, the trained YOLOv4-R model is used to intelligently detect the position and shape direction of the asphalt pavement damage.
[0011] Further, in step 1, the preprocessing includes uniformly adjusting the size of the image, image enhancement processing, and image renaming; wherein the image enhancement processing includes random rotation, cutting, inversion and color fine tuning.
[0012] Further, step 2 is specifically: the asphalt pavement damage is labeled by using a rotating bounding box, compared with a horizontal bounding box, the rotating bounding box outputs 5 parameters, which are the frame center coordinates (Xc, Yc), the frame horizontal width W, the frame vertical height H, and the rotation angle θ with the X axis positive direction.
[0013] Further, step 3.1, input optimization of the model: adjust the input layer parameter setting, so that the model reads the required parameters in order, including the rotation angle θ of the labeled box;
[0014] Step 3.2, output optimization of the model: adjust the output layer parameter setting of the model, so that the model outputs the rotation angle α of the corresponding detection box;
[0015] Step 3.3, optimize the loss function part related to the position and angle of the detection box, which is specifically expressed as:
[0016]
[0017] In the formula, λc and λ θ are the proportional coefficients of the detection box coordinates and the angle, s 2 is the grid number, that is, the row s x column s, and B is the number of detection boxes containing the detected object; The value is 0 or 1, if the grid and the detection box contain the detected object at the same time, the value is 1, otherwise the value is 0.
[0018] The application also discloses a computer device, which comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to realize the steps of the method.
[0019] The application also discloses a computer readable storage medium, which stores computer programs / instructions, and the computer programs / instructions are executed by a processor to realize the steps of the method.
[0020] The application also discloses a computer program product, which comprises computer programs / instructions, and the computer programs / instructions are executed by a processor to realize the steps of the method.
[0021] Beneficial effects: Compared with the prior art, the application has the following remarkable advantages:
[0022] (1) The application aims to accurately capture the damage of the asphalt pavement by using the artificial neural network model YOLOv4. Unlike the horizontal detection frame technology adopted by the traditional YOLO software, the application adopts a rotating detection frame to capture the shape and direction of the asphalt pavement damage. There are significant differences between the two in technology. First, in the labeling of the target, the application needs to use a rotating labeling frame to label the asphalt pavement damage target, and the labeling parameters are one parameter θ more than the conventional horizontal labeling frame, which is used to describe the rotation angle of the asphalt pavement damage. Secondly, when the model outputs the corresponding detection frame, an additional parameter α is also output to control the rotation of the detection frame.
[0023] (2) The application aims to use a rotating detection frame to detect the damage of the asphalt pavement. Compared with the traditional horizontal detection frame, the rotating detection frame is more suitable for the actual detection of the asphalt pavement damage, especially for the detection of inclined pavement cracks and potholes, and can more accurately capture the shape and direction of the pavement damage. There are two key innovations in the technical implementation: first, when labeling the asphalt pavement damage, a rotating labeling frame is used for labeling, and the output parameters of the model after labeling are five, which is one parameter θ more than the traditional horizontal labeling frame, which is used to describe the rotation angle of the pavement cracks or potholes. Second, the YOLOv4 model used is improved, so that the detection frame parameter output by the model when predicting contains the predicted rotation angle α, so that the target detection frame rotates accordingly.
[0024] (3) The rotating detection frame model proposed by the application is used to detect the collected asphalt pavement damage images, and the detection results are compared with the detection results of YOLOv4. It is shown that the model using the rotating detection frame can improve the detection accuracy mAP from 48% to 74%. Moreover, the model using the rotating detection frame can more accurately capture the shape and direction of the asphalt pavement damage, and is especially suitable for the detection of inclined pavement cracks and potholes. BRIEF DESCRIPTION OF DRAWINGS
[0025] Figure 1 Examples of asphalt pavement crack labeling (a) horizontal labeling frame (b) rotating labeling frame
[0026] Figure 2For the asphalt pavement pothole labeling example (a) horizontal label frame (b) rotating label frame;
[0027] Figure 3 For the comparison of two label frames;
[0028] Figure 4 For the invention implementation flowchart;
[0029] Figure 5 For the asphalt pavement damage image training set enhancement processing example (a) pavement pothole (b) pavement crack;
[0030] Figure 6 For the rotating detection frame model YOLOv4-R and YOLOv4 model detection result mAP comparison (a) YOLOv4 model mAP (b) rotating detection frame model mAP;
[0031] Figure 7 For the rotating detection frame model YOLOv4-R and YOLOv4 model confusion matrix comparison (a) YOLOv4 model confusion matrix (b) rotating detection frame model confusion matrix;
[0032] Figure 8 For the YOLOv4 detection result example (a) pavement crack (b) pavement pothole;
[0033] Figure 9 For the rotating detection frame model detection result example (a) pavement crack (b) pavement pothole; DETAILED DESCRIPTION
[0034] The technical solutions of the present application will be further described below in conjunction with the drawings.
[0035] Figure 1 and Figure 2 respectively show the actual asphalt pavement crack and pothole situation, it can be seen that the pavement damage often does not present along the horizontal or vertical direction, and more general situation is often at a certain angle with the horizontal line. Figure 1 In the diagonal seam, if a horizontal label frame is used for labeling, as shown in Figure 1 (a), several label frames often need to be used to completely frame the diagonal seam, and in this example Figure 1 two horizontal label frames are used in (a). If only one horizontal label frame is used, it will lead to a too large irrelevant area, affecting the accuracy of the YOLO model recognition. Figure 1 (b) shows the labeling situation using a rotating label frame, which has obvious advantages, as it only uses one rotating label frame to frame the entire diagonal seam, which will help improve the accuracy of subsequent model recognition. At the same time, since the detection frame predicted by the subsequent model is also correspondingly rotated, the shape (such as horizontal seam, vertical seam, diagonal seam) and direction of the detected target object can be given.
[0036] Figure 2 The annotation of potholes using two annotation methods is shown. Figure 2 (a) shows the annotation of two potholes in the figure using a conventional horizontal annotation box, Figure 2 (b) shows the annotation of the same potholes using a rotating annotation box. It is needless to say that the irrelevant area enclosed by the rotating annotation box is less, that is, the annotation of the detection target is more compact and reasonable. Correspondingly, the model trained using this kind of annotation set is more accurate in recognizing the target object.
[0037] Based on the above understanding, the present application proposes to use a rotating annotation box to annotate asphalt pavement damage. Unlike the horizontal annotation box, the rotating annotation box will output 5 parameters, namely the box center coordinates (Xc, Yc), the box horizontal width W, the box vertical height H, and the rotation angle θ with the X-axis positive direction, as shown in Figure 3 .
[0038] The asphalt pavement damage image set used in the present application is collected on several streets in Nantong City, including Qingnian Road, Garden Road and Tongjing Avenue. The image set includes 1107 asphalt pavement cracks and 691 potholes. Therefore, the present application mainly detects two types of asphalt pavement damage, namely cracks and potholes. Before training the YOLOv4 model, the collected asphalt pavement damage images are pre-processed, including unifying the image size to 1024x1024 pixels, renaming the images, etc. The specific process of the present application is shown in Figure 4 . After pre-processing, the images are annotated. Because we want to compare the different effects of using horizontal annotation boxes and rotating annotation boxes, the present application uses both of these annotation methods for this image set. After using horizontal annotation boxes and rotating annotation boxes for annotation, the annotated results are exported respectively, which are used to train the YOLOv4 model and the improved YOLOv4 model using rotating annotation boxes (hereinafter referred to as YOLOv4-R model). The detection box output by the YOLOv4-R model will include an additional parameter α to control the rotation of the detection box compared to the conventional horizontal detection box.
[0039] The asphalt pavement damage image set is divided into training set, validation set and test set according to the ratio of 0.75:0.15:0.15. For the pictures in the training set, before inputting the model for training, they are first enhanced, including random rotation, cutting, inversion and color fine-tuning, etc. So that the pictures used in each round of training are different, so that the trained neural network model is more robust. An example of the training set picture after enhancement is shown in Figure 5 . Figure 5 (a) shows a photo of asphalt pavement damage containing a pothole. It can be seen that the photo has been rotated and color fine-tuned. Figure 5(b) shows a picture containing a crack in the road surface, and the picture has also been rotated and light adjusted.
[0040] Then the enhanced training set pictures are used to train the YOLOv4 model and the YOLOv4-R model respectively. It needs to be emphasized that the image set pictures used to train these two models are the same, but the annotations of the road damage are different, that is, one uses horizontal annotation box to train the YOLOv4 model, and the other uses rotated annotation box to train the YOLOv4-R model. After training, the test set pictures are used to test the trained models, and the AP (average precision), mAP (average precision mean) and confusion matrix are used to measure the accuracy of the model.
[0041] Figure 6 The recognition results of the two models under different IoU (intersection over union) indicator levels (0.5, 0.75, 0.9) are shown. Figure 6 (a) is the AP value of the YOLOv4 model for identifying two types of asphalt pavement damage. For the sake of uniformity, here we compare the indicators at IoU = 0.5 level. Under this condition, the recognition accuracy AP of the YOLOv4 model for cracks is 0.41, and the recognition accuracy for potholes is 0.55, as shown in Figure 6 (a). The total mAP is 0.48. As shown in Figure 6 (b) gives the recognition results of the YOLOv4-R model. Under the IoU = 0.5 level, the recognition accuracy AP of the model for asphalt pavement cracks is 0.72, and the recognition accuracy for potholes is 0.77, and the total mAP is 0.74. As can be seen, by introducing the rotated detection box, the mAP indicator of the model is improved by 26%.
[0042] Figure 7 For the detection of test set pictures using the two models, the corresponding confusion matrix of the detection results. Figure 7 (a) is the confusion matrix obtained by using the YOLOv4 model. The matrix is composed of 3 rows and 3 columns, and the rows and columns are composed of cracks, potholes and no value. Here, no value means that the model does not detect the target object, or the model misjudges the picture background as the target object. As can be seen from Figure 7 (a), the model misjudges 1 case of pothole as crack, and does not detect 131 cracks and 45 potholes. In addition, 23 picture backgrounds are misjudged as cracks, and 13 picture backgrounds are misjudged as potholes. From Figure 7 (a), it can be inferred that the YOLOv4 model has a high misjudgment rate and low recognition accuracy. Figure 7(b) shows the recognition results of the YOLOv4-R model on the test set picture road surface damage. First, the YOLOv4-R model does not misjudge between the two categories of cracks and potholes. Second, the increase in the amount of data on the matrix diagonal means an increase in the number of objects correctly identified by the model, indicating an improvement in the accuracy of the model.
[0043] Finally, Figure 8 and Figure 9 respectively show examples of the results of using the YOLOv4 model and the YOLOv4-R model to detect cracks and potholes in asphalt pavement. Figure 8 (a) shows the recognition results of the YOLOv4 model on a diagonal crack in the pavement. As can be seen from Figure 8 (a), the YOLOv4 model uses horizontal detection boxes, so the accuracy is low, and a diagonal crack uses three horizontal detection boxes, with the lower left two detection boxes overlapping. Figure 8 (b) shows the recognition results of the YOLOv4 model on potholes. As can be seen, the YOLOv4 model accurately captures the potholes and correctly classifies them.
[0044] Figure 9 The recognition results of the YOLOv4-R model on the test set asphalt pavement picture targets are given. It must be noted that Figure 8 and Figure 9 are the same asphalt pavement pictures. Figure 9 (a) shows that after using the YOLOv4-R model, only one rotated detection box is needed to completely recognize the entire diagonal crack. As can be seen from Figure 9 (a), the YOLOv4-R model not only has high detection accuracy, but also clearly shows the shape and direction of the crack, making it more suitable for use in actual recognition of asphalt pavement damage. Figure 9 (b) shows the recognition results of the YOLOv4-R model on potholes. After using rotated detection boxes, the detection boxes are basically anchored to the potholes along their spatial orientation, clearly showing the shape and direction of the potholes.
[0045] In summary, the present application proposes a detection method that not only improves the detection accuracy of neural network models, but also reflects the shape and orientation of target objects. This method first labels target objects based on rotated bounding boxes, and then optimizes the YOLOv4 neural network to output corresponding rotated detection boxes, called the YOLOv4-R model. By detecting the asphalt pavement image data collected in the field and comparing it with the YOLOv4 detection results, the mAP value of the YOLOv4-R model is 26% higher than that of the YOLOv4 model, and the detection results not only accurately locate the pavement damage location and predict the class, but also reflect the shape and direction of the asphalt pavement damage.
Claims
1. An intelligent detection method capable of reflecting the position and shape direction of damage of an asphalt pavement, characterized in that, The method comprises the following steps: Step 1, collecting asphalt pavement data and preprocessing the data; Step 2, marking the data after preprocessing by rotating the marking box to obtain the marked data set; Step 3, optimizing the YOLOv4 neural network to obtain the YOLOv4-R model, taking the marked data set as input and the detection box as output, training the YOLOv4-R model to obtain the trained YOLOv4-R model; Step 4, intelligently detecting the position and shape direction of the asphalt pavement damage through the trained YOLOv4-R model.
2. The intelligent detection method capable of reflecting the position and shape direction of the damage of the asphalt pavement according to claim 1, characterized in that, In step 1, the preprocessing includes uniformly adjusting the size of the image, image enhancement processing, and image renaming; wherein the image enhancement processing includes random rotation, cutting, inversion, and color fine tuning.
3. The intelligent detection method capable of reflecting the position and shape direction of the damage of the asphalt pavement according to claim 1, characterized in that, Step 2 is specifically: using a rotating marking box to mark the asphalt pavement damage, compared with a horizontal marking box, the rotating marking box outputs 5 parameters, which are the box center coordinates (Xc, Yc), the box horizontal width W, the box vertical height H, and the rotation angle θ with the X axis positive direction.
4. The intelligent detection method capable of reflecting the position and shape direction of the damage of the asphalt pavement according to claim 3, characterized in that, Step 3 is specifically: Step 3.1, input optimization of the model: adjust the input layer parameter setting to make the model read the required parameters in order, including the rotation angle θ of the marking box; Step 3.2, output optimization of the model: adjust the output layer parameter setting of the model to make the model output the rotation angle α of the corresponding detection box; Step 3.3, optimize the loss function part related to the position and angle of the detection box, which is specifically expressed as: In the formula, λc and λ θ are the proportional coefficients of the detection frame coordinates and angle, respectively, s 2 is the grid number, i.e., row s x column s, and B is the number of detection frames containing the detected object; is 0 or 1, and is 1 if the grid and the detection frame both contain the detected object, and is 0 otherwise.
5. A computer apparatus comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 4. The processor executes the computer program to realize the steps of the method of claim 1.
6. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to realize the steps of the method of claim 1.
7. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to realize the steps of the method of claim 1.