Asphalt pavement disease intelligent detection method based on YOLOv5 algorithm

Abstract

The invention discloses an intelligent asphalt pavement disease detection method based on a YOLOv5 algorithm, and relates to the technical field of intelligent expressway and road maintenance, and the method comprises the steps: obtaining video data through a vehicle-mounted or airborne pan-tilt camera, and constructing an asphalt pavement disease image data set through manual marking; a YOLOv5 target detection model is adopted, an efficient multi-scale attention mechanism is introduced, and preliminary training is carried out; detecting a real-time database on the basis of a pre-trained improved YOLOv5 target detection model, and manually checking a detection result so as to optimize the model; and performing redundant data de-duplication on continuous multi-frame image information by using a dynamic planning method, correcting the image by using a chessboard method to obtain the actual size of the target, and performing evaluation according to a disease standard. Therefore, by adopting the method, the labor cost and the calculation cost can be saved, the pressure of storage requirements can be reduced, the real-time detection effect can be realized, and the method can be widely applied to asphalt pavement disease detection.

Description

Technical Field

[0001] The present invention relates to the field of smart highway and road maintenance technology, and in particular to an intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm. Background Art

[0002] With the completion of highway planning and construction goals and the continuous improvement and development of highway traffic networks, highway maintenance issues have become increasingly prominent. How to improve the level of informationization, specialization, and intelligence of highway maintenance through technical means is one of the urgent issues to be solved in the field of highway maintenance. Using target detection algorithms to detect and classify highway asphalt pavement defects and perform pixel-level precise positioning, and more intuitively and accurately obtain the overall picture of pavement defects, is of great significance to highway pavement maintenance and is an important basis for realizing intelligent road maintenance.

[0003] At present, studies have shown that the YOLO algorithm can be used to detect asphalt pavement defects. However, in the small target detection task in the field of road damage identification, the existing methods have problems such as model overfitting, low computational efficiency and complex convolution. In addition, during the detection process, the size of the detection target cannot be accurately estimated, which is easy to cause large errors, and usually the same lane and the same defect are detected and counted multiple times, resulting in data redundancy.

[0004] Therefore, there is an urgent need for a YOLO algorithm that can have good detection performance on target size and position information in the small target detection task of road damage identification, so as to reduce data redundancy and improve calculation speed, and provide a reference for improving the accuracy of asphalt pavement disease detection. Summary of the invention

[0005] The purpose of the present invention is to provide an intelligent asphalt pavement disease detection method based on the YOLOv5 algorithm, so that in the asphalt pavement disease detection, for small target detection tasks, it can accurately determine the size and position information of the target and improve the road damage recognition accuracy.

[0006] To achieve the above object, the present invention provides an intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm, comprising the following steps: S1. Obtain video data of asphalt pavement damage through a vehicle-mounted or airborne pan / tilt binocular camera; S2. Manually label the video data of asphalt pavement diseases and construct an asphalt pavement disease image dataset; S3. Based on the efficient multi-scale attention mechanism, an improved YOLOv5 target detection model is obtained; S4. Using the constructed asphalt pavement disease image dataset, the improved YOLOv5 target detection model is preliminarily trained; S5. Based on the improved YOLOv5 target detection model after preliminary training, the real-time database is tested, and the test result data is manually verified. The verification result is imported into the real-time database as a new data set, and the improved YOLOv5 target detection model after preliminary training is corrected and optimized; S6. Based on the dynamic programming method, the pre-selection box is selected by accumulating confidence, and redundant data deduplication is performed on the information of the same disease of the asphalt pavement through the pre-selection box; S7. Based on the deduplicated asphalt pavement disease data, the pixel coordinate system and the real coordinate system are associated by the chessboard method to obtain the actual size of the detection target, and the asphalt pavement disease is evaluated using the morphological fractal method.

[0007] Preferably, in step S2, the manual annotation includes six types of label information, namely, transverse cracks, longitudinal cracks, mesh cracks, pits, block repairs and strip repairs.

[0008] Preferably, in step S3, an efficient multi-scale attention mechanism preserves accurate location information and integrates contextual information of different scales by modeling long-term dependencies.

[0009] Preferably, step S4 includes providing validation set data feedback based on a constructed and annotated asphalt pavement disease image dataset through a loss function curve, adjusting the hyperparameters of the model in turn, and dynamically adjusting the step size of each parameter to perform preliminary training on the improved YOLOv5 target detection model.

[0010] Preferably, step S7 includes three-dimensional modeling, disease segmentation and size assessment; In stereo modeling, the chessboard method is used to correct the image and obtain the parallax image of the asphalt road surface; In the disease segmentation, the crack skeleton of the asphalt pavement is obtained through the refinement algorithm, and the pixels containing the disease are set to 1, and the pixels without the disease are set to 0 to obtain a binary image, and then the area of ​​the disease is obtained; In the size assessment, the size of asphalt pavement defects is assessed based on the principle of triangulation.

[0011] Preferably, correcting the image using the chessboard method includes using the chessboard to calculate the homography matrix; according to the internal camera parameter settings, the height and angle of the camera, the road surface disease image is projected and transformed using the homography matrix to obtain a top view of the road surface disease.

[0012] Preferably, obtaining the asphalt road parallax image includes: The disparity image of the road scene is obtained through global modeling, and then the local point disparity and disparity candidate algorithm are obtained by feature point matching. The obtained disparity image is optimized to obtain the asphalt road surface disparity image.

[0013] Therefore, the present invention adopts the above-mentioned intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm, which has the following technical effects: (1) Based on the YOLOv5 algorithm, an efficient multi-scale attention mechanism is introduced. By modeling long-range dependencies, it can preserve accurate location information and integrate contextual information of different scales, so that the neural network can better extract features. Combined with the newly added small target detection head, it can effectively improve the detection accuracy of small targets.

[0014] (2) The pavement defect identification and characterization methods are divided into three-dimensional modeling, defect segmentation, and size assessment. Through the refinement of the algorithm and point-by-point calculation, the size assessment of the overall defect is achieved, the accuracy of target size estimation is improved, and a reliable basis is provided for subsequent target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of an embodiment of an intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm; Figure 2 It is a YOLOv5 algorithm optimization flow chart of an embodiment of an intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm; Figure 3 The figure is a schematic diagram of asphalt pavement image correction in an embodiment of an intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm, wherein (a) is a schematic diagram of chessboard calibration, and (b) is a projected asphalt pavement image. DETAILED DESCRIPTION

[0016] The present invention can be explained in more detail by the following examples. The purpose of disclosing the present invention is to protect all changes and improvements within the scope of the present invention. The present invention is not limited to the following examples.

[0017] like Figure 1 As shown, the intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm provided by the present invention comprises the following steps: S1. Use a vehicle-mounted or airborne PTZ binocular camera to obtain image data of asphalt pavement defects for the first time, obtain the camera's internal and external parameters, distortion parameters, etc. for camera calibration, reduce the error caused by distortion, and better establish the connection between image coordinates and real-world coordinates. The video resolution is 1920*1080, 30FPS RGB images, and the vehicle speed is 60km / h to achieve real-time detection.

[0018] S2. Manually label the acquired image data and construct an asphalt pavement disease image dataset.

[0019] Among them, this embodiment uses DarkLabel software for manual annotation, and gives the image six types of label information through annotation, including transverse cracks, longitudinal cracks, mesh cracks, potholes, block repairs and strip repairs; at the same time, data enhancement is performed on the highway asphalt pavement disease dataset to expand the dataset, including pixel transformation and geometric transformation.

[0020] After labeling, the database is divided into training set, test set and validation set in a ratio of 8:1:1, so that it can effectively evaluate the generalization ability of the model and reduce the risk of overfitting.

[0021] S3. Introduce the efficient multi-scale attention mechanism (EMA) into the YOLOv5 target detection model, that is, improve the YOLOv5 target detection model. Among them, the EMA attention mechanism models long-term dependencies, saves accurate location information, and integrates context information of different scales, so that the neural network can better extract features.

[0022] The introduction of the EMA attention mechanism significantly improves the performance of the model and helps the model better generalize to unseen data, especially data with more noise.

[0023] S4. Based on the asphalt pavement disease image dataset, the improved YOLOv5 target detection model is preliminarily trained as follows: Based on the constructed and labeled asphalt pavement disease image dataset, the validation set data feedback is provided through the loss function curve, and the hyperparameters such as the number of model iterations, learning rate, and attenuation weight are adjusted in turn to carry out preliminary training of the improved YOLOv5 target detection model; among them, the dataset format is coco, the initial learning rate is 0.001, and the Adam algorithm is used in the target detection process to accelerate convergence and improve the stability of training by dynamically adjusting the step size of each parameter.

[0024] The asphalt pavement test set was used to evaluate the trained improved YOLOv5 target detection model, and the average precision (AP), mean average precision (mAP) and average detection time of the model were obtained respectively. The comprehensive evaluation indicators were compared and the optimal asphalt pavement disease detection method was selected.

[0025] S5, such as Figure 2 As shown, based on the improved YOLOv5 target detection model after preliminary training, the real-time database is tested to obtain the second batch of test result data; the second batch of test result data is manually checked and used as a new data set to perform error correction optimization on the improved YOLOv5 target detection model after preliminary training to obtain a more accurate model and improve the robustness of the model; and using the test results as a new data set can greatly reduce labor costs.

[0026] S6. De-duplicate redundant data. For the phenomenon of data redundancy, the idea of ​​dynamic programming is used to find the pre-selected box with the largest cumulative confidence. For consecutive multi-frame asphalt road images, the same disease only needs to retain the GPS information of one shooting location, the crack area in the image, and the disease information, thereby effectively reducing the amount of data and saving computing costs and storage requirements for subsequent calculations.

[0027] S7. Based on the test results, the highway asphalt pavement defects are identified and evaluated. The pavement defect identification and characterization are mainly divided into the following three steps: three-dimensional modeling, defect segmentation and size evaluation, as follows: (1) Stereo modeling: Based on the internal and external parameters and distortion parameters of the binocular camera, the chessboard method is used to correct the asphalt pavement image, and the disparity image of the road scene is obtained through global modeling. Then, the GMS feature point matching is used to obtain the local point disparity and disparity candidate algorithm, and then the disparity image is optimized to obtain a high-precision asphalt pavement disparity image, making the cracks in the road surface more three-dimensional, providing more accurate image information for subsequent pavement disease assessment.

[0028] (2) Defect segmentation: The crack image is detected and segmented through deep learning methods, and the crack skeleton is obtained by using a refinement algorithm. The pixels containing defects are set to 1 and the pixels without defects are set to 0 to obtain a binary image. The number of pixels with a value of 1 is calculated to calculate the area of ​​the defect, in preparation for size assessment.

[0029] (3) Size assessment: Based on the camera parameter settings, camera height and angle, the homography matrix is ​​used for projection transformation to project the rectangular frame of the asphalt road defect into the top view, eliminating the effect of the image being larger near and smaller far away. The pixel coordinates and size are associated with the actual size to solve the technical bottleneck of the difficulty in accurately assessing the size of the defect in intelligent inspection, thereby obtaining the actual size of the asphalt pavement defect and then evaluating it according to the defect standard.

[0030] Therefore, the above-mentioned intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm saves a lot of labor costs and computing costs, and reduces the pressure of storage requirements; the model has higher accuracy and stronger generalization ability, and can achieve real-time detection effect at a detection speed of 60km / h; and the computing cost is low and the price is low, so it can be widely used in asphalt pavement defect detection.

[0031] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. An intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm, characterized in that: The following steps are involved: S1. Obtain video data of asphalt pavement damage through a vehicle-mounted or airborne pan / tilt binocular camera; S2. Manually label the video data of asphalt pavement diseases and construct an asphalt pavement disease image dataset; S3. Based on the efficient multi-scale attention mechanism, an improved YOLOv5 target detection model is obtained; S4. Using the constructed asphalt pavement disease image dataset, the improved YOLOv5 target detection model is preliminarily trained; S5. Based on the improved YOLOv5 target detection model after preliminary training, the real-time database is tested, and the test result data is manually verified. The verification result is imported into the real-time database as a new data set, and the improved YOLOv5 target detection model after preliminary training is corrected and optimized; S6. Based on the dynamic programming method, the pre-selection box is selected by accumulating confidence, and redundant data deduplication is performed on the information of the same disease of the asphalt pavement through the pre-selection box; S7. Based on the deduplicated asphalt pavement disease data, the pixel coordinate system and the real coordinate system are associated by the chessboard method to obtain the actual size of the detection target, and the asphalt pavement disease is evaluated using the morphological fractal method.

2. The intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm according to claim 1 is characterized in that: In step S2, manual annotation includes six types of label information, including transverse cracks, longitudinal cracks, mesh cracks, pits, block repairs and strip repairs.

3. The intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm according to claim 1 is characterized in that: In step S3, the efficient multi-scale attention mechanism preserves accurate location information and integrates contextual information of different scales by modeling long-term dependencies.

4. The intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm according to claim 1 is characterized in that: In step S4, based on the constructed and labeled asphalt pavement disease image dataset, validation set data feedback is provided through the loss function curve, the hyperparameters of the model are adjusted in turn, and the step size of each parameter is dynamically adjusted to perform preliminary training on the improved YOLOv5 target detection model.

5. The intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm according to claim 1 is characterized in that: Step S7 includes three-dimensional modeling, disease segmentation and size assessment; In stereo modeling, the chessboard method is used to correct the image and obtain the parallax image of the asphalt road surface; In the disease segmentation, the crack skeleton of the asphalt pavement is obtained through the refinement algorithm, and the pixels containing the disease are set to 1, and the pixels without the disease are set to 0 to obtain a binary image, and then the area of ​​the disease is obtained; In the size assessment, the size of asphalt pavement defects is assessed based on the principle of triangulation.

6. The intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm according to claim 5 is characterized in that: The chessboard method is used to correct the image, including using the chessboard to calculate the homography matrix; according to the internal camera parameter settings, the height and angle of the camera, the homography matrix is ​​used to project the road surface disease image to obtain a top view of the road surface disease.

7. The intelligent detection method for asphalt pavement defects based on the YOLOv5 algorithm according to claim 5 is characterized in that: Obtaining asphalt road parallax images includes: The disparity image of the road scene is obtained through global modeling, and then the local point disparity and disparity candidate algorithm are obtained by feature point matching. The obtained disparity image is optimized to obtain the asphalt road surface disparity image.

Images