Road surface pit detection method based on improved YOLOv5s
By embedding the coordinate attention mechanism in YOLOv5s and using the MPDIoU border loss function, the accuracy and generalization ability of pavement pit detection are improved, and the problem of poor detection of small pit detection in the prior art is solved.
Patent Information
- Application Number
- CN202411640045.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-18
- Publication Date
- 2025-03-11
AI Technical Summary
In the prior art, it is difficult to effectively detect pits with small areas and blurred edges in pavement pit detection, which are easily confused with pavement patches and debris.
By improving YOLOv5s, embedding the coordinate attention mechanism, and replacing the original CIoU loss function with MPDIoU border loss function, enhancing the target feature extraction capability and detection accuracy.
The detection accuracy and generalization ability of small pits are improved, the identification ability of road pits is enhanced, and missed detection situations are reduced.
Smart Images
Figure CN119600551A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a road pothole detection method based on improved YOLOv5s. Background Art
[0002] A pothole generally refers to an irregular area of the road that is sunken and has a certain depth. Potholes can pose a traffic safety hazard, leading to an increase in traffic accident rates, while also affecting traffic efficiency.
[0003] With the rapid development of artificial intelligence technology, target detection technology based on deep learning has achieved many breakthrough results. Target detection methods based on deep learning are mainly divided into two categories. One is the two-stage detection algorithm based on candidate boxes, which has the characteristics of slow speed and high precision. Typical algorithms include RCNN, Fast R-CNN, Faster R-CNN, etc.; the other is the single-stage detection method based on regression, which has the characteristics of fast speed and low precision. Typical algorithms include SSD and YOLO series. Although the two-stage target detection method R-CNN series based on candidate regions has achieved very good detection results, due to the relatively high precision and speed required for road pothole detection, the speed performance of the R-CNN series of algorithms cannot meet the actual application requirements. The network model parameters of the single-stage target detection algorithm represented by SSD and YOLO are relatively small, and they are superior in real-time performance, but a little unsatisfactory in accuracy. With the continuous update of the YOLO series, the widely used YOLOv5s has both high recognition speed and recognition accuracy, has good application value in road pothole detection, and is suitable for deployment on edge devices such as vehicle systems. However, the YOLOv5s model also has some limitations. For example, it has poor detection results for numerous and inconspicuous small pits, and dense pits are prone to missed detection. The recognition accuracy of YOLOv5s in scenes with small targets needs to be improved.
[0004] In summary, the difficulty of road pothole detection based on traditional methods lies in the irregularity of pothole shapes and different degrees of edge blur. Potholes with larger areas and clear edges are easier to detect, but when the pothole area is small, it is easy to be confused with road patches and debris, making it difficult to detect. Summary of the invention
[0005] The purpose of the present invention is to provide a road pothole detection method based on improved YOLOv5s, so as to solve the problems in the road pothole detection method in the prior art that the pothole shape is irregular and the degree of edge blur is different. The potholes with larger area and clear edges are easier to detect, but when the pothole area is small, it is easy to be confused with road patches and debris and difficult to detect.
[0006] To achieve the above object, the present invention provides a road pothole detection method based on improved YOLOv5s, and the road pothole detection method based on improved YOLOv5s comprises the following steps:
[0007] S1: sharpen the image to be detected in the UDTIRI road pothole dataset;
[0008] S2: Embed the coordinate attention mechanism in the backbone network of the YOLOv5s framework to obtain the YOLOv5s-CA network model;
[0009] S3: Use the MPDIoU border loss function to replace the original CIoU loss function;
[0010] S4: Train the YOLOv5s-CA network model built above on the training set and verify the training process with the validation set;
[0011] S5: Test the performance of the trained YOLOv5s-CA network model on the test set.
[0012] Wherein, in step S1, the Laplace operator is used to perform sharpening processing on the image in the spatial domain.
[0013] Among them, in step S2, the attention mechanism specifically encodes channel relationships and long-term dependencies through precise location information, and the specific operations are divided into Coordinate information embedding and Coordinate Attention generation.
[0014] In step S2, the Coordinate information embedding is specifically to decompose the global pooling into a pair of 1D feature encoding operations:
[0015]
[0016] Among them, in step S2, the specific steps of generating the CoordinateAttention are:
[0017] After the transformation in the Coordinate information embedding, the extracted feature map is concatenated according to the spatial dimension to obtain feature map of the shape;
[0018] The feature map is passed through the F1 convolution transformation function (1×1 convolution) and the nonlinear activation function to generate the f intermediate feature map:
[0019] f=δ(F1([z h ,z w ]));
[0020] Split f into two tensors f according to the spatial dimension h and f w , the shapes are and Then, perform F h and F w Convolution transformation function (1×1 convolution) and Sigmoid activation function to get g h and g w Coordinate attention:
[0021] g h =σ(F h (f h ))
[0022] g w =σ(F w (f w ));
[0023] G h and g w Multiply it with the original input feature to get an output of the same size as the input:
[0024]
[0025] Complete the generation of the CoordinateAttention.
[0026] Among them, the specific steps of step S4 are: configure the network model environment, select the Ubuntu22.04 system and the PyTorch deep learning framework for training, and prepare at least one graphics card that supports cuda acceleration;
[0027] Set the network model parameters:
[0028] According to the size of computer memory and video memory, set the size of YOLOv5s-CA network input image to 640*640, the path corresponding to the data set, the number of recognized types to 1 and the class name pothole, the pre-trained weight file yolov5s.pt, batch_size=16, the number of iterations epoch=300, the initial learning rate lr=0.01 and the learning rate adjustment strategy;
[0029] Load the pre-trained weights and train until the network model converges. Use the validation set for verification during this process. No gradient backpropagation is required during verification. Save the trained weight file after training is completed.
[0030] The specific steps of step S5 are: loading the trained network model weight file, and sending the test set to the trained YOLOv5-CA model to obtain the prediction result;
[0031] Set a confidence threshold to obtain the bounding box coordinates, confidence, and category probability of all predicted boxes that may contain pits;
[0032] Remove redundant detection boxes through non-maximum suppression and draw the final detection results;
[0033] According to the real box position of the test set and the prediction results obtained by the model, the average precision (mAP), accuracy and recall of the model are calculated to systematically evaluate the model.
[0034] The present invention discloses a road pothole detection method based on improved YOLOv5s, which makes the edge of the pothole more prominent and the feature more obvious by sharpening the input image, and improves the lightweight framework of YOLOv5s, improves the backbone network in combination with the coordinate attention mechanism, enhances the target feature extraction capability, and improves the border loss function by using MPDIOU to replace the original CIOU loss function to further improve the detection capability. When the method is used to detect road potholes, the method has stronger generalization capability and higher recognition accuracy than the original YOLOv5s method, and has a stronger detection effect on small potholes. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0036] Figure 1 It is a flow chart of the road pothole detection method based on improved YOLOv5s provided by the present invention.
[0037] Figure 2 This is an image sharpening processing effect diagram provided by the present invention.
[0038] Figure 3 This is a schematic diagram of embedding Coordinate information provided by the present invention.
[0039] Figure 4 It is a schematic diagram of splicing the Coordinate information feature graph provided by the present invention.
[0040] Figure 5 This is the structure of the CA mechanism provided by the present invention.
[0041] Figure 6 It is a schematic diagram of the structure of the improved YOLOv5s-CA model provided by the present invention.
[0042] Figure 7 It is a schematic diagram of the problems existing in the CIoU border loss function used in the prior art provided by the present invention.
[0043] Figure 8 It is a schematic diagram of calculating the MPDIoU border loss function provided by the present invention.
[0044] Fig. 9 It is a schematic diagram of the evaluation results of the UDTIRI dataset provided by the present invention on the YOLOv5 model of the prior art.
[0045] Fig.10 It is a schematic diagram of the evaluation results of the UDTIRI dataset provided by the present invention on the YOLOv5-CA model of the present technical solution.
[0046] Fig.11 It is a schematic diagram of the road pothole detection effect of the YOLOv5 model of the prior art provided by the present invention.
[0047] Fig.12 It is a schematic diagram of the road pothole detection effect of the YOLOv5-CA model of the technical solution provided by the present invention. DETAILED DESCRIPTION
[0048] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0049] See also Figures 1 to 12 The present invention provides a road pothole detection method based on improved YOLOv5s, and the road pothole detection method based on improved YOLOv5s comprises the following steps:
[0050] S1: sharpen the image to be detected in the UDTIRI road pothole dataset;
[0051] S2: Embed the coordinate attention mechanism (CA mechanism) in the backbone network of the YOLOv5s framework to obtain the YOLOv5s-CA network model;
[0052] S3: Use the MPDIoU border loss function to replace the original CIoU loss function;
[0053] S4: Train the YOLOv5s-CA network model built above on the training set and verify the training process with the validation set;
[0054] S5: Test the performance of the trained YOLOv5s-CA network model on the test set.
[0055] In this embodiment, the input image is sharpened to make the edge of the pothole more prominent and the feature more obvious, and the lightweight framework of YOLOv5s is improved by combining the coordinate attention mechanism to improve the backbone network to enhance the target feature extraction capability. At the same time, the bounding box loss function is improved and the original CIOU loss function is replaced by MPDIOU to further improve the detection capability. When the above method is used to detect potholes in the road, it has stronger generalization ability and higher recognition accuracy than the original YOLOv5s method, and has a stronger detection effect on small potholes.
[0056] In step S1, the UDTIRI road pothole dataset is a road pothole dataset proposed by the MIAS Group of Tongji University. The dataset has well-annotated ground truth data and provides multiple formats (such as VOC, COCO, YOLO, etc.) for road pothole detection tasks. The dataset includes 1,000 pothole images, which are taken under different lighting and humidity conditions.
[0057] Furthermore, the technical solution uses the Laplace operator to directly sharpen the image in the spatial domain. The Laplace operator is an edge operator of the second-order derivative and can be simplified by differential approximation. The formula is:
[0058]
[0059] The 8-neighborhood convolution kernel used in this technical solution is:
[0060] 1 1 1 1 -8 1 1 1 1
[0061] Laplace sharpening emphasizes edges by enhancing the second-order derivatives in an image.
[0062] The basic principle of Laplace sharpening: when the grayscale of the central pixel of a neighborhood is lower than the average grayscale of other pixels in its neighborhood, the grayscale of this central pixel should be further reduced; when it is higher, the grayscale of the central pixel should be further increased, thereby achieving image sharpening processing.
[0063] Laplace's sharpening formula:
[0064]
[0065] Among them, f(x,y) represents the original image, and g(x,y) represents the sharpened image.
[0066] For the 8-neighborhood convolution kernel:
[0067] -f(x+1,y)-f(x-1,y+1)-f(x,y+1)-f(x+1,y+1)
[0068] The template is represented as:
[0069]
[0070] The image is sharpened by the 8-neighborhood Laplacian operator. Figure 2 shown.
[0071] In step S2, the attention mechanism specifically encodes channel relationships and long-term dependencies through precise location information, and the specific operations are divided into Coordinate information embedding and Coordinate Attention generation.
[0072] The Coordinate information embedding is specifically to decompose the global pooling into a pair of 1D feature encoding operations:
[0073]
[0074] Specifically, given an input feature map of size C×H×W, we first use a pooling kernel of size (H, 1) or (1, W) to encode each channel along the horizontal and vertical coordinates, respectively, as Figure 3 Therefore, the output of the cth channel with height h can be expressed as:
[0075]
[0076] Similarly, the output of the cth channel with width w can be written as:
[0077]
[0078] The above two transformations aggregate features along two spatial directions respectively to obtain a pair of direction-aware feature maps.
[0079] The pair of direction-aware feature maps generated in this way enables CA to capture long-range dependencies within a single channel and also helps preserve precise location information, allowing the network to locate objects more accurately.
[0080] In step S2, the specific steps of generating the CoordinateAttention are:
[0081] After the transformation in the Coordinate information embedding, the extracted feature map is concatenated according to the spatial dimension to obtain The feature map of the shape, such as Figure 4 As shown, r is used to control the reduction rate of the block;
[0082] The feature map is passed through the F1 convolution transformation function (1×1 convolution) and the nonlinear activation function to generate the f intermediate feature map:
[0083] f=δ(F1([z h ,z w ]));
[0084] Split f into two tensors f according to the spatial dimension h and f w , the shapes are and Then, perform F h and F w Convolution transformation function (1×1 convolution) and Sigmoid activation function to get g h and g w Coordinate Attention:
[0085] g h =σ(F h (f h ))
[0086] g w =σ(F w (f w ));
[0087] G h and g w Multiply it with the original input feature to get an output of the same size as the input:
[0088]
[0089] Complete the generation of the CoordinateAttention.
[0090] In this embodiment, the technical solution embeds the position information into the channel attention by introducing a coordinate attention CA module into the YOLOv5s backbone network. Different from the channel attention that converts the input into a single feature vector through 2D global pooling, CA decomposes the channel attention into two 1D feature encoding processes that aggregate features along different directions. Its advantage is that it can capture long-range dependencies along one spatial direction and retain precise position information along another spatial direction. Then, the generated feature maps are encoded separately to form a pair of direction-aware and position-sensitive feature maps, which can be complementarily applied to the input feature map to enhance the representation of the target of interest. The CA structure is as follows: Figure 5 As shown, the CA module is Figure 6The figure shows the backbone feature extraction part embedded in the YOLOv5s network. Specifically, the CA module is embedded before the SPP module, and the features extracted by the CA module are pooled and fused by the SPP.
[0091] Since the YOLOv5s border loss in the prior art is calculated using CIoU Loss, CIoU reflects the difference in aspect ratios rather than the actual gap between the length, width, position and confidence, this will cause certain problems in detection, such as Figure 7 As shown in the figure on the left, the outer frame is the actual detection frame, and the inner frame is the prediction frame. When the center points of the two frames coincide and the aspect ratio is consistent, CIoU will fail. In addition, in the figure on the right, when multiple prediction frames overlap in a large area, it cannot reflect the actual situation.
[0092] In step S3, in order to make up for the shortcomings of CIoU, this technical solution uses the MPDIoU border loss function to replace the original loss function. MPDIoU is a new bounding box similarity comparison metric based on the minimum point distance, which directly minimizes the distance between the upper left corner and the lower right corner of the predicted bounding box and the actual annotated bounding box. MPDIoU includes all relevant factors considered in the existing loss function, namely overlapping or non-overlapping areas, center point distance, width and height deviation, and simplifies the calculation process, such as Figure 8 shown.
[0093] The specific steps of step S4 are: configure the network model environment, select the Ubuntu22.04 system and the PyTorch deep learning framework for training, and prepare at least one graphics card that supports cuda acceleration;
[0094] Set the network model parameters:
[0095] According to the size of computer memory and video memory, set the size of YOLOv5s-CA network input image to 640*640, the path corresponding to the data set, the number of recognized types to 1 and the class name pothole, the pre-trained weight file yolov5s.pt, batch_size=16, the number of iterations epoch=300, the initial learning rate lr=0.01 and the learning rate adjustment strategy;
[0096] Load the pre-trained weights and train until the network model converges. Use the validation set for verification during this process. No gradient backpropagation is required during verification. Save the trained weight file after training is completed.
[0097] In this embodiment, during the training phase, each bounding box B predicted by the model is forced to prd =[x prd ,y prd ,w prd ,hprd ] T Close to its true bounding box B gt =[x gt ,y gt ,w gt ,h gt ] T :
[0098]
[0099] Among them, B gt is the set of actual bounding boxes; Θ is the parameter of the deep model used for regression; the typical form of L is L n -norm.
[0100] The loss function formula of MPDIoU is as follows:
[0101]
[0102]
[0103]
[0104] L MPDIoU =1-MPDIoU.
[0105] The specific steps of step S5 are: loading the trained network model weight file, sending the test set to the trained YOLOv5-CA model to obtain the prediction result;
[0106] Set a confidence threshold to obtain the bounding box coordinates, confidence, and category probability of all predicted boxes that may contain pits;
[0107] Remove redundant detection boxes through non-maximum suppression and draw the final detection results;
[0108] According to the real box position of the test set and the prediction results obtained by the model, the average precision mAP, accuracy and recall rate of the model are calculated, and the model is systematically evaluated. The evaluation results are shown in the following table. Fig. 9 and Fig.10 shown.
[0109] Model P R <![CDATA[mAP 0.5 ]]> <![CDATA[mAP 0.5:0.95 ]]> YOLOv5s 0.6572 0.8151 0.778 0.4358 YOLOv5s-CA 0.729 0.8121 0.8215 0.4325
[0110] From the above table, Fig. 9 and Fig.10 It can be seen that this technical solution is basically consistent with the original YOLOv5 model in terms of the two evaluation indicators of recall rate R and mAP0.5:0.95, but has a significant improvement in the two indicators of accuracy P and mAP0.5 compared with the original method.
[0111] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.
Claims
1. A road pothole detection method based on improved YOLOv5s, characterized in that: The steps include: S1: sharpen the image to be detected in the UDTIRI road pothole dataset; S2: Embed the coordinate attention mechanism in the backbone network of the YOLOv5s framework to obtain the YOLOv5s-CA network model; S3: Use the MPDIoU border loss function to replace the original CIoU loss function; S4: Train the YOLOv5s-CA network model built above on the training set and verify the training process with the validation set; S5: Test the performance of the trained YOLOv5s-CA network model on the test set.
2. The road pothole detection method based on improved YOLOv5s according to claim 1, characterized in that: In step S1, the Laplace operator is used to sharpen the image in the spatial domain.
3. The road pothole detection method based on improved YOLOv5s as claimed in claim 2, characterized in that: In step S2, the attention mechanism specifically encodes channel relationships and long-term dependencies through precise location information, and the specific operations are divided into Coordinate information embedding and Coordinate Attention generation.
4. The road pothole detection method based on improved YOLOv5s as claimed in claim 3, characterized in that: In step S2, the Coordinate information is embedded by decomposing the global pooling into a pair of 1D feature encoding operations:
5. The road pothole detection method based on improved YOLOv5s as claimed in claim 4, characterized in that: In step S2, the specific steps of generating the CoordinateAttention are: After the transformation in the Coordinate information embedding, the extracted feature map is concatenated according to the spatial dimension to obtain feature maps of shapes; The feature map is passed through the F1 convolution transformation function (1×1 convolution) and the nonlinear activation function to generate the f intermediate feature map: f=δ(F1([z h ,z w ])); Split f into two tensors f according to the spatial dimension h and f w , the shapes are and Then, perform F h and F w Convolution transformation function (1×1 convolution) and Sigmoid activation function to get g h and g w Coordinate attention: g h =σ(F h (f h )) g w =σ(F w (f w )); G h and g w Multiply it with the original input feature to get an output of the same size as the input: Complete the generation of the CoordinateAttention.
6. The road pothole detection method based on improved YOLOv5s as claimed in claim 5, characterized in that: The specific steps of step S4 are: configure the network model environment, select the Ubuntu22.04 system and the PyTorch deep learning framework for training, and prepare at least one graphics card that supports cuda acceleration; Set the network model parameters: According to the size of computer memory and video memory, set the size of YOLOv5s-CA network input image to 640*640, the path corresponding to the data set, the number of recognized types to 1 and the class name pothole, the pre-trained weight file yolov5s.pt, batch_size=16, the number of iterations epoch=300, the initial learning rate lr=0.01 and the learning rate adjustment strategy; Load the pre-trained weights and train until the network model converges. Use the validation set for verification during this process. No gradient backpropagation is required during verification. Save the trained weight file after training is completed.
7. The road pothole detection method based on improved YOLOv5s according to claim 6, characterized in that: The specific steps of step S5 are: loading the trained network model weight file, sending the test set to the trained YOLOv5-CA model to obtain the prediction result; Set a confidence threshold to obtain the bounding box coordinates, confidence, and category probability of all predicted boxes that may contain pits; Remove redundant detection boxes through non-maximum suppression and draw the final detection results; According to the real box position of the test set and the prediction results obtained by the model, the average precision (mAP), accuracy and recall of the model are calculated to systematically evaluate the model.