Road disease detection method and device based on improved YOLOv5

Abstract

The invention relates to a road disease detection method and device based on improved YOLOv5, and the method comprises the steps: obtaining a road image, carrying out the preprocessing of the road image, inputting the preprocessed road image into a trained road disease detection model, and obtaining a detection result; wherein the road disease detection model comprises the following steps: inserting an MASE module between a fourth network layer and a sixteenth network layer of YOLOv5; an MASE module is inserted between the ninth network layer and the tenth network layer; an MASE module is inserted between the fourteenth network layer and the nineteenth network layer; the output characteristic of the sixth network layer is bridged to the input end of the nineteenth network layer; the output characteristic of the eighth network layer is bridged to the input end of the 22th network layer; and obtaining a road disease detection model. According to the method, the fusion effect of the multi-scale features is enhanced, and the problem of loss of tiny disease features in multiple down-sampling stages is solved.

Description

Technical Field

[0001] The present invention belongs to the field of target detection, and in particular relates to a road damage detection method and device based on improved YOLOv5. Background Art

[0002] Road maintenance is essential to ensure smooth and safe urban traffic. Maintaining good road conditions can improve driving comfort and safety and reduce vehicle wear and tear and accidents. Road defect detection is an important aspect of road maintenance, which can solve problems such as cracks and potholes, which affect driving comfort and may cause traffic accidents. Timely and accurate detection and repair of these defects are essential to ensure road safety and extend the service life of roads. Ignoring these problems may make the problems worse under natural or human influences. Therefore, road defect detection has become an important part of road maintenance and traffic management.

[0003] Traditional pavement disease detection methods mainly rely on manual inspection and algorithms based on traditional image processing and machine learning. Although manual inspection methods are intuitive and highly accurate, they are inefficient and subjective, easily affected by human factors, and have high labor costs. Based on traditional image processing and machine learning algorithms, such as support vector machines, they rely on manually designed features and rules. This human factor directly affects the detection performance, making it difficult to cope with complex and changing road environments, and the detection accuracy and robustness are limited. With the rapid development of deep learning technology, object detection methods based on convolutional neural networks (CNNs), such as single-stage methods, directly perform object classification and bounding box regression on images. The YOLO series of algorithms and SSD are typical representatives of single-stage methods. They maintain the accuracy of the calculation speed as much as possible. As a representative, the YOLO series of models are widely used in various object detection tasks with their high-speed and high-precision detection performance. Among them, the YOLOv5 model further improves the detection accuracy while maintaining the detection speed, and is one of the cutting-edge methods in the current field of object detection.

[0004] However, due to the complexity of defects, there are several challenges when using single-stage object detection methods for road defect detection: (1) Single-stage methods perform poorly when detecting small targets because these methods make predictions on high-level feature maps, which have low resolution and are prone to loss of small target information. In road defect detection, the identification of tiny defects is particularly difficult because these defects occupy fewer pixels in the image and are easily ignored or misdetected; (2) Compared with two-stage methods, single-stage methods are usually slightly inferior in detection accuracy. This is because single-stage detectors perform target classification and bounding box regression simultaneously in one forward propagation, making it difficult to fully process details in complex scenes. When detecting multi-scale and irregular road defects, the accuracy may not be ideal. In particular, in road defect detection, due to the irregularity of defect shapes and the diversity of backgrounds, the foreground and background information are easily confused, resulting in unstable detection results. Summary of the invention

[0005] In order to solve the problems existing in the background technology, one aspect of the present invention provides a road damage detection algorithm based on improved YOLOv5 to improve the accuracy and robustness of road damage detection and ensure the reliability of the detection system in a complex environment, including: acquiring a road image and preprocessing it, inputting the preprocessed road image into a trained road damage detection model to obtain a detection result; wherein the road damage detection model includes: inserting a MASE module between the 4th network layer and the 16th network layer of YOLOv5; inserting a MASE module between the 9th network layer and the 10th network layer; inserting a MASE module between the 14th network layer and the 19th network layer; connecting the output features of the 6th network layer to the input end of the 19th network layer; connecting the output features of the 8th network layer to the input end of the 22nd network layer; and obtaining a road damage detection model.

[0006] Preferably, the step of connecting the output features of the 6th network layer to the input of the 19th network layer includes: first subjecting the output features of the 6th network layer to a 1x1 convolution, and adjusting the number of channels to be consistent with the remaining features of the input 19th network layer; and concatenating the features adjusted by the 1x1 convolution with the remaining features of the input 19th network layer in the channel dimension at the 19th network layer.

[0007] Preferably, the method of connecting the output features of the 8th network layer to the input end of the 22nd network layer includes: first subjecting the output features of the 8th network layer to 1x1 convolution, adjusting the number of channels to be consistent with the remaining features of the input 22nd network layer; and splicing the features adjusted by 1x1 convolution with the remaining features of the input 22nd network layer in the channel dimension at the 22nd network layer.

[0008] Preferably, the MASE module includes: a channel attention mechanism module and a spatial attention mechanism module; the channel attention mechanism module performs attention processing on the input feature map to obtain a channel attention weight matrix; the spatial attention mechanism module performs attention processing on the input feature map to obtain a spatial attention weight matrix; the channel attention weight matrix and the spatial attention weight matrix are multiplied to obtain a comprehensive weight matrix; the comprehensive weight matrix and the input features of the MASE module are multiplied to obtain the output features of the MASE module.

[0009] Preferably, the channel attention mechanism module first performs global average pooling or global maximum pooling on the input features of the MASE module to generate a 1*1*C vector z1, where C represents the number of channels of the MASE module input features; the vector z1 is input into two fully connected layers to obtain a 1*1*C channel attention weight matrix. There is a hyperparameter R between the two fully connected layers. After passing through the first fully connected layer, the dimension of the vector z1 changes from (1*1*C) to (1*1*C / R), and then passes through the second fully connected layer, the dimension changes from (1*1*C / R) to (1*1*C) channel attention weight matrix; the activation function of the first fully connected layer is ReLU, and the activation function of the second fully connected layer is Sigmoid.

[0010] Preferably, the spatial attention mechanism module first compresses the number of channels to C / ratio through a 1×1 convolution on the input features of the MASE module, and then activates it through ReLU; then restores the number of channels to the original C through another 1×1 convolution; and finally generates a spatial attention weight matrix through a Softmax activation function.

[0011] Preferably, the road damage detection model is trained using a HIoU loss function, which is expressed as:

[0012]

[0013] Among them, IoU represents the intersection-over-union ratio of the predicted box and the real box in the road disease area; A represents the real box of the road disease area, and B represents the predicted box of the road disease area; c represents the diagonal length of the minimum enclosing rectangle containing the real box A and the predicted box B; v is used to measure the similarity of the aspect ratio of the real box A and the predicted box B; A c Represents the area of ​​the minimum enclosing rectangle of the real box A and the predicted box B; ρ 2 represents the distance between the center of the real box A and the predicted box B; w B Indicates the width of the prediction box B; h B Indicates the height of the prediction box B; w A Indicates the width of the real box A; h A Represents the width of the ground-truth box A.

[0014] Another aspect of the present invention provides a road damage detection device based on improved YOLOv5, comprising a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory, and is used to execute the computer program stored in the memory, so that the road damage detection device based on improved YOLOv5 performs the road damage detection method based on improved YOLOv5.

[0015] Another aspect of the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the road damage detection method based on improved YOLOv5.

[0016] The present invention has at least the following beneficial effects

[0017] The present invention improves the feature extraction capability in complex backgrounds by integrating an improved Mixed Attention Squeeze-and-Excitation (MASE) module, effectively enhances the ability to distinguish foreground and background information, and solves the problem of occlusion and confusion of disease features by diverse background elements. In addition, the Cross-layer Enhancement Network (CLEN) is introduced to improve the feature fusion framework, enhance the fusion effect of multi-scale features, and solve the problem of loss of tiny disease features in multiple downsampling stages. At the same time, a Hybrid IoU Loss (HIoU) function is designed to calculate the loss more comprehensively, thereby meeting the detection challenges of irregular-shaped diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0019] Figure 2 Schematic diagram of the framework structure of the road disease detection model of the present invention;

[0020] Figure 3 It is a schematic diagram of the framework structure of the MASE module of the present invention;

[0021] Figure 4 The training result diagram of the present invention on the CWNU dataset;

[0022] Figure 5 Detection heat map after introducing MASE on the CWNU dataset;

[0023] Figure 6 Comparison of detection before and after the introduction of CLEN on the CWNU dataset;

[0024] Figure 7 Comparison of detection before and after the introduction of HIoU on the CWNU dataset;

[0025] Figure 8 The training result diagram of the present invention on the RDD2022 dataset;

[0026] Fig. 9 Detection heat map after introducing MASE on the RDD2022 dataset;

[0027] Fig.10 Comparison of detection before and after the introduction of CLEN on the RDD2022 dataset;

[0028] Fig.11 Detection comparison before and after introducing HIoU on the RDD2022 dataset. DETAILED DESCRIPTION

[0029] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0030] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0031] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0032] See also Figure 1 and Figure 2The present invention provides a road damage detection algorithm based on improved YOLOv5 to improve the accuracy and robustness of road damage detection and ensure the reliability of the detection system in complex environments, including: acquiring a road image and preprocessing it, inputting the preprocessed road image into a trained road damage detection model to obtain a detection result; wherein the road damage detection model includes: inserting a MASE module between the 4th network layer and the 16th network layer of YOLOv5; inserting a MASE module between the 9th network layer and the 10th network layer; inserting a MASE module between the 14th network layer and the 19th network layer; connecting the output features of the 6th network layer to the input end of the 19th network layer; connecting the output features of the 8th network layer to the input end of the 22nd network layer; and obtaining a road damage detection model.

[0033] In this embodiment, the road disease detection model is trained using the CWNU dataset or the RDD2022 dataset. In the CWNU dataset, the disease identification uses the x1x2y1y2 coordinate system format. In order to adapt the dataset to the requirements of the YOLO algorithm, this embodiment converts these identifiers into the YOLO common TXT format. The conversion process ensures the accuracy of the annotation information and provides a standardized data input format for subsequent model training. The CWNU dataset is divided into a training set and a validation set at a ratio of 8:2. The final image contains 630 cracks, 390 transverse cracks, 425 longitudinal cracks, 1120 block cracks, 105 oil spills, 411 looseness, 96 potholes, and 806 repairs.

[0034] In the RDD2022 dataset, the disease identification is annotated in XML format. In order to adapt the dataset to the requirements of the YOLO algorithm, this embodiment converts these identifications into the TXT format commonly used by YOLO. This conversion process ensures the accuracy and consistency of the annotation information and provides a standardized data input format for subsequent model training. The RDD2022 dataset is divided into a training set and a validation set in a ratio of 7:3. The training set contains 3060 images and the validation set contains 1313 images. The longitudinal cracks, transverse cracks, cracks, potholes, and repair disease types of these images are trained.

[0035] Preferably, the step of connecting the output features of the 6th network layer to the input of the 19th network layer includes: first subjecting the output features of the 6th network layer to a 1x1 convolution, and adjusting the number of channels to be consistent with the remaining features of the input 19th network layer; and concatenating the features adjusted by the 1x1 convolution with the remaining features of the input 19th network layer in the channel dimension at the 19th network layer.

[0036] Preferably, the method of connecting the output features of the 8th network layer to the input end of the 22nd network layer includes: first subjecting the output features of the 8th network layer to 1x1 convolution, adjusting the number of channels to be consistent with the remaining features of the input 22nd network layer; and splicing the features adjusted by 1x1 convolution with the remaining features of the input 22nd network layer in the channel dimension at the 22nd network layer.

[0037] In the original YOLOV5 model, PANet is prone to lose small-scale features during multiple downsampling processes, resulting in poor detection of small-scale diseases. Therefore, cross-layer connections are introduced in this embodiment to strengthen the fusion of high-level semantic information and low-level detail features, greatly improving the model's ability to detect micro-disease features. The 6th network layer in the YOLOV5 model is located in the early stage of the network, focusing on extracting basic features such as edges and textures. This layer maintains high resolution by minimizing downsampling, which is crucial for identifying micro-diseases; the 19th network layer is located in the middle of the network, while retaining detail features, it also incorporates some semantic information. This intermediate layer effectively summarizes information from each layer and provides a richer feature representation for the subsequent fusion process. By combining semantic and detail features at this stage, cross-layer connections enhance the model's ability to accurately capture and retain micro-disease features, thereby improving the ability to detect small target diseases. Similarly, the 8th and 22nd network layers further support cross-layer connection strategies. The improved feature fusion framework can significantly improve the model's detection accuracy and robustness for small target disease features, thereby improving the overall detection performance.

[0038] See also Figure 3 Preferably, the MASE module includes: a channel attention mechanism module and a spatial attention mechanism module; the channel attention mechanism module performs attention processing on the input feature map to obtain a channel attention weight matrix; the spatial attention mechanism module performs attention processing on the input feature map to obtain a spatial attention weight matrix; the channel attention weight matrix and the spatial attention weight matrix are multiplied to obtain a comprehensive weight matrix; the comprehensive weight matrix and the input features of the MASE module are multiplied to obtain the output features of the MASE module.

[0039] Preferably, the channel attention mechanism module first performs global average pooling or global maximum pooling on the input features of the MASE module to generate a 1*1*C vector z1, where C represents the number of channels of the MASE module input features; the vector z1 is input into two fully connected layers to obtain a 1*1*C channel attention weight matrix. There is a hyperparameter R between the two fully connected layers. After passing through the first fully connected layer, the dimension of the vector z1 changes from (1*1*C) to (1*1*C / R), and then passes through the second fully connected layer, the dimension changes from (1*1*C / R) to (1*1*C) channel attention weight matrix; the activation function of the first fully connected layer is ReLU, and the activation function of the second fully connected layer is Sigmoid.

[0040] Preferably, the spatial attention mechanism module first compresses the number of channels to C / ratio through a 1×1 convolution on the input features of the MASE module, and then activates it through ReLU; then restores the number of channels to the original C through another 1×1 convolution; and finally generates a spatial attention weight matrix through a Softmax activation function.

[0041] This embodiment improves the traditional SE (Squeeze-and-Excitation) module to design a MASE module. First, a spatial attention mechanism is introduced to retain spatial information. This mechanism generates spatial weights by applying convolution and activation functions to the feature map, thereby retaining more spatial information. This enhancement function can achieve more accurate feature extraction when processing disease images with background diversity and irregular shapes. Secondly, considering the global relationship between channels, the patented method incorporates the spatial position relationship into the feature map. Applying the spatial attention mechanism to the feature map can more effectively distinguish different areas in the same feature map. When complex backgrounds and irregularly shaped diseases exist at the same time, it can better distinguish these areas, thereby improving the accuracy of feature extraction. When the improved attention mechanism module is integrated into the YOLOv5 network, the present invention pays special attention to the importance of its placement. After many experiments, the module was finally placed in the following three positions: between the 4th network layer and the 16th network layer, between the 9th network layer and the 10th network layer, and between the 14th network layer and the 19th network layer.

[0042] Preferably, the road damage detection model is trained using a HIoU loss function, which is expressed as:

[0043]

[0044] Among them, IoU represents the intersection-over-union ratio of the predicted box and the real box in the road disease area; A represents the real box of the road disease area, and B represents the predicted box of the road disease area; c represents the diagonal length of the minimum enclosing rectangle containing the real box A and the predicted box B; v is used to measure the similarity of the aspect ratio of the real box A and the predicted box B; A c Represents the area of ​​the minimum enclosing rectangle of the real box A and the predicted box B; ρ 2 represents the distance between the center of the real box A and the predicted box B; w B Indicates the width of the prediction box B; h B Indicates the height of the prediction box B; w A Indicates the width of the real box A; h A Represents the width of the ground-truth box A.

[0045] In this embodiment, the overlap between the predicted box and the real box, the center point distance, and the consistency of the aspect ratio are taken into consideration. The constructed HIoU loss function can comprehensively consider the geometric relationship and overlap between the predicted box and the real box, thereby improving the accuracy and robustness of disease detection when dealing with irregular disease shapes.

[0046] Experimental simulation

[0047] First, the data is loaded and enhanced; after the two data sets are divided into training sets and test sets according to the ratio, they are transformed into 640*640 size and input into the network, and then mosiac data enhancement is used. According to the set number of training rounds, the method of the present invention conducts detailed training and testing experiments. Specifically, the number of training rounds is set to 300 rounds, the batch size is 32, and the learning rate is 0.0005. Under these conditions, the model shows good convergence and stability during the training process.

[0048] After training begins, the calculated loss value is used to perform a back propagation algorithm based on the preset parameters to gradually adjust and optimize the various parameter values ​​in the network. This process is repeated over and over again, and through multiple iterations, the performance of the model on the training data is gradually improved, ultimately achieving high-precision detection of road damage.

[0049] After the model training is completed, the weight file obtained by training is loaded and the test data set is tested. After the test is completed, a folder will be generated, in which the test results, that is, the image files after the test, are saved. Through these image files, the disease targets can be located and classified for visual display. Through visual comparison, it can be clearly seen whether the improved model of the present invention can solve the existing challenges.

[0050] Experimental conditions:

[0051] The hardware and software environment configurations of the present invention are as follows: CPU: 12 cores, Xeon(R) Platinum 8255C; GPU: NVIDIA RTX 2080Ti (11GB); operating system: Ubuntu 18.04; Python: 3.8; PyTorch: 1.7.0; CUDA: 11.0; cuDNN: 8.0.4. The relevant training parameters are set as follows: input image size: 640×640; Batch Size: 32; Epoch: 300; Early Stopping: 50; Weight Decay: 0.0005; Momentum: 0.937; Mosaic Enhancement: True; Mixup: False.

[0052] Experimental data:

[0053] The data sets used in the present invention are CWNU data set and RDD2022 data set. The first CWNU data set is obtained by collecting images of highway pavement by an image acquisition vehicle. The resolution of each image is 3692×2147, and it contains a total of 8 types of pavement diseases. They are cracks, transverse cracks, longitudinal cracks, block cracks, oil spills, looseness, potholes, and repairs. After using the script to obtain the universal txt format of yolo, the training set and the validation set are divided according to the ratio of 8:2. The final image contains 630 cracks, 390 transverse cracks, 425 longitudinal cracks, 1120 block cracks, 105 oil spills, 411 looseness, 96 potholes, and 806 repairs.

[0054] The second RDD2022 dataset comes from the Global Road Damage Detection Challenge. The present invention only uses all the datasets in China_Drone and China_MotorBike, which are from Dongji Avenue in Nanjing, China and Jiulong Lake Campus of Southeast University, respectively. The images are included in various situations such as normal light, shadow environment, and stains. The resolution of each image is 512x512, and a total of 4373 pictures are included. After using the script to convert the XML format to the universal txt format of Yolo, the training set and the verification set are divided according to the ratio of 7:3. The training set contains 3060 images and the verification set contains 1313 images. In the present invention, the longitudinal cracks, transverse cracks, cracks, potholes, and repair disease types of these images are trained.

[0055] Evaluation indicators:

[0056] In the experiment, the following indicators are used to verify the improved effect of the present invention: mean average precision (mAP), precision (Precision), recall rate (Recall), number of parameters (Params) and the number of floating-point operations (Giga Floating-point Operations, GFLOPs) performed by the model during the reasoning process.

[0057] Ablation experiment:

[0058] Ablation experiments are conducted under the improved model of the present invention to verify whether the various improvements proposed by the present invention are indeed effective. Table 1 and Table 2 show the ablation experiment data on the CWNU dataset and the RDD2022 dataset respectively. Figure 4 , Figure 8 The training results of the original method and the proposed method are compared on the CWNU dataset and the RDD2022 dataset, respectively. The horizontal axis represents the training rounds, and the vertical axis represents the corresponding indicators.

[0059] Table 1 Ablation experiments on the CWNU dataset

[0060]

[0061]

[0062] From Table 1, we can see that after introducing MASE, CLEN (Cross-layer Enhancement Network, CLEN, cross-layer connection), HIoU and combining them into the model, all indicators of the model are significantly improved. The average accuracy of the original model is 46.7%, and the average accuracy of the improved model is 57.1%, an increase of 10.4 percentage points; the precision is increased from 55% to 65.1%, an increase of 10.1 percentage points; the recall rate is increased from 46.9% to 53.6%, an increase of 6.7 percentage points.

[0063] After the introduction of the MASE module, the recall rate of the model increased significantly to 51.6%, and the mAP@0.5 increased to 51.0%, indicating that the MASE module effectively enhances the feature extraction capability under complex backgrounds. It can greatly improve detail processing when various intricate background elements mask or confuse disease features. After the introduction of HIoU, the accuracy of the model and mAP@0.5 increased to 59.8% and 49.1%, respectively, showing the significant improvement of HIoU in bounding box positioning accuracy. After the introduction of CLEN, the mAP@0.5 of the model increased to 52.9%, indicating that CLEN significantly enhanced the processing capability of small-scale features by improving feature fusion.

[0064] At the same time, in order to further prove the above conclusions, the detection effects before and after the improvement are visualized. The present invention visualizes the improved results for different improvement points, such as Figure 5 , Figure 6 , Figure 7 As shown in the figure, the detection images after the introduction of MASE, CLEN and HIoU are respectively shown. It can be seen that compared with the original method, the method of the present invention can significantly improve the road disease detection effect and enhance the accuracy.

[0065] Table 2 Ablation experiments on the RDD2022 dataset

[0066]

[0067]

[0068] Similarly, it can be seen from Table 2 that after MASE, CLEN, and HIoU were introduced on the RDD2022 dataset and combined with each other to focus on the model, the average accuracy of the original model was 83.1%, and the average accuracy of the improved model was 85.5%, an increase of 2.3 percentage points, and the recall rate increased from 79.1% to 81.9%, an increase of 2.8 percentage points. It can be seen that the method of the present invention is not only useful on a certain dataset, but also has universality on different datasets. It also improves the problems of difficult small target detection, insufficient detection accuracy, and false detection or missed detection in complex backgrounds. The specific effect diagram is as follows Fig. 9 , Fig.10 , Fig.11 As shown in the figure, it is also shown that the introduction of the MASE module successfully enhances the feature extraction capability under complex backgrounds, solving the problem of various complex background elements covering or confusing disease features; the introduction of CLEN solves the problem of missing detection due to the disappearance of tiny disease features after multiple downsampling; the introduction of HIoU strengthens the positioning and classification capabilities of irregular disease detection.

[0069] In order to further demonstrate the superiority of the present invention and to more comprehensively evaluate the performance of the improved model of the present invention, different networks are selected to compare the algorithm of the present invention. Table 3 and Table 4 show the comparison experiments of different networks on the CWNU dataset and the RDD2022 dataset respectively.

[0070] Table 3 Performance comparison with different networks on the CWNU dataset

[0071]

[0072]

[0073] Table 4 Performance comparison with different networks on the RDD2022 dataset

[0074]

[0075] From Table 3 and Table 4, we can see that after the comparative experiments of different networks, while having the highest mAP, the method of the present invention also performs very well in terms of model parameter quantity and computational complexity. This proves the effectiveness and superiority of the road damage detection algorithm based on the improved YOLOv5 implemented by the present invention.

[0076] In summary, the present invention proposes a road damage detection algorithm based on improved YOLOv5. The model of the present invention combines the MASE module, CLEN network and HIoU loss function, which can achieve efficient and accurate detection in complex backgrounds and irregularly shaped damage. Experimental results show that the model of the present invention significantly improves the feature extraction capability, optimizes feature fusion, and enhances the detection frame positioning accuracy. These advances effectively solve the problems related to shadow occlusion, complex road surface and small target damage detection.

[0077] Another aspect of the present invention provides a road damage detection device based on improved YOLOv5, comprising a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory, and is used to execute the computer program stored in the memory, so that the road damage detection device based on improved YOLOv5 performs the road damage detection method based on improved YOLOv5.

[0078] Another aspect of the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the road damage detection method based on improved YOLOv5.

[0079] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A road damage detection method based on improved YOLOv5, characterized in that: include: A road image is obtained and preprocessed, and the preprocessed road image is input into a trained road damage detection model to obtain a detection result; wherein the road damage detection model includes: inserting a MASE module between the 4th network layer and the 16th network layer of YOLOv5; inserting a MASE module between the 9th network layer and the 10th network layer; inserting a MASE module between the 14th network layer and the 19th network layer; connecting the output feature of the 6th network layer to the input end of the 19th network layer; connecting the output feature of the 8th network layer to the input end of the 22nd network layer; and obtaining a road damage detection model.

2. A road damage detection method based on improved YOLOv5 according to claim 1, characterized in that: The step of cross-connecting the output features of the 6th network layer to the input end of the 19th network layer includes: firstly subjecting the output features of the 6th network layer to a 1x1 convolution, and adjusting the number of channels to be consistent with the remaining features of the input 19th network layer; and splicing the features adjusted by the 1x1 convolution with the remaining features of the input 19th network layer in the channel dimension at the 19th network layer.

3. A road damage detection method based on improved YOLOv5 according to claim 1, characterized in that: The method of connecting the output features of the 8th network layer to the input end of the 22nd network layer includes: first subjecting the output features of the 8th network layer to a 1x1 convolution, adjusting the number of channels to be consistent with the remaining features of the input 22nd network layer; and splicing the features adjusted by the 1x1 convolution with the remaining features of the input 22nd network layer in the channel dimension at the 22nd network layer.

4. A road damage detection method based on improved YOLOv5 according to claim 1, characterized in that: The MASE module includes: a channel attention mechanism module and a spatial attention mechanism module; the channel attention mechanism module performs attention processing on the input feature map to obtain a channel attention weight matrix; the spatial attention mechanism module performs attention processing on the input feature map to obtain a spatial attention weight matrix; the channel attention weight matrix and the spatial attention weight matrix are multiplied to obtain a comprehensive weight matrix; the comprehensive weight matrix and the input features of the MASE module are multiplied to obtain the output features of the MASE module.

5. A road damage detection method based on improved YOLOv5 according to claim 4, characterized in that: The channel attention mechanism module first performs global average pooling or global maximum pooling on the input features of the MASE module to generate a 1*1*C vector z1, where C represents the number of channels of the MASE module input features; the vector z1 is input into two fully connected layers to obtain a 1*1*C channel attention weight matrix. There is a hyperparameter R between the two fully connected layers. After passing through the first fully connected layer, the dimension of the vector z1 changes from (1*1*C) to (1*1*C / R), and then passes through the second fully connected layer, the dimension changes from (1*1*C / R) to (1*1*C) channel attention weight matrix; the activation function of the first fully connected layer is ReLU, and the activation function of the second fully connected layer is Sigmoid.

6. A road damage detection method based on improved YOLOv5 according to claim 4, characterized in that: The spatial attention mechanism module first compresses the number of channels to C / ratio through a 1×1 convolution on the input features of the MASE module, and then activates it through ReLU; then restores the number of channels to the original C through another 1×1 convolution; finally, the spatial attention weight matrix is ​​generated through the Softmax activation function.

7. The road damage detection method based on improved YOLOv5 according to claim 1, characterized in that: The road damage detection model is trained using the HIoU loss function, which is expressed as: Among them, IoU represents the intersection-over-union ratio of the predicted box and the real box in the road disease area; A represents the real box of the road disease area, and B represents the predicted box of the road disease area; c represents the diagonal length of the minimum enclosing rectangle containing the real box A and the predicted box B; v is used to measure the similarity of the aspect ratio of the real box A and the predicted box B; A c Represents the area of ​​the minimum enclosing rectangle of the real box A and the predicted box B; ρ 2 represents the distance between the center of the real box A and the predicted box B; w B Indicates the width of the prediction box B; h B Indicates the height of the prediction box B; w A Indicates the width of the real box A; h A Represents the width of the ground-truth box A.

8. A road damage detection device based on improved YOLOv5, characterized in that: It comprises a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory and is used to execute the computer program stored in the memory, so that the road damage detection device based on the improved YOLOv5 performs the road damage detection method based on the improved YOLOv5 described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a program, characterized in that: When the program is executed by the processor, the road damage detection method based on improved YOLOv5 described in any one of claims 1 to 8 is implemented.

Images