Improved YOLOv11 pavement disease detection method

Abstract

The invention discloses an improved YOLOv11 pavement disease detection method, which comprises the following steps of: 1, carrying out road inspection, shooting to obtain a pavement disease image data set, carrying out manual marking, and establishing a marked image data set; 2, dividing the marked image into a training set, a verification set and a test set; 3, constructing a pavement disease detection model; 4, training the pavement disease detection model based on the training data set and the verification set, and storing the trained model; and 5, evaluating the trained model on a test set. The pavement disease detection model comprises a backbone network, a neck network and a detection head network; the backbone network firstly extracts basic features from an original image and outputs the basic features; the neck network receives the basic features and generates fusion features; and the detection head network directly predicts the position, category and confidence of the target based on the fused features, and finally realizes pavement disease detection.

Description

Technical Field

[0001] This invention relates to the field of highway pavement maintenance technology, and in particular to an improved method for detecting pavement defects using YOLOv11. Background Technology

[0002] Over long-term use, road surfaces inevitably develop various defects due to the combined effects of traffic loads, natural environment, and material aging, reducing driving comfort and affecting driving safety. By capturing road surface images through vehicle-mounted cameras and utilizing deep learning algorithms, feature patterns of defects can be automatically learned from large amounts of road surface image data. This allows for the construction of effective models to automatically detect and classify road surface defects, significantly improving detection efficiency and accuracy. In road surface defect detection, YOLOv11 can quickly process large amounts of road surface image data, detecting and identifying common road surface defects.

[0003] However, in practical applications, it has been found that YOLOv11 still has certain limitations in detecting small targets such as micro-cracks and small potholes in road surface defects, as well as defects on complex textured roads. Complex scenes such as slow lanes, occlusion by moving and static objects, and shadows can interfere with the model's extraction and recognition of defect features, leading to a decrease in detection accuracy. Furthermore, small defects occupy a small proportion of pixels in the image, and their features are not obvious, making them easy for the model to ignore or misjudge. Therefore, it is necessary to improve YOLOv11 to enhance its performance in detecting defects in complex scenes and small targets in road surface defects.

[0004] Therefore, improving the accuracy of YOLOv11 in detecting small targets and complex scenes in road surface distress images is an important problem that needs to be solved. However, a review of relevant materials yielded no results. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide an improved method for detecting pavement defects using YOLOv11, addressing the shortcomings of the existing technology.

[0006] To address the aforementioned technical problems, this invention discloses an improved method for detecting pavement defects using YOLOv11, comprising the following steps:

[0007] Step 1: Conduct road inspections, capture road surface damage image datasets, manually annotate them, and create an annotated image dataset;

[0008] Step 2: Divide the labeled images into training, validation, and test sets;

[0009] Step 3: Construct a pavement distress detection model based on the improved YOLOv11;

[0010] Step 4: Train the pavement distress detection model based on the training dataset and validation set, and save the trained model;

[0011] Step 5: Evaluate the trained model on the test set.

[0012] The pavement defect detection model comprises a backbone network, a neck network, and a head network. The backbone network first extracts multi-scale feature maps from the original image as basic feature outputs. The neck network receives the basic features and generates higher-quality fused features by enhancing the semantic information and detailed expression of the features. The head network then directly predicts the location, category, and confidence level of the target based on the fused features. The three networks are connected in sequence to form a pavement defect detection system, from feature extraction to feature optimization.

[0013] The overall architecture of the road surface distress detection model is as follows: Figure 2 This model uses the YOLOv11 model as the base model and constructs a cross-stage global context space-aware attention module (CSP-GCAA) as the feature extraction module of the backbone network, replacing the C3K2 feature extraction module in the base model. In the neck network, the structure of the neck network in the base model is changed, and a hierarchical feature enhancement neck network (HFENeck) is proposed.

[0014] The cross-stage global context space awareness attention module CSP-GCAA, due to the fact that the C3k2 module structure essentially enhances gradient flow through cross-stage partial connections (CSP), has a small convolutional kernel size and insufficient feature mixing depth between channels. When processing objects with large-scale changes, more layers need to be stacked to compensate for the insufficient receptive field, which makes it more difficult to extract features from shallow networks and insufficient extraction of high-level semantic information. In complex backgrounds or scenes with dense multiple objects, edge and texture details may be lost.

[0015] Based on this, this invention proposes the CSP-GCAA module. By introducing dilated convolutions and multi-scale convolutions, these are concatenated and fused to enhance the local receptive field. A spatial attention module is then embedded, allowing the network to autonomously focus on globally important regions and guide the weight allocation of local features. CSP-GCAA modifies C3K2, expanding the coverage of local features and introducing global context through an attention mechanism, thus enhancing the ability to detect occluded and large targets. The structure of the CSP-GCAA module is as follows: Figure 4 As shown, the structure of the CSP-GCAA module is as follows: Split module, Conv1 module, four parallel Conv modules, Concat1 module, Coordinate Attention module, Bottleneck module, Concat2 module, and Conv2 module.

[0016] The feature map input to CSP-GCAA first passes through the Split module, which divides the feature map channels into three parts: one part serves as the input to the Bottleneck module, one part as the input to the Concat2 module, and one part as the input to a Conv1 module. The output of the Conv1 module then serves as the input to four parallel Conv modules. Two of the Conv modules have a 3×3 kernel size, one Conv module uses dilated convolution, and the other Conv module has a 5×5 kernel size. The outputs of the four parallel Conv modules serve as the input to the Concat1 module, which concatenates the data. The output of the Concat1 module then serves as the input to the Coordinate Attention module, which in turn serves as the input to the Concat2 module. The output of the Bottleneck module also serves as the input to the Concat2 module, which in turn serves as the input to the Conv2 module. The output of the Conv2 module is then used as the output feature map of CSP-GCAA.

[0017] The original neck network feature fusion path is relatively fixed and lacks dynamic adjustment capabilities, making it difficult to perfectly fuse small target features extracted from the backbone network layers P3, P4, and P5. In complex scenes, features at different scales cannot be optimally combined, limiting the model's ability to detect diverse targets. For example, in scenes where dense small and large targets coexist, the fixed fusion method cannot effectively balance the utilization of features from targets of different scales, leading to a decrease in detection accuracy. Therefore, the Hierarchical Feature Enhancement Neck Network (HFENeck) is proposed. The HFENeck network structure is as follows: Figure 2 As shown, HFENeck adds a connection between layers P2 and P3. The features of layer P2 are processed by the Spatial Depth Transformation Convolutional Module (SPD-Conv) and fused together with layer P3 and upsampling layer through the proposed Hierarchical Pixel Fusion Perception Module (HPFPM). At the same time, the upsampling between layers P4 and P5 is enhanced and finely fused through the HPFPM module, thereby improving the model's ability to detect small objects and align features in complex scenes.

[0018] As the highest resolution feature layer in YOLOv11, the P2 layer is directly responsible for detecting small targets, but the original features may have rich details but weak semantics.

[0019] The Spatial Depth Transform Convolutional Module (SPD-Conv) enhances the spatial geometric information and channel depth semantics of features while maintaining high resolution by separating spatial convolution and depth convolution, thus avoiding the excessive smoothing or dilution of small target features by traditional convolution.

[0020] The Hierarchical Pixel Fusion Perception Module (HPFPM) consists of a CBS module, an SDP module, an overlay module, and a Concat module. Upper-layer features F1 and lower-layer features F2 are simultaneously input into the HPFPM module. Lower-layer features F2 are first processed by the CBS module for feature extraction. The output of the CBS module and the upper-layer features F1 are simultaneously input into the SDP module for feature extraction at different levels. The output of the SDP module and the upper-layer features F1 are input into the overlay module for element-wise addition of the feature maps. The output of the overlay module and the output of the CBS module are input into the Concat module for concatenation. The output of the Concat module is used as the output of the HPFPM.

[0021] Basic YOLOv11 uses Concat to simply add features from different levels, ignoring pixel-level spatial relationships between features. HPFPM, based on a Spatial Dependency Awareness Module (SDP), performs spatial correlation fusion of features between levels, avoiding the problem of physical concatenation but semantic disconnect between features. The constructed hierarchical pixel fusion awareness module is as follows: Figure 5 As shown, it performs convolution operations on the input lower-level feature F2, and then uses the upper-level feature F1 through the SPD module to capture long-distance spatial correlation dependencies, thereby improving the utilization rate of small target features.

[0022] The SDP module uses an attention mechanism to capture the spatial location pixel effects of features at different levels, enriching the features of tiny objects. Its structure is as follows: Figure 6 As shown.

[0023] The structure of the SDP module is as follows: Query(Q), Key(K), and Value(V) matrices are calculated through three separate 1×1 convolutions, where Q is derived from F1, and K and V are derived from... Export, From lower-level features The feature is obtained after extraction by the CBS module. Since the neck network needs to perform upsampling on one hand and receive features from several layers in the backbone network on the other, the HPFPM module in the neck network is used for the fusion of two feature layers. These two feature layers are divided into upper feature F1 and lower feature F2. The SDP module receives feature F2' after F2 is extracted by the CBS module. The {Q, K, V} matrix is ​​divided into multiple feature blocks, assuming the dimension of {Q, K, V} is C×... × The dimension of the partitioned feature blocks is n× × ×C, where C is the number of channels in F1. × It's F1 resolution, then the corresponding feature blocks are extracted. and , and The dimension is × ×C, calculate their similarity, and obtain the similarity matrix. :

[0024]

[0025] After applying the cross-attention mechanism to each feature block of Q, these blocks are aggregated according to their spatial location to form a new feature with rich spatial dependencies, which is then added to F1.

[0026] Beneficial effects:

[0027] After improving the backbone and neck network of YOLOv11 respectively—specifically by using the cross-stage global context space-aware attention module CSP-GCAA to improve the backbone network and using the hierarchical feature enhancement neck network HFENeck to improve the neck network—the improved model exhibits significant advantages in multiple aspects of object detection compared to the original YOLOv11. The specific advantages and effects are as follows:

[0028] (1) The CSP-GCAA module expands the coverage of local features by introducing dilated convolution and multi-scale convolution, enabling richer detailed information to be extracted in shallow networks. When dealing with objects with large-scale changes, it does not require excessive stacking of layers to compensate for insufficient receptive field. At the same time, the embedded spatial attention module enables the backbone network to autonomously focus on globally important regions, enhancing the extraction of key features and making the extraction of high-level semantic information more complete. Meanwhile, this module can effectively reduce the loss of edge and texture details, improve the model's adaptability to different scenes, and better extract discriminative features whether detecting occluded targets, large targets, or small targets.

[0029] (2) The neck network HFENeck can better integrate features at different levels. By properly processing and integrating features at different scales, it can more accurately align features when integrating high-resolution shallow features and deep features with rich semantic information, avoiding information loss caused by feature misalignment, thereby improving the quality of feature fusion. At the same time, the specially designed modules and fusion strategies of HFENeck ensure that the detailed features of small targets are not overly diluted or lost when passing through the neck network, and can better combine contextual information to improve the detection performance of small targets.

[0030] (3) The improved model has enhanced feature extraction capabilities for occluded and small targets in complex scenes, and can detect more targets that might otherwise be missed, thereby improving the average accuracy and recall rate of target detection at different scales. At the same time, the improved model performs more stably when facing challenging scenes such as complex backgrounds, dense targets, and changes in lighting. Attached Figure Description

[0031] Figure 1 This is a flowchart of an improved YOLOv11 pavement defect detection method according to an embodiment of the present invention.

[0032] Figure 2 This is a schematic diagram of the structure of an improved YOLOv11 pavement defect detection method according to an embodiment of the present invention.

[0033] Figure 3 This is a structural diagram of the cross-stage global context space-aware attention module in an embodiment of the present invention.

[0034] Figure 4 This is a structural diagram of the Coordinate Attention module in an embodiment of the present invention.

[0035] Figure 5 This is a structural diagram of the hierarchical pixel fusion sensing module in an embodiment of the present invention.

[0036] Figure 6 This is a structural diagram of the spatial dependency perception module in an embodiment of the present invention.

[0037] Figure 7 This is a comparison example diagram of model results in an embodiment of the present invention.

[0038] Figure 8 This is a diagram showing the field test results of the model in an embodiment of the present invention. Detailed Implementation

[0039] like Figure 1 As shown, an improved method for detecting pavement defects using YOLOv11 includes the following steps:

[0040] Step 1: Conduct road inspections, capture road surface damage image datasets, manually annotate them, and create an annotated image dataset;

[0041] In this embodiment, to better evaluate the detection effect of the pavement distress detection method based on the improved YOLOv11 disclosed in this invention, the publicly available RDD2022 dataset is used. All images in the dataset are high-resolution images, and road damage has been labeled according to the standard YOLO format, with the labeling categories being D00 longitudinal cracks, D10 transverse cracks, D20 mesh cracks, and D40 potholes. 4000 images from RDD2022_China_Drone are selected as the dataset for this embodiment.

[0042] Step 2: Divide the labeled images into training, validation, and test sets;

[0043] In this embodiment, the RDD2022 dataset of 4478 Chinese road images is specifically divided into a training set, a validation set, and a test set in an 8:1:1 ratio.

[0044] Step 3: Construct a pavement distress detection model based on the improved YOLOv11;

[0045] Step 4: Train the pavement distress detection model based on the training dataset and validation set, and save the trained model;

[0046] Step 5: Evaluate the trained model on the test set.

[0047] The pavement defect detection model includes a backbone network, a neck network, and a detection head network. The backbone network first extracts multi-scale feature maps from the original image as basic feature outputs. The neck network receives the basic features and generates fused features by enhancing the semantic information and detailed expression of the features. The detection head network then directly predicts the location, category, and confidence of the target based on the fused features. The three are connected in sequence to form a pavement defect detection model, from feature extraction to feature optimization.

[0048] The road surface defect detection model is based on the YOLOv11 model. A cross-stage global context space awareness attention module CSP-GCAA is constructed as the feature extraction module of the backbone network. In the neck network, a hierarchical feature enhancement neck network HFENeck is proposed.

[0049] The cross-stage global context spatial awareness attention module CSP-GCAA enhances the local receptive field by introducing dilated convolutions and multi-scale convolutions, then embedding a spatial attention module. This allows the network to autonomously focus on globally important regions and guide the weight allocation of local features. Figure 4As shown, the structure of the CSP-GCAA module includes: Split module, Conv1 module, four parallel Conv modules, Concat1 module, Coordinate Attention module, Bottleneck module, Concat2 module, and Conv2 module.

[0050] The feature map input to CSP-GCAA first passes through the Split module, which divides the feature map channels into three parts: one part serves as the input to the Bottleneck module, one part as the input to the Concat2 module, and one part as the input to a Conv1 module. The output of the Conv1 module then serves as the input to four parallel Conv modules. Two of the Conv modules have a 3×3 kernel size, one Conv module uses dilated convolution, and the other Conv module has a 5×5 kernel size. The outputs of the four parallel Conv modules serve as the input to the Concat1 module, which concatenates the data. The output of the Concat1 module then serves as the input to the Coordinate Attention module, which in turn serves as the input to the Concat2 module. The output of the Bottleneck module also serves as the input to the Concat2 module, which in turn serves as the input to the Conv2 module. The output of the Conv2 module is then used as the output feature map of CSP-GCAA.

[0051] The hierarchical feature enhancement neck network HFENeck adds a connection between layers P2 and P3. The features of layer P2 are processed by a spatial depth transformation convolution module and then fused with layer P3 and the upsampling layer through a hierarchical pixel fusion perception module HPFPM. At the same time, the upsampling between layers P4 and P5 is enhanced and finely fused through the HPFPM module.

[0052] The spatial depth transformation convolution module enhances the spatial geometric information and channel depth semantics of features while maintaining high resolution by separating spatial convolution and depth convolution, thus avoiding excessive smoothing or dilution of small target features by traditional convolution.

[0053] The Hierarchical Pixel Fusion Perception Module (HPFPM) consists of a CBS module, an SDP module, an overlay module, and a Concat module. Upper-layer features F1 and lower-layer features F2 are simultaneously input into the HPFPM module. Lower-layer features F2 are first processed by the CBS module for feature extraction. The output of the CBS module and the upper-layer features F1 are simultaneously input into the SDP module for feature extraction at different levels. The output of the SDP module and the upper-layer features F1 are input into the overlay module for element-wise addition of the feature maps. The output of the overlay module and the output of the CBS module are input together into the Concat module for concatenation. The output of the Concat module serves as the output of the HPFPM.

[0054] The SDP module uses an attention mechanism to capture the spatial location pixel effects of features at different levels, enriching the features of tiny objects.

[0055] The structure of the SDP module is as follows: The Query matrix Q, Key matrix K, and Value matrix V are calculated through three separate 1×1 convolutions. The Q matrix is ​​derived from the upper-layer feature F1, and the K and V matrices are derived from... The {Q, K, V} matrix is ​​derived and divided into multiple feature blocks, assuming the dimension of {Q, K, V} is C×. × The dimension of the partitioned feature blocks is n× × ×C, where C is the number of channels in the upper-layer feature F1. × It is the resolution of the upper-layer feature F1, and then the corresponding feature blocks are extracted. and , and The dimension is × ×C, calculate their similarity, and obtain the similarity matrix. :

[0056]

[0057] After applying the cross-attention mechanism to each feature block of the Q matrix, these blocks are aggregated according to their spatial location to form a new feature with rich spatial dependencies, which is then added to the upper-layer feature F1.

[0058] In this embodiment, the experimental platform is based on the Windows 11 operating system, uses the PyTorch deep learning framework, with PyTorch version torch-2.4.0+cu126, Python version 3.10.11, and is equipped with an NVIDIA GeForce RTX3090 GPU.

[0059] The improved Y0L0v11 hyperparameter configuration involved in this embodiment is as follows: number of iterations is 200, batch size is 16, initial learning rate is 0.01, learning rate weight decay is 0.0005. After training and validation, the latest weight parameters are saved for subsequent model validation.

[0060] In this embodiment, accuracy ( ), recall rate ) and the exact mean ( The model is evaluated using the following indicators:

[0061]

[0062]

[0063]

[0064] in, To accurately detect the number of diseases. The number of items incorrectly detected as diseases. This represents the number of diseases that were missed during detection.

[0065] To verify the detection performance of the improved model, a comparative experiment was conducted between the improved YOLOv11 model and the original YOLOv11. Table 1 shows the detection results of the algorithm on the test set. The data in the table shows that the improved YOLOv11 has a significant advantage in detection accuracy, performing the best among all participating models. Compared with the original YOLOv11 algorithm, the improved model demonstrates superior performance in precision, recall, and accuracy. and @0.5:0.95 were 2.6 percentage points, 2.9 percentage points, 2.3 percentage points, and 3.5 percentage points higher, respectively.

[0066] Furthermore, when this model is applied to real-world scenarios, its detection performance for road surface defects is as follows: Figure 8 As shown.

[0067] Table 1. Detection results of different models under RDD2022_China_Drone

[0068] Model Accuracy (%) Recall rate (%) （%） @0.5:0.95（%） YOLOv11 58.9 56.3 57.5 37.8 This invention 61.5 59.2 59.8 41.3

[0069] This invention provides an improved method for detecting pavement defects using YOLOv11. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. An improved method for detecting road surface diseases of YOLOv11, characterized in that, The steps include the following: Step 1: Conduct road inspections, capture road surface damage image datasets, manually annotate them, and create an annotated image dataset; Step 2: Divide the labeled images into training, validation, and test sets; Step 3: Construct a pavement distress detection model based on the improved YOLOv11; Step 4: Train the pavement distress detection model based on the training dataset and validation set, and save the trained model; Step 5: Evaluate the trained model on the test set.

2. The method for improving YOLOv11 road disease detection according to claim 1, characterized in that, The road surface defect detection model includes a backbone network, a neck network, and a detection head network. The backbone network first extracts multi-scale feature maps from the original image as basic feature outputs. The neck network receives the basic features and generates fused features by enhancing the semantic information and detailed expression of the features. The detection head network directly predicts the target's location, category, and confidence level based on the fused features. These three elements are sequentially linked to form a system that progresses from feature extraction to feature optimization, ultimately achieving road surface defect detection.

3. The improved YOLOv11 pavement distress detection method according to claim 2, characterized in that, The road surface defect detection model is based on the YOLOv11 model. A cross-stage global context space awareness attention module CSP-GCAA is constructed as the feature extraction module of the backbone network. In the neck network, a hierarchical feature enhancement neck network HFENeck is proposed.

4. The method for improving YOLOv11 road disease detection according to claim 3, characterized in that, The cross-stage global context spatial awareness attention module CSP-GCAA introduces dilated convolution and multi-scale convolution, splices and fuses them to enhance the local receptive field, and then embeds a spatial attention module, allowing the network to autonomously focus on globally important regions and guide the weight allocation of local features. The structure of the CSP-GCAA module includes: Split module, Conv1 module, four parallel Conv modules, Concat1 module, Coordinate Attention module, Bottleneck module, Concat2 module and Conv2 module.

5. The method for improving YOLOv11 road disease detection according to claim 4, characterized in that, The feature map input to CSP-GCAA first passes through the Split module, which divides the feature map channels into three parts: one part serves as the input to the Bottleneck module, one part as the input to the Concat2 module, and one part as the input to a Conv1 module. The output of the Conv1 module then serves as the input to four parallel Conv modules. Two of the Conv modules have a 3×3 kernel size, one Conv module uses dilated convolution, and the other Conv module has a 5×5 kernel size. The outputs of the four parallel Conv modules serve as the input to the Concat1 module, which concatenates the data. The output of the Concat1 module then serves as the input to the Coordinate Attention module, which in turn serves as the input to the Concat2 module. The output of the Bottleneck module also serves as the input to the Concat2 module, which in turn serves as the input to the Conv2 module. The output of the Conv2 module is then used as the output feature map of CSP-GCAA.

6. The method for road disease detection of improved YOLOv11 according to claim 3, characterized in that, The hierarchical feature enhancement neck network HFENeck adds a connection between layers P2 and P3. The features of layer P2 are processed by a spatial depth transformation convolution module and then fused with layer P3 and the upsampling layer through a hierarchical pixel fusion perception module HPFPM. At the same time, the upsampling between layers P4 and P5 is enhanced and finely fused through the HPFPM module.

7. The method for improving YOLOv11 road disease detection according to claim 6, characterized in that, The spatial depth transformation convolution module enhances the spatial geometric information and channel depth semantics of features while maintaining high resolution by separating spatial convolution and depth convolution, thus avoiding excessive smoothing or dilution of small target features by traditional convolution.

8. The method for improving YOLOv11 road disease detection according to claim 6, characterized in that, The Hierarchical Pixel Fusion Perception Module (HPFPM) consists of a CBS module, an SDP module, an overlay module, and a Concat module. Upper-layer features F1 and lower-layer features F2 are simultaneously input into the HPFPM module. Lower-layer features F2 are first processed by the CBS module for feature extraction. The output of the CBS module and the upper-layer features F1 are simultaneously input into the SDP module for feature extraction at different levels. The output of the SDP module and the upper-layer features F1 are input into the overlay module for element-wise addition of the feature maps. The output of the overlay module and the output of the CBS module are input together into the Concat module for concatenation. The output of the Concat module serves as the output of the HPFPM.

9. The improved YOLOv11 pavement distress detection method according to claim 8, characterized in that, The SDP module uses an attention mechanism to capture the spatial location pixel effects of features at different levels, enriching the features of tiny objects.

10. The improved YOLOv11 pavement distress detection method according to claim 9, characterized in that, The structure of the SDP module is as follows: the Query matrix Q, Key matrix K, and Value matrix V are calculated through three separate 1×1 convolutions. The Q matrix is ​​derived from the upper-layer feature F1, while the K and V matrices are derived from the feature... Export, From lower-level features After extraction by the CBS module, the {Q, K, V} matrix is ​​divided into multiple feature blocks, assuming the dimension of {Q, K, V} is C× × The dimension of the partitioned feature blocks is n× × ×C, where C is the number of channels in the upper-layer feature F1. × It is the resolution of the upper-layer feature F1, and then the corresponding feature blocks are extracted. and , and The dimension is × ×C, calculate their similarity, and obtain the similarity matrix. : ； After applying the cross-attention mechanism to each feature block of the Q matrix, these blocks are aggregated according to their spatial location to form a new feature with rich spatial dependencies, which is then added to the upper-layer feature F1.

Images