A road flaw detection method based on wavelet domain attention and multi-scale detection

Abstract

The application discloses a frequency domain detail enhancement road flaw detection method based on YOLO11n. The method sets a C2PSA_WDAM module after a SPPF module of a YOLO11n backbone network, enhances crack edges, pit groove boundaries and road surface texture details in P5 / 32 deep semantic features through a wavelet domain attention, simultaneously increases a P2 high-resolution detection branch in a detection head, and expands a P3, P4, P5 three-scale detection structure into a P2, P3, P4, P5 four-scale detection structure, so that the road flaw detection effect is improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent road defect detection and computer vision target detection technology. Specifically, it relates to a road defect detection method based on wavelet domain attention and multi-scale detection. In particular, it relates to a method that uses YOLO11n as a benchmark model, introduces a wavelet domain attention module into the deep layer of the backbone network, and adds a P2 high-resolution detection branch to the detection head to realize the detection of road defects such as road cracks, fissures, and potholes. Background Technology

[0002] In the fields of road maintenance and traffic inspection, the automatic detection of road defects is a crucial aspect of ensuring safe road operation. Over long-term use, roads are susceptible to damage such as longitudinal cracks, transverse cracks, alligator cracks, and potholes due to factors such as vehicle loads, natural environment, and material aging. Failure to detect and address these defects promptly can lead to their expansion, impacting road quality and driving safety.

[0003] Traditional road defect detection methods mostly rely on manual inspection or traditional image processing techniques. Manual inspection suffers from low efficiency, strong subjectivity, and high labor intensity; traditional image processing methods usually rely on manually designed features, which are poorly adaptable to scenarios such as complex road backgrounds, shadow occlusion, lighting changes, low contrast, and narrow cracks, and are prone to missed detections and false detections.

[0004] With the development of deep learning technology, the YOLO series of object detection models have been applied to road defect detection tasks due to their advantages such as fast detection speed and convenient deployment. However, existing YOLO detection models still have shortcomings in handling small-scale, elongated, and low-contrast road defects: on the one hand, deep network features are prone to losing high-frequency details such as crack edges and pavement damage textures during downsampling; on the other hand, the existing multi-scale detection structure still has limited spatial localization capabilities for small-target road defects.

[0005] Therefore, in order to address the problems of small target size, weak edges, and easy loss of texture details in complex road scenarios, there is an urgent need to propose a road defect detection method that can take into account deep semantic modeling, high-frequency detail enhancement, and small target detection capabilities, so as to improve the detection accuracy and robustness of road defects such as longitudinal cracks, transverse cracks, alligator cracks, and potholes. Summary of the Invention

[0006] The purpose of this invention is to provide a road defect detection method based on wavelet domain attention and multi-scale detection. It aims to address the problems of small-scale, weak-edge, and easily lost texture details of road defects such as cracks, fissures, and potholes in complex road backgrounds. By introducing a wavelet domain attention mechanism into the YOLO11n backbone network and adding a P2 high-resolution detection branch to the detection head, the model's ability to detect high-frequency texture details and small-scale targets of road defects is enhanced, thereby improving the accuracy and robustness of road defect detection.

[0007] This invention is implemented as follows: A road defect detection method based on wavelet domain attention and multi-scale detection includes the following steps: Step 1: Obtain a preprocessed road defect detection image dataset. The dataset contains road defect images of various categories, with a uniform image resolution of 512×512 pixels, and is divided into training set, validation set and test set. Step 2: Construct a road defect detection model based on the improved YOLO11n architecture. In the P5 / 32 deep semantic feature of the model backbone network, replace the original C2PSA module with the C2PSA_WDAM module. The C2PSA_WDAM module introduces the WDAM wavelet domain attention module to enhance the frequency domain detail expression ability in the deep features. Step 3: The WDAM wavelet domain attention module performs wavelet decomposition on the input feature map to obtain low-frequency semantic components and high-frequency detail components. The low-frequency semantic components are used to perform window attention modeling, and the high-frequency detail components are used to perform gating enhancement on the low-frequency attention features, thereby improving the model's ability to perceive crack edges, pothole boundaries and road surface damage textures. Step 4: Add a high-resolution detection branch P2 to the detection head, expanding the original YOLO11n three-scale detection structure of P3, P4, and P5 into a four-scale detection structure of P2, P3, P4, and P5. Finally, the input of the Detect layer is the four-scale features of P2, P3, P4, and P5, so as to enhance the model's ability to locate and identify small-scale road defects. Step 5: Train the road defect detection model using the training set, adjust the model parameters using the validation set and save the optimal model weights, input the road image to be detected into the trained optimal model, and output the road defect category and location information.

[0008] The beneficial effects of this invention are as follows: (1) In the deep semantic feature extraction stage of YOLO11n backbone network, the present invention introduces C2PSA_WDAM module, which explicitly models low-frequency semantic information and high-frequency texture detail information through wavelet decomposition, so that the model can more fully perceive the edges of road cracks, potholes and the fine damage texture of the road surface.

[0009] (2) The WDAM wavelet domain attention module in this invention uses high-frequency subbands to gating and enhance the attention of low-frequency windows, which strengthens the expression of high-frequency details while maintaining the deep semantic modeling capability, and helps to improve the detection effect of road defects in complex road backgrounds, shadow occlusion, lighting changes and low contrast scenes.

[0010] (3) The present invention adds a high-resolution detection branch P2 to the detection head, and expands the detection structure into a four-scale detection structure of P2, P3, P4 and P5, which can retain more shallow spatial location information and improve the model's ability to locate and identify small targets such as slender cracks, small potholes and early road surface defects. Attached Figure Description

[0011] Figure 1 is a flowchart of a road defect detection method based on wavelet domain attention and multi-scale detection provided by an embodiment of the present invention; Figure 2 is an overall structural diagram of the improved YOLO11n road defect detection network provided in an embodiment of the present invention; Figure 3 is a structural diagram of the C2PSA_WDAM module provided in an embodiment of the present invention; Figure 4 is a structural diagram of the WDAM wavelet domain attention module provided in an embodiment of the present invention; Figure 5 is a structural diagram of the P2-P5 multi-scale detection head provided in an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0013] This embodiment provides a road defect detection method based on wavelet domain attention and multi-scale detection. Using YOLO11n as the baseline model, this method introduces a C2PSA_WDAM module at the deep semantic features of the backbone network and adds a P2 high-resolution detection branch to the detection head, thus forming a road defect detection model that combines deep semantic modeling, high-frequency detail enhancement, and small target detection capabilities.

[0014] As shown in Figure 1, the road defect detection method based on wavelet domain attention and multi-scale detection provided by this embodiment of the invention includes the following steps: S1: Acquire road defect detection image data and perform preprocessing.

[0015] This embodiment uses the RDD2022 road defect detection dataset. The original dataset was cleaned, and unlabeled images were removed. This dataset contains four types of road defects: D00, D10, D20, and D40, corresponding to longitudinal cracks, transverse cracks, alligator cracks or network cracks, and potholes, respectively. The dataset contains a total of 23,767 images, of which 16,636 are in the training set, 3,565 in the validation set, and 3,566 in the test set, with a partitioning ratio of approximately 70:15:15.

[0016] In the preprocessing stage, the road images are uniformly adjusted to 512×512 pixels. Let the original road image be: The input image is obtained after resizing: in, This represents road defect images input to the model. The dataset includes scenes with complex road backgrounds, shadow occlusion, lighting variations, low contrast, thin cracks, and small defects, which can be used to validate the model's detection capabilities in complex road environments.

[0017] S2: Construct an improved YOLO11n road defect detection model.

[0018] As shown in Figure 2, this embodiment uses YOLO11n as the baseline model to construct an improved YOLO11n road defect detection network. The input image size is 512×512×3. After the image is downsampled step by step by the backbone network, feature layers of different scales such as P2 / 4, P3 / 8, P4 / 16 and P5 / 32 are formed.

[0019] In the deep semantic feature extraction stage of the backbone network, this embodiment replaces the original C2PSA module with the C2PSA_WDAM module. This module is located after the SPPF module, corresponding to the P5 / 32 deep feature position. Its deep backbone structure can be represented as follows: Suppose the deep features input to the C2PSA_WDAM module are: Where B represents the batch size, C represents the number of channels, and H and W represent the feature map height and width, respectively. The C2PSA_WDAM module is used to perform frequency domain detail enhancement on deep semantic features, outputting the enhanced deep features:

[0020] S3: Enhance deep frequency domain features through the C2PSA_WDAM module.

[0021] As shown in Figure 3, the C2PSA_WDAM module introduces the WDAM wavelet domain attention module into the C2PSA basic structure. This module uses wavelet decomposition to divide the input features into low-frequency semantic components and high-frequency detail components. The low-frequency components are mainly used for semantic modeling, while the high-frequency components are mainly used to represent crack edges, pothole boundaries, and road surface damage textures.

[0022] Specifically, the C2PSA_WDAM module receives P5 / 32 deep semantic features from SPPF, processes them using the C2PSA infrastructure, and then feeds them into the WDAM wavelet domain attention module. The WDAM module performs frequency domain enhancement on the input features and outputs enhanced P5 / 32 deep semantic features, which are then used in the subsequent Neck feature fusion network.

[0023] S4: Construct the WDAM wavelet domain attention module.

[0024] As shown in Figure 4, the WDAM wavelet domain attention module is the core frequency domain detail enhancement module in this embodiment. Let the input features be: First, the input feature X is decomposed into a first-order wavelet using Haar wavelet transform to obtain the low-frequency component. and high-frequency components The process can be represented as follows: in, It represents low-frequency semantic components, preserving the overall structure and deep semantic information in road images; It represents high-frequency detail components, used to characterize local details such as road crack edges, texture variations, and pothole boundaries.

[0025] Furthermore, low-frequency components Query features, key features, and value features are generated through QKV mapping. : in, This represents a 1×1 convolution. Represents depthwise convolution. This indicates that the features are divided into three parts: Q, K, and V along the channel dimension.

[0026] At the same time, high frequency components and The data is stitched together, and directional texture enhancement features are extracted through high-frequency convolutional branches: Then, this high-frequency texture enhancement feature is used to enhance the V feature in the low-frequency attention: in, This represents element-wise multiplication. Through this process, high-frequency texture information is incorporated into low-frequency semantic attention computation, enabling the model to simultaneously focus on the local details of cracks and damaged areas during deep semantic modeling.

[0027] Subsequently, Q, K, and V are divided into shifted windows, and attention is calculated within each window. Window attention calculation can be expressed as: in, Indicates relative position offset. Let A represent the learnable temperature parameter, O represent the attention weight matrix, and O represent the windowed attention output. After completing the windowed attention calculation, the feature space structure is restored through window reconstruction and inverse shift operations, and the low-frequency attention output is obtained through output projection. Furthermore, the three high-frequency sub-bands After concatenation, input the high-frequency gating branch to generate gating weights: This gating weight is used to enhance low-frequency attention output: Through the aforementioned gating enhancements, the model can modulate the low-frequency semantic attention output using high-frequency texture details, making it pay more attention to crack edges, pothole boundaries, and minor road surface damage areas.

[0028] Meanwhile, high-frequency sub-band After high-frequency output branching processing, the enhanced high-frequency subband is obtained: Subsequently, the low-frequency characteristics after gating enhancement were... With enhanced high-frequency subband Inputting the inverse wavelet transform module, the frequency domain enhanced features are reconstructed: In addition, the WDAM module also sets up a local detail residual branch, which performs depthwise convolution, batch normalization, SiLU activation, and 1×1 convolution on the original input feature X to obtain local detail features: Finally, the WDAM module output is:

[0029] in, The parameters are scaled for learnable details. Through this residual fusion method, the model preserves and enhances local texture representation while completing frequency domain reconstruction.

[0030] S5: Construct the P2-P5 multi-scale detection head.

[0031] As shown in Figure 5, this embodiment adds a high-resolution detection branch (P2) to the detection head, expanding the original YOLO11n three-scale detection structure (P3, P4, P5) into a four-scale detection structure (P2, P3, P4, P5). The final input to the Detect layer is:

[0032] Specifically, P2 corresponds to a 1 / 4 downsampling scale, P3 to a 1 / 8 downsampling scale, P4 to a 1 / 16 downsampling scale, and P5 to a 1 / 32 downsampling scale. When the input image size is 512×512 pixels, the P2 high-resolution feature map size is approximately 128×128, which can retain more shallow spatial location information and is helpful for detecting thin cracks, small potholes, and early road defects.

[0033] The detection head output includes category prediction and bounding box regression results. Category prediction is used to determine the type of road damage, and bounding box regression is used to predict the location of the damage target. For each candidate target, the model output can be represented as: Where c represents the road defect category, s represents the confidence level, and b represents the predicted bounding box. The bounding box can be represented as: Where x and y represent the coordinates of the center point of the bounding box, and w and h represent the width and height of the bounding box, respectively. After result fusion and non-maximum suppression processing, the final road defect detection results are output.

[0034] S6: Model training and detection output.

[0035] In this embodiment, the input size is 512×512, the number of training epochs is 300, the batch size is 32, the optimizer is SGD, the early stopping policy patience is set to 100, cache is set to False, workers are set to 0, and AMP mixed precision training is disabled (amp=False). For data augmentation, close_mosaic is set to 0 to keep Mosaic augmentation enabled throughout the training process, while other augmentation strategies such as HSV augmentation and random flipping use the default YOLO settings.

[0036] The loss function used during training follows the default detection loss of YOLO11n, including bounding box regression loss, classification loss, and DFL distribution focus loss. The total loss can be expressed as:

[0037] in, This represents the bounding box regression loss. Represents classification loss. This represents the distribution focus loss.

[0038] During training, the parameters of the improved YOLO11n road defect detection model are updated using the training set, and the model parameters are adjusted and the optimal model weights are saved using the validation set. After the model training is completed, the road image to be detected is input into the optimal model. After backbone feature extraction, C2PSA_WDAM frequency domain detail enhancement, P2-P5 multi-scale feature fusion, and detection head prediction, the road defect category and location information are output.

[0039] In summary, this embodiment introduces the C2PSA_WDAM module deep into the YOLO11n backbone network, enabling the model to explicitly model low-frequency semantics and high-frequency details using wavelet decomposition. Simultaneously, by adding a high-resolution P2 detection branch, the detection head possesses four-scale detection capabilities (P2, P3, P4, and P5). This method is suitable for detecting road defects such as D00, D10, D20, and D40 in complex road environments, enhancing the model's ability to perceive fine cracks, small-scale defects, and pavement damage textures.

[0040] The specific embodiments of the present invention have been described above with reference to the accompanying drawings, but these descriptions do not constitute a limitation on the scope of protection of the present invention. Any modifications or variations that can be made by those skilled in the art based on the technical solutions of the present invention without creative effort should still fall within the scope of protection of the present invention.

Claims

1. A road defect detection method based on wavelet domain attention and multi-scale detection, characterized in that, The method includes the following steps: Step 1: Acquire road defect image data and preprocess it to obtain road image samples that meet the model input requirements; Step 2: Construct a road defect detection model based on YOLO11n, which includes a backbone network, a feature fusion network, and a detection head; Step 3: Set up the C2PSA_WDAM module in the P5 / 32 deep semantic feature extraction stage of the backbone network. The C2PSA_WDAM module is located after the SPPF module and is used to perform frequency domain detail enhancement on the P5 / 32 deep semantic features. Step 4: The C2PSA_WDAM module includes a C2PSA basic structure and a WDAM wavelet domain attention module. The WDAM wavelet domain attention module divides the deep semantic features processed by the C2PSA basic structure into frequency sub-bands and generates enhanced P5 / 32 deep semantic features based on the modulation relationship between the structural response sub-band and the directional texture response sub-band. Step 5: Add P2 / 4 detection feature output nodes to the detection head to expand the original P3, P4, P5 three-scale detection structure into a P2, P3, P4, P5 four-scale detection structure; Step 6: Input the road image to be detected into the road defect detection model, and obtain the road defect detection result through the C2PSA_WDAM module and the four-scale detection head.

2. The road defect detection method based on wavelet domain attention and multi-scale detection as described in claim 1, characterized in that, In step 3, the C2PSA_WDAM module includes a C2PSA basic structure and a WDAM wavelet domain attention module. The WDAM wavelet domain attention module is set after the C2PSA basic structure and is used to enhance the detailed response in the P5 / 32 deep semantic features while maintaining the deep semantic expression.

3. The road defect detection method based on wavelet domain attention and multi-scale detection as described in claim 1, characterized in that, In step 3, the YOLO11n backbone network sequentially includes a Conv module, a C3k2 module, an SPPF module, and a C2PSA_WDAM module at the P5 / 32 scale. The C2PSA_WDAM module receives the P5 / 32 deep semantic features output by the SPPF module and outputs the enhanced P5 / 32 deep semantic features.

4. The road defect detection method based on wavelet domain attention and multi-scale detection as described in claim 1, characterized in that, In step 4, the WDAM wavelet domain attention module divides the input features into frequency sub-bands to obtain structural response sub-bands and directional texture response sub-bands. The structural response sub-band is used for attention modeling, and the directional texture response sub-band is used to generate texture modulation features filter_hv. The initial value feature V_inp is modulated by the filter_hv to obtain the value feature V that participates in the attention calculation.

5. The road defect detection method based on wavelet domain attention and multi-scale detection as described in claim 4, characterized in that, The WDAM wavelet domain attention module inputs the directional texture response subband into the high_gate gated branch and the high_out subband generation branch; wherein, the high_gate gated branch is used to modulate the structural response attention output, and the high_out subband generation branch is used to generate directional detail features participating in subband reconstruction. The reconstructed features are then fused with the local compensation features output by the local detail residual branch.

6. The road defect detection method based on wavelet domain attention and multi-scale detection as described in claim 1, characterized in that, In step 5, the detection head adds a P2 / 4 detection feature output node to the original P3, P4, and P5 three-scale detection head of YOLO11n; The P5 / 32 deep semantic features, enhanced by the C2PSA_WDAM module, are used as the highest-level semantic input. After top-down feature propagation and bottom-up scale backpropagation, four detection features, P2 / 4, P3 / 8, P4 / 16 and P5 / 32, are formed and input into the Detect layer.

Images