Road damage detection method, equipment and medium
By improving the DMC-YOLOv11 model of the YOLOv11 model and optimizing feature map fusion using depthwise separable convolution and cross-view interaction modules, the paper solves the problem of insufficient accuracy of single-stage detection algorithms in road damage detection and achieves efficient and accurate road damage detection.
Patent Information
- Application Number
- CN202510801809.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-16
- Publication Date
- 2025-09-19
AI Technical Summary
Existing single-stage target detection algorithms have insufficient detection accuracy in road damage detection, especially when dealing with small-sized targets and complex background scenes, which are prone to missed detections and misjudgments.
An improved DMC-YOLOv11 model is adopted. By applying depthwise separable convolution to the Backbone and Neck parts of the YOLOv11 model, adding the MCFN module and introducing the cross-view interaction module CVIM, the fusion processing of feature maps is optimized and the feature extraction and detection accuracy are enhanced.
The model's detection speed and accuracy have been significantly improved, and it can effectively identify small-scale road damage in complex backgrounds, thereby improving detection accuracy and robustness.
Smart Images
Figure CN120673367A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of road damage detection, and in particular to a road damage detection method, equipment and medium. Background Art
[0002] As the highway network expands and the number of cars increases, road maintenance challenges intensify. Affected by factors such as construction, climate, and load, roads are prone to problems such as blurred markings, cracks, and potholes. If not addressed promptly, these problems can worsen and endanger safety and economic well-being. Road damage detection can help identify problems promptly, facilitate traffic planning, and repair roads, thereby reducing accident rates. Therefore, developing efficient road damage detection methods is of great significance.
[0003] Amid the rapid development of artificial intelligence (AI), existing deep learning-based detection algorithms for road damage detection are primarily categorized as two-stage and one-stage. Two-stage detection algorithms involve two processing steps: first, generating candidate regions for potential targets using mechanisms such as sliding windows; second, classifying and localizing these candidate regions. Single-stage detection algorithms, on the other hand, construct a single neural network model, process the input image, and employ a dense sliding window approach. The neural network directly outputs the bounding box coordinates and class probabilities for each object at each location, completing both target localization and classification in one step, without requiring the pre-generation of candidate regions.
[0004] Compared with two-stage object detection algorithms, single-stage object detection algorithms are more efficient and time-efficient during model training and inference, meeting the real-time requirements of road damage detection. However, this algorithm has limitations in detection accuracy, particularly when dealing with small objects, complex backgrounds, and multi-scale damage features, leading to missed detections and misjudgments. Summary of the Invention
[0005] The present invention aims to provide a road damage detection method, device and medium, which can improve the efficiency and accuracy of road damage detection.
[0006] To solve the above technical problems, an embodiment of the present invention provides a road damage detection method, comprising the following steps: Road damage images are acquired as experimental datasets through a mobile shooting instrument; A DMC-YOLOv11 model optimized based on the YOLOv11 model is constructed, and the DMC-YOLOv11 model is trained using the dataset to obtain a road damage detection model. The DMC-YOLOv11 model replaces standard convolutional layers in the Backbone and Neck parts of the original YOLOv11 model with depthwise separable convolutions. An MCFN module is added to the C3K2 module of the original YOLOv11 model to form a C3K2-MCFN module. Subsequently, all C3K2 modules in the Neck part of the original YOLOv11 model are replaced with C3K2-MCFN modules. A cross-view interaction module (CVIM) is introduced into the Neck part of the original YOLOv11 model to concatenate feature maps generated by the Backbone and Neck modules to complete the fusion of feature maps on the left and right sides. The road damage detection model is used to detect undetected road damage images to obtain road damage recognition results.
[0007] In some optional embodiments, the standard convolutional layers in the Backbone and Neck parts of the original YOLOv11 model are replaced with depthwise separable convolutions. The depthwise separable convolution operation specifically includes: Deep convolution uses independent convolution kernels to perform convolution on each channel of the input road damage image, without involving data interaction between channels. After convolution, each input channel outputs a single-channel image, and the number of output channels remains the same as the number of input channels. Point-by-point convolution The convolution kernel processes a single channel image and mixes multiple channel images through linear combination operations to adjust the number of channels and generate feature images of the specified number of output channels.
[0008] In some optional embodiments, the depthwise separable convolution is expressed by the following formula: Where, is the convolution kernel size, is the number of input channels, is the number of output channels, is the size of the output feature map.
[0009] In some optional embodiments, after adding the MCFN module to the C3K2 module of the original YOLOv11n to form the C3K2-MCFN module, all the C3K2 modules in the Neck part of the original YOLOv11 model are replaced with the C3K2-MCFN module, specifically including: Add the MCFN module to the back of the Bottleneck structure, replace the Bottleneck structure in C3K2 with the Bottleneck structure containing the MCFN module, and form the C3K2-MCFN module. The steps of the MCFN module are as follows: Using point-by-point convolution For the input feature image Perform calculations, double the number of channels, and output as feature images ; The feature image The network is divided into multiple branches, and the multi-core convolution performs a deep convolution operation on each branch using three convolution kernels of different sizes to output multi-scale local features; the multi-scale local features are spliced and convolved point by point. Complete dimensionality reduction and output feature image , the MCFN module formula is: in, It is a multi-core convolution operation; It is the GELU activation function, which is used to introduce nonlinearity.
[0010] In some optional embodiments, the cross-view interaction module CVIM is introduced into the Neck part of the original YOLOv11 model to perform a splicing operation on the feature maps generated by the Backbone and Neck modules to complete the fusion processing of the feature maps on the left and right sides, specifically including: In the Neck part of the original YOLOv11 model, the three CVIM module input interfaces are connected to the output interfaces of the three C3K2-MCFN modules responsible for outputting to the detection head. This allows the output data of the C3K2-MCFN modules to flow into the corresponding interfaces in sequence for subsequent processing by the CVIM modules. The input of the CVIM module is the two-part feature map output by the C3K2-MCFN module, and a cross-view attention mechanism is used to generate a cross-view attention map for the two-part feature map information output by the C3K2-MCFN module. The output of the cross-view interaction module CVIM is to transmit the cross-view attention map to two paths respectively: the first path uses the output cross-view attention map as the input data of the detection head; the second path uses the output cross-view attention map as the input data of the subsequent convolution module, and the output data of the convolution module further enters the Neck part process for processing.
[0011] In some optional embodiments, the step of generating a cross-view attention map for the two parts of feature map information output by the C3K2-MCFN using a cross-view attention mechanism specifically includes: The two feature maps output by the C3K2-MCFN module , As input, output cross-attempt fusion features The expression is as follows: in, The query matrix obtained by projecting the source view features; The key matrix and value matrix obtained by projecting the target view features; H, W, C are the height, width and number of channels of the feature map respectively; for Point-wise convolution; for Depthwise convolution.
[0012] The same calculation logic and processing flow as above are used to input the two feature maps output by the C3K2-MCFN module. , Process and get the output cross-attempt fusion feature The expression is as follows: By element-by-element addition, the cross-view features after interaction are fused. , The two-part feature map output by the C3K2-MCFN module , Perform fusion to generate cross-attempt attention maps , : in, This is a trainable channel scale parameter whose value can be adjusted dynamically during model training. To ensure training stability, this parameter is initially set to 0.
[0013] An embodiment of the present invention also provides a computer device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned road damage detection model method.
[0014] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program can execute the above-mentioned road damage detection model method.
[0015] The road damage detection method provided by the present invention has at least the following beneficial effects: The road damage detection method based on the DMC-YOLOv11 model in the present invention applies depthwise separable convolution to replace the traditional convolution operation of the original model in the Backbone and Neck parts. This can identify and eliminate redundancy in feature maps, strengthen the correlation between feature maps, significantly reduce the number of model parameters, and greatly improve the inference speed. The C3K2-MCFN module is applied to the Neck part to replace the C3K2 module of the original model. This can segment high-dimensional features and capture local correlations between features from different receptive fields, thereby enhancing the model's ability to extract complex features. The cross-view interaction module is introduced in the Neck part to splice the feature maps generated by the Backbone and Neck modules, comprehensively capture and integrate multi-view feature information, and provide the detection head with high-quality input data with richer semantics and more accurate representation. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0017] Figure 1 This is a flowchart of a road damage detection method provided according to an embodiment of the present invention; Figure 2 1. A schematic diagram of the structure of a DMC-YOLOv11 model improved based on YOLOv11 according to an embodiment of the present invention; Figure 3 is a schematic diagram of a depth-wise separable convolution structure provided according to an embodiment of the present invention; Figure 4 1 is a schematic diagram of a C3K2-MCFN module structure provided according to an embodiment of the present invention; Figure 51 is a schematic diagram of the structure of a cross-view interaction module CVIM provided according to an embodiment of the present invention; DETAILED DESCRIPTION
[0018] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0019] One embodiment of the present invention relates to a road damage detection method. The implementation details of the road damage detection method of this embodiment are described in detail below. The following content is only provided for ease of understanding and is not required for implementing this solution.
[0020] The specific process of the road damage detection method of this embodiment can be as follows: Figure 1 Shown, including: Step 101: Acquire road damage images as an experimental dataset using a mobile photographing device.
[0021] Specifically, images of road damage are captured using instruments such as drones or smartphones mounted on the back of motorcycles. The types of road damage images include longitudinal cracks, transverse cracks, cracks, potholes, damaged crosswalks, damaged paint, and damaged manhole covers. The collected road damage images are divided into train_set, valid_set, and test_set in a ratio of 8:1:1.
[0022] Step 102: construct a DMC-YOLOv11 model optimized based on the YOLOv11 model, and train the DMC-YOLOv11 model using the data set to obtain a road damage detection model.
[0023] The steps of the DMC-YOLOv11 model are as follows: The standard convolutional layers in the Backbone and Neck parts of the original YOLOv11 model are replaced with depthwise separable convolutions. After adding the MCFN module to the C3K2 module of the original YOLOv11 model to form the C3K2-MCFN module, all the C3K2 modules in the Neck part of the original YOLOv11 model are replaced with C3K2-MCFN modules. The cross-view interaction module CVIM is introduced into the Neck part of the original YOLOv11 model to splice the feature maps generated by the Backbone and Neck modules to complete the fusion processing of the feature maps on the left and right sides. The DMC-YOLOv11 model is as follows: Figure 2As shown in the figure, it mainly consists of three parts: the backbone network, the neck, and the head. The backbone part includes components such as DWConv, C3K2, SPPF, and C2PSA, which extract features from the input road damage image; the neck part includes modules such as CVIM and C3K2-MCFN, which fuse features at different levels and enhance feature expression; and the head part performs target detection.
[0024] The FPN+PAN structure used in the YOLOv11 model improves detection performance, but it also comes with a significant increase in the number of parameters and computational complexity, which undoubtedly puts considerable pressure on the deployment and practical application of the model. To solve this problem, the standard convolutional layers in the Backbone and Neck parts of the original YOLOv11 model are replaced with depthwise separable convolutions, thereby achieving efficient extraction of multi-scale targets and local detail features, significantly reducing the computational cost of the model; the depthwise separable convolutional structure is as follows: Figure 3 As shown in Figure 3, depth-wise separable convolution splits the traditional convolution into two steps: depth-wise convolution and point-wise convolution.
[0025] The standard convolution operation process is: after using several multi-channel convolution kernels to process the input multi-channel road damage image, the output feature map is used to extract the spatial features of the road features; the standard convolution formula is: Where, is the convolution kernel size, is the number of input channels, is the number of output channels, is the size of the output feature map.
[0026] Depthwise separable convolution decomposes the standard product operation and separates the spatial feature extraction and channel feature extraction into two steps: depthwise convolution and pointwise convolution. Depthwise convolution uses independent convolution kernels to perform convolution on each channel of the input road damage image, without involving data interaction between channels. After convolution, each input channel outputs a single channel image, and the number of output channels is consistent with the number of input channels. Pointwise convolution uses The convolution kernel processes a single channel image and mixes multiple channel images through linear combination operations to adjust the number of channels and generate feature images of the specified number of output channels. The formula for depthwise separable convolution is:
[0027] Where, is the convolution kernel size, is the number of input channels, is the number of output channels, is the size of the output feature map.
[0028] Then, the ratio of the computational cost of depth-wise separable convolution to standard convolution is: The C3K2 module in the YOLOv11 model uses standard convolution and Bottleneck structure to extract global features. However, this method has limitations in utilizing local detail features of the image, making it difficult to effectively guarantee the accuracy and reliability of the detection results when processing small-sized targets and complex damage features. To solve this problem, the MCFN module is added to the C3K2 module of the original YOLOv11 model to form the C3K2-MCFN module, and then all the C3K2 modules in the Neck part of the original YOLOv11 model are replaced with C3K2-MCFN modules. The addition of the MCFN module enables the module to achieve efficient extraction of local detail information under different receptive fields through multi-kernel convolution and multi-scale representation learning. The addition of this module can effectively capture the feature details of multi-scale targets and greatly improve the model's feature extraction performance for multi-scale targets; the MCFN module structure is as follows Figure 4 As shown, the structures of the C3K2-MCFN module, the Bottleneck-MCFN module and the MCFN module are described in detail. The steps of the MCFN module are as follows:
[0029] Using point-by-point convolution For the input feature image Perform calculations, double the number of channels, and output as feature images ; The feature image The network is divided into multiple branches, and the multi-core convolution performs a deep convolution operation on each branch using three convolution kernels of different sizes to output multi-scale local features; the multi-scale local features are spliced and convolved point by point. Complete dimensionality reduction and output feature image , the MCFN module formula is: in, It is a multi-core convolution operation; It is the GELU activation function, which is used to introduce nonlinearity.
[0030] In the Neck part of the original YOLOv11 model, the cross-view interaction module CVIM is introduced to realize multi-view information fusion. This module can extract and integrate the spatial and local detail features of each view. In the road damage detection task, it effectively improves the detection accuracy and recognition effect for small targets and complex background scenes. In the Neck part of the original YOLOv11 model, the three cross-view interaction module CVIM input interfaces are respectively connected to the output interfaces of the three C3K2-MCFN modules responsible for output to the detection head, so that the data output by the C3K2-MCFN module flows into the corresponding interface in turn, and is subsequently processed by the cross-view interaction module CVIM. The structure of the cross-view interaction module CVIM is as follows Figure 5 As shown, the input is the two-part feature map output by the C3K2-MCFN module , , first processed by point-by-point convolution PWConv and depth convolution DepthConv, then the image features are transformed in dimension and multiplied, the attention weight is calculated by the SoftMax function, and then processed by point-by-point convolution PWConv, and finally added to the original input features to achieve cross-view feature fusion update.
[0031] The input of the CVIM module is the two-part feature map output by the C3K2-MCFN module. The cross-view attention mechanism is used to generate a cross-view attention map for the two-part feature map information output by the C3K2-MCFN module. The specific steps are as follows: The two feature maps output by the C3K2-MCFN module , As input, output cross-attempt fusion features The expression is as follows: in, The query matrix obtained by projecting the source view features; The key matrix and value matrix obtained by projecting the target view features; H, W, C are the height, width and number of channels of the feature map respectively; for Point-wise convolution; for Depthwise convolution.
[0032] The same calculation logic and processing flow as above are used to input the two feature maps output by the C3K2-MCFN module. , Process and get the output cross-attempt fusion feature The expression is as follows: By element-by-element addition, the cross-view features after interaction are fused. , The two-part feature map output by the C3K2-MCFN module , Perform fusion to generate cross-attempt attention maps , : in, This is a trainable channel scale parameter whose value can be adjusted dynamically during model training. To ensure training stability, this parameter is initially set to 0.
[0033] The output of the cross-view interaction module CVIM is to transmit the cross-view attention map to two paths respectively: the first path uses the output cross-view attention map as the input data of the detection head; the second path uses the output cross-view attention map as the input data of the subsequent convolution module, and the output data of the convolution module further enters the Neck part process for processing.
[0034] Step 103: Detect the undetected road damage image using the road damage detection model to obtain a road damage recognition result.
[0035] Evaluation metrics were used to assess the detection performance of the DMC-YOLOv11 road damage detection model, measuring its detection accuracy and verifying its generalization and robustness. The DMC-YOLOv11 road damage detection model was checked to see if it detected longitudinal cracks, transverse cracks, cracks, potholes, damaged crosswalks, damaged paint, and damaged manhole covers, checking for any missed detections.
[0036] The steps of the various methods above are divided only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are within the scope of protection of the present invention. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of the invention.
[0037] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.
[0038] That is, those skilled in the art will understand that all or part of the steps in the above-described method embodiments can be implemented by instructing the relevant hardware through a program. The program is stored in a storage medium and includes a number of instructions for causing a device (such as a microcontroller or chip) or a processor to execute all or part of the steps in the method embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0039] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present invention, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.
Claims
1. A road damage detection method, characterized in that: The method comprises: Road damage images are acquired as experimental datasets through a mobile shooting instrument; A DMC-YOLOv11 model optimized based on the YOLOv11 model is constructed, and the DMC-YOLOv11 model is trained using the dataset to obtain a road damage detection model. The DMC-YOLOv11 model replaces standard convolutional layers in the Backbone and Neck parts of the original YOLOv11 model with depthwise separable convolutions. An MCFN module is added to the C3K2 module of the original YOLOv11 model to form a C3K2-MCFN module. Subsequently, all C3K2 modules in the Neck part of the original YOLOv11 model are replaced with C3K2-MCFN modules. A cross-view interaction module (CVIM) is introduced into the Neck part of the original YOLOv11 model to concatenate feature maps generated by the Backbone and Neck modules to complete the fusion of feature maps on the left and right sides. The road damage detection model is used to detect undetected road damage images to obtain road damage recognition results.
2. The road damage detection method according to claim 1, wherein: The standard convolutional layers in the Backbone and Neck parts of the original YOLOv11 model are replaced with depthwise separable convolutions. The depthwise separable convolution operation specifically includes: Deep convolution uses independent convolution kernels to perform convolution on each channel of the input road damage image, without involving data interaction between channels. After convolution, each input channel outputs a single-channel image, and the number of output channels remains the same as the number of input channels. Point-by-point convolution The convolution kernel processes a single channel image and mixes multiple channel images through linear combination operations to adjust the number of channels and generate feature images of the specified number of output channels.
3. The road damage detection method according to claim 2, wherein: The depth-wise separable convolution is expressed by the following formula: Where, is the convolution kernel size, is the number of input channels, is the number of output channels, is the size of the output feature map.
4. The road damage detection method according to claim 1, wherein: The method adds the MCFN module to the C3K2 module of the original YOLOv11 model to form the C3K2-MCFN module, and then replaces all the C3K2 modules in the Neck part of the original YOLOv11 model with the C3K2-MCFN module, specifically including: Add the MCFN module to the back of the Bottleneck structure, replace the Bottleneck structure in C3K2 with the Bottleneck structure containing the MCFN module, and form the C3K2-MCFN module. The steps of the MCFN module are as follows: Using point-by-point convolution For the input feature image Perform calculations, double the number of channels, and output as feature images ; The feature image The network is divided into multiple branches, and the multi-core convolution performs a deep convolution operation on each branch using three convolution kernels of different sizes to output multi-scale local features; the multi-scale local features are spliced and convolved point by point. Complete dimensionality reduction and output feature image , the MCFN module formula is: in, It is a multi-core convolution operation; is the GELU activation function, which is used to introduce nonlinearity.
5. The method of road damage detection model according to claim 1, characterized in that The cross-view interaction module CVIM is introduced into the Neck part of the original YOLOv11 model to splice the feature maps generated by the Backbone and Neck modules to complete the fusion processing of the feature maps on the left and right sides. Specifically, the following steps are performed: In the Neck part of the original YOLOv11 model, the three cross-view interaction module (CVIM) input interfaces are connected to the output interfaces of the three C3K2-MCFN modules responsible for outputting to the detection head. This allows the output data of the C3K2-MCFN modules to flow into the corresponding interfaces in sequence for subsequent processing by the cross-view interaction module (CVIM). The input of the cross-view interaction module CVIM is the two-part feature map output by the C3K2-MCFN module, and a cross-view attention mechanism is used to generate a cross-view attention map for the two-part feature map information output by the C3K2-MCFN module; The output of the cross-view interaction module CVIM is to transmit the cross-view attention map to two paths respectively: the first path uses the output cross-view attention map as the input data of the detection head; the second path uses the output cross-view attention map as the input data of the subsequent convolution module, and the output data of the convolution module further enters the Neck part process for processing.
6. The road damage detection method according to claim 5, wherein: The step of using the cross-view attention mechanism to generate a cross-view attention map for the two parts of feature map information output by the C3K2-MCFN specifically includes: The two feature maps output by the C3K2-MCFN module , As input, output cross-attempt fusion features The expression is as follows: in, The query matrix obtained by projecting the source view features; The key matrix and value matrix obtained by projecting the target view features; H, W, C are the height, width and number of channels of the feature map respectively; for Point-wise convolution; for Depthwise convolution.
7. Using the same calculation logic and processing flow as above, the two feature maps output by the C3K2-MCFN module are input , Process and get the output cross-attempt fusion feature The expression is as follows: By element-by-element addition, the cross-view features after interaction are fused. , The two-part feature map output by the C3K2-MCFN module , Perform fusion to generate cross-attempt attention maps , : in, This is a trainable channel scale parameter whose value can be adjusted dynamically during model training. To ensure training stability, this parameter is initially set to 0.
8. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so as to enable the at least one processor to perform the road damage detection method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can execute any one of the road damage detection methods defined in claims 1 to 6.