A road surface defect detection method based on an improved YOLOv8 model

Abstract

This invention relates to the field of pavement defect detection technology and provides a pavement defect detection method based on an improved YOLOv8 model, comprising the following steps: acquiring an image of the pavement to be detected; embedding a multi-scale deformable attention module into the original CSPDarknet-53 structure to extract multi-scale features from the pavement image; processing the extracted features using an adaptive feature control module to ensure that computational resources are focused on the defect region; performing feature fusion processing using KANFusion-Neck; performing regression and classification tasks on the fused features using a detection head; and obtaining the pavement defect detection result based on the predictions of the classification and regression tasks. This invention combines a multi-scale deformable attention mechanism, dynamic feature control, and a high-order nonlinear feature fusion method, which can improve the detection accuracy of small cracks, optimize the allocation of computational resources, and enhance the recognition ability of different types of defects, making it suitable for scenarios such as UAV inspection, autonomous driving, and intelligent road maintenance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of road surface defect detection, and particularly relates to a road surface defect detection method based on an improved YOLOv8 model. BACKGROUND

[0002] In recent years, with the growth of intelligent transportation and infrastructure maintenance needs, road surface defect detection technology has become an important research direction for road safety and management. Currently, the main road surface defect detection methods include traditional manual inspection, image processing-based methods, and deep learning detection models. However, these methods still have many challenges in practical application:

[0003] 1. High cost and low efficiency of manual inspection:

[0004] Traditional manual inspection relies on manual observation and recording, which is limited by weather, environment, and personnel experience, and cannot meet the needs of large-scale road maintenance;

[0005] 2. Poor adaptability of image processing-based methods to complex environments:

[0006] Traditional image processing methods (such as edge detection and histogram analysis) are easily affected by factors such as light changes, shadow interference, and road surface material diversity, leading to false positives or false negatives;

[0007] 3. Existing problems of deep learning detection models:

[0008] (1) Difficulty in detecting small cracks: Due to the small size and blurred edges of cracks, traditional convolutional neural networks (CNN) tend to ignore these features, leading to missed detection of small cracks;

[0009] (2) Waste of computing resources and low detection efficiency: Current detection models use the same computing method for all regions, leading to unreasonable allocation of computing resources and increased detection costs;

[0010] (3) Insufficient detection accuracy for different types of defects: Existing models lack adaptive ability when dealing with different types of defects (such as cracks, potholes, and peeling), leading to classification errors or insufficient feature representation.

[0011] Therefore, it is necessary to optimize the existing technology to improve the detection accuracy of small cracks, optimize the allocation of computing resources, and enhance the recognition ability of different types of defects. SUMMARY

[0012] The purpose of the embodiments of the present application is to provide a road surface defect detection method based on an improved YOLOv8 model, aiming to solve the problems raised in the background technology.

[0013] The embodiment of the application is implemented in the following manner: a road surface defect detection method based on an improved YOLOv8 model, comprising the following steps:

[0014] obtaining a to-be-detected road surface image;

[0015] embedding a multi-scale deformable attention module on the structure of the original CSPDarknet-53 to perform multi-scale feature extraction on the to-be-detected road surface image; processing the extracted features by using an adaptive feature regulation module to ensure that the computing resources are focused on the defect area; performing feature fusion processing by using a KAN fusion neck module (KANFusion-Neck), wherein the KANFusion-Neck is configured with a linear-nonlinear fusion block (LinNonFusion Block) for replacing the original DarknetBottleneck in the PAFPN to enhance the feature extraction capability; performing regression and classification tasks on the fused features by using a detection head;

[0016] obtaining a road surface defect detection result based on the prediction of the classification task and the prediction of the regression task.

[0017] The embodiment of the application combines the multi-scale deformable attention mechanism, the dynamic feature regulation, and the high-order nonlinear feature fusion method, and can improve the small crack detection precision, optimize the computing resource allocation, enhance the recognition ability of different types of defects, and is suitable for unmanned aerial vehicle inspection, automatic driving, intelligent road maintenance, and the like. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 a whole network structure diagram of a road surface defect detection method based on an improved YOLOv8 model provided by the embodiment of the application;

[0019] Figure 2 a network structure diagram of an offset prediction network provided by the embodiment of the application;

[0020] Figure 3 a network structure diagram of an attention weight prediction network provided by the embodiment of the application;

[0021] Figure 4 a network structure diagram of an adaptive feature regulation module provided by the embodiment of the application;

[0022] Figure 5 a network structure diagram of a WNLC for enhancing a feature neck provided by the embodiment of the application;

[0023] Figure 6 a detection result visualization comparison of YOLOv8 (left) and the embodiment (right) provided by the embodiment of the application. DETAILED DESCRIPTION

[0024] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0025] The specific implementation of the present application is described in detail below in combination with specific examples.

[0026] As shown in the flowchart of a road surface defect detection method based on an improved YOLOv8 model provided by an embodiment of the present application, comprising the following steps: Figure 1

[0027] S1, obtaining a to-be-detected road surface image;

[0028] S2, the road surface defect detection network receives the to-be-detected road surface image, first, the improved CSPDarknet-53 backbone network performs multi-scale feature extraction on the to-be-detected road surface image, the improved CSPDarknet-53 backbone network is embedded with a multi-scale deformable attention module (MSDAM) and introduces a self-adaptive gradient-guided feature enhancement mechanism on the basis of the original CSPDarknet-53 structure, compared with the fixed size convolution kernel used by the traditional convolutional neural network (CNN), the MSDAM module combines the geometric self-adaptive ability of deformable convolution and the global modeling ability of self-attention mechanism, thereby significantly improving the feature expression ability, especially in small target detection and complex defect morphology adaptability;

[0029] Then, the features extracted by the backbone network are processed by the adaptive feature regulation module to ensure that the computing resources focus on the defect area;

[0030] Next, enter KANFusion-Neck for feature fusion processing, and LinNonFusion Block is configured in KANFusion-Neck, LinNonFusion Block is integrated in the feature fusion path of KANFusion-Neck, specifically: it is used to replace the original DarknetBottleneck in PAFPN to enhance the feature extraction ability: first, the compression convolution reduces the number of channels and reduces the amount of calculation, while extracting global information, then the WNLC transformation improves the feature expression ability through a learnable single-variable nonlinear function, and then the expansion convolution restores the number of channels to ensure information integrity;

[0031] Finally, the obtained features are processed by the detection head to perform regression and classification tasks;​​​

[0032] S3. Based on the predictions from the classification task and the regression task, obtain the road surface defect detection results.

[0033] Specifically, in step S2, the multi-scale deformable attention module is configured to process the offset prediction network and the attention weight prediction network in parallel.

[0034] Among them, such as Figure 2 As shown, the offset prediction network adopts a two-branch architecture:

[0035] High-frequency detail branches through two A standard convolution followed by a dilation of 2 Dilated convolutions are used to specifically learn the local fine-grained migrations caused by crack edges and texture details. The low-frequency structure branch is passed through average pooling followed by two dilated convolutions with increasing dilation rates, then adaptive pooling and upsampling to learn the global migration caused by the overall defect structure. The outputs of the two branches are adaptively weighted using a learnable fusion gate, with the gate weights obtained through... The output dimension is generated by convolution and the sigmoid function, dynamically adjusting the contribution ratio of high- and low-frequency offsets based on the input feature characteristics; Where H is the number of attention heads, S is the number of scales, and P is the number of sampling points at each location;

[0036] like Figure 3 As shown, the attention weight prediction network adopts a local-global dual-path design:

[0037] The local path uses grouped convolution to maintain feature independence and enhances the response of important features through channel attention mechanism, focusing on the importance assessment of local details of defects; the global path captures long-distance dependencies through downsampling and upsampling operations and combines spatial convolution to model the global contextual relationship between defects and background; after the outputs of the two paths are concatenated, they pass through the attention refinement module, and further optimize the weight distribution through group normalization and gating mechanism. Finally, Softmax normalization is performed in the sampling point dimension to ensure that the sum of weights is 1.

[0038] The multi-scale deformable attention module adopts a multi-scale feature pyramid architecture, including... Four scale levels are used to capture defect features in parallel at different spatial resolutions;

[0039] For each spatial location in the feature map This module performs the following operations:

[0040] (1) First, the predicted offset is superimposed on the uniformly distributed reference point grid to obtain the deformed sampling position. The calculation formula is:

[0041] ;

[0042] in For a reference point network with a regular distribution, The predicted offset. An adjustable scaling factor controls the range of position adjustment;

[0043] (2) Then, the features are sampled at each scale level by bilinear interpolation algorithm to obtain the geometrically adapted feature representation;

[0044] (3) Next, the multi-scale attention weighted sum is calculated. For each scale s and attention head h, the output feature is calculated as follows:

[0045] ;

[0046] in The attention weights are normalized. Features sampled at deformed locations using bilinear interpolation;

[0047] (4) Finally, all attention heads and output features at all scale levels are concatenated along the channel dimension, the channel dimension is adjusted by a linear projection layer, and added to the input features through residual connection to keep the output feature size consistent with the input.

[0048] In step S2, such as Figure 4 The diagram shown is the network structure of the adaptive feature adjustment module, which performs the following operations:

[0049] (1) Based on the features extracted from the backbone network, the local gradient change rate is calculated to measure the complexity of the features. The gradient calculation method is adopted:

[0050] ;

[0051] ;

[0052] ;

[0053] in Represents pixels in the input feature map The value, Representing the gradient in the horizontal direction, it calculates the difference between left and right adjacent pixels, reflecting horizontal edge information. Representing the gradient in the vertical direction, it calculates the difference between adjacent pixels, reflecting vertical edge information. The gradient magnitude represents the edge strength at that point and is used to identify defect areas. A threshold T is set to divide regions with rich details (high gradient value) and structurally stable regions (low gradient value).

[0054] (2) Next, adaptive region filtering is performed based on the gradient value. Generate region mask The calculation method is dynamically determined, and the calculation method is as follows:

[0055] ;

[0056] in A region mask is used to distinguish between complex and simple regions. This indicates that the region contains obvious edge features (cracks, pits, etc.), belonging to a complex region. A global perception strategy (self-attention mechanism) is used for feature extraction. This indicates that the region is relatively flat and has low computational requirements, so a local CNN can be used for fast computation.

[0057] Complex features are extracted using a self-attention mechanism to enhance the feature representation ability of defective regions. The calculation method is as follows:

[0058] ;

[0059] ;

[0060] ;

[0061] Calculate feature relevance (attention weights):

[0062] ;

[0063] Calculate the weighted feature output:

[0064] ;

[0065] in, Given the input feature matrix, , , These represent the query, key, and value matrices, respectively. , , These represent the learnable weight matrix, The scaling factor represents the feature dimension and prevents the gradient from becoming too large. This represents the attention weight matrix, which is normalized using softmax and then used for weighted calculation. It is the final feature output, which contains remote dependency information;

[0066] Through attention mechanisms, the network can capture defect information from a distance, improving the detection accuracy of small cracks and pits;

[0067] (3) Since road surface defects may undergo deformation (such as changes in crack direction), it is necessary to adjust the position of the feature window to ensure that the network focuses on the key area. The calculation method is as follows:

[0068] ;

[0069] in The calculated offset matrix controls the adjustment direction of the feature window. This represents the learnable parameter matrix, which is continuously optimized during training to adapt to the defect states of different road surfaces. This operation can adjust the position of features, enabling the network to automatically align with defect areas and improve detection accuracy.

[0070] (4) Finally, multi-scale information integration is performed, combining local CNN and global perception strategies to ensure that the computational strategies for different regions are optimal. The computation method is as follows:

[0071] ;

[0072] in This represents the characteristics of the final output. Represents global characteristics. This represents the features extracted by a local CNN.

[0073] In step S2, the LinNonFusion Block performs the following operations:

[0074] (1) Input features are processed Convolution reduces dimensionality:

[0075] ;

[0076] in The input features are of size . , represent Convolution is used to reduce computation while preserving key information. These are the features after dimensionality reduction, with a size of ,

[0077] (2) In order to improve the feature fusion capability, the dimensionality-reduced features are processed. Perform WNLC transformation (e.g.) Figure 5 As shown in the image, the specific steps are as follows:

[0078] ;

[0079] in These are learnable weights used to control the contributions of different features. It is a nonlinear transformation function, defined as:

[0080] ;

[0081] in It is a learnable parameter matrix. It is a bias term. It is a non-linear activation function that learns higher-order non-linear relationships through WNLC transformation, enabling the network to combine different features more flexibly, improving the model's adaptability to different types of defects, while avoiding information loss, so that the network can both preserve the edge details of cracks and enhance the overall shape of pits.

[0082] (3) Features after WNLC transformation pass Convolution expands back to the original number of channels:

[0083] ;

[0084] in It is the output feature, and the input feature. With the same number of channels C, and to ensure smooth gradients and prevent information loss, residual connections are introduced. Their purpose is to allow the model to learn both linear and nonlinear feature representations simultaneously, thereby enhancing detection capabilities.

[0085] .

[0086] To verify the effectiveness and superiority of the pavement defect detection method based on the improved YOLOv8 model proposed in this invention, several mainstream target detection models and recently published pavement defect detection methods were selected as comparison benchmarks. Quantitative and qualitative analyses were performed on the public dataset RDD2022, and the results are as follows: Figure 6 As shown in Table 1:

[0087] Table 1. Performance of our method and baseline method under various evaluation indicators.

[0088]

[0089] Figure 6 In the diagram, the red bounding box represents the road damage area that the YOLOv8 model can detect; the green bounding box represents the road damage area that the YOLOv8 model cannot detect but that the embodiments of the present invention can successfully detect. It should be noted that all damage marked with red boxes can also be detected by the embodiments of the present invention.

[0090] As can be seen from the results in the table above, the method proposed in this embodiment outperforms other models in all evaluation metrics, achieving a comprehensive performance improvement:

[0091] In terms of overall detection accuracy index mAP, this method achieves 26.55, surpassing all comparison models;

[0092] In terms of positioning accuracy mAP50, this method achieves 56.42, which is 2.72 percentage points higher than YOLOv10-n's 53.70 and 3.5 percentage points higher than YOLOv8-s's 52.92. This indicates that this method not only improves detection capability but also achieves a breakthrough in target positioning accuracy, especially with superior robustness at an IoU threshold of 0.5.

[0093] In terms of detection performance for targets of different sizes, this method outperforms other models for small targets (mAPs=18.52), medium targets (mAPm=22.27), and large targets (mAPl=30.23), with the most significant improvement in small target detection performance. This indicates that this method has a balanced performance improvement when dealing with targets of different scales, and has achieved a significant breakthrough in solving the traditional problem of small target detection.

[0094] In object detection tasks, small crack detection has always been a highly challenging aspect. According to the COCO dataset definition, a small target refers to a pixel area smaller than... The target is small in size, has few pixels and limited feature information in the image, which easily leads to missed detection and false detection. However, our method achieves a small target detection performance (mAPs) of 18.52, which far exceeds other models. It is 4.74 percentage points higher than YOLOv10-n, 8.27 percentage points higher than YOLOv5-s, and 11.6 percentage points higher than YOLOv7-tiny.

[0095] This significant advantage stems primarily from the improvements made in the embodiments of the present invention in the following aspects:

[0096] First, in terms of feature extraction and fusion, this embodiment of the invention employs a multi-scale deformable attention module (MSDAM). Through a dual-path design of high-frequency detail branches and low-frequency structure branches, it effectively captures the edges of cracks and the overall morphology of pits. The MSDAM module contains four scale levels, capturing defect features in parallel at different spatial resolutions, thus solving the problem that traditional CNNs have difficulty in transmitting small target features to deep networks due to their large convolution stride. In contrast, although YOLOv10-n improves inference efficiency by eliminating NMS and optimizing the model architecture, it may be insufficient in small target feature extraction, resulting in an mAPs of only 13.78.

[0097] Secondly, regarding the allocation of computational resources, this embodiment of the invention introduces an adaptive feature control module, which dynamically divides regions rich in detail (high gradient value) and structurally stable regions (low gradient value) by using a gradient change rate threshold T. For high gradient regions (such as crack edges), a self-attention mechanism is used for global perception, while for low gradient regions (such as flat roads), a local CNN is used for fast computation. This dynamic computation strategy effectively reduces redundant computation and optimizes resource allocation. In contrast, YOLOv8-s uses the same computation method for all regions, resulting in unreasonable allocation of computational resources and increased detection costs.

[0098] Third, in terms of feature representation capability, the embodiments of the present invention enhance the nonlinear representation capability of features through LinNonFusion Block. By using a learnable univariate nonlinear function, the feature representation capability is improved. This design enables the network to combine different features more flexibly and improve the model's adaptability to different types of defects. In contrast, YOLOv7-tiny performs poorly in small object detection (mAPs=6.92), which may be related to its insufficient optimization of data augmentation strategy.

[0099] In summary, the embodiments of the present invention have fully demonstrated their effectiveness in improving the detection accuracy of small cracks, optimizing the allocation of computing resources, and enhancing the ability to identify different types of defects, providing strong technical support for automated inspection of road defects, safety early warning for autonomous driving, and intelligent road maintenance.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting road surface defects based on an improved YOLOv8 model, characterized in that, Includes the following steps: Acquire an image of the road surface to be detected; A multi-scale deformable attention module is embedded into the original CSPDarknet-53 structure to perform multi-scale feature extraction on the road surface image to be detected; utilizing... The adaptive feature adjustment module processes the extracted features to ensure that computational resources are focused on the defect region; the KANFusion-Neck module is used for feature fusion processing, which is configured with a linear-nonlinear fusion block LinNonFusion Block to replace the original DarknetBottleneck in PAFPN and enhance the feature extraction capability; the detection head is used to perform regression and classification tasks on the fused features. Based on predictions from classification tasks and regression tasks, pavement defect detection results are obtained. The step of processing the extracted features using the adaptive feature control module to ensure that computing resources are focused on the defect area is as follows: Based on the extracted features, the local gradient rate of change is calculated to measure the complexity of the features. A gradient calculation method is used. ； ； ； in Represents pixels in the input feature map The value, Representing the gradient in the horizontal direction, it calculates the difference between left and right adjacent pixels, reflecting horizontal edge information. Representing the gradient in the vertical direction, it calculates the difference between adjacent pixels, reflecting vertical edge information. The gradient magnitude represents the edge strength at that point, used to identify defect areas. A threshold T is set to distinguish between regions rich in detail and structurally stable regions. Perform adaptive region filtering based on gradient values. Generate region mask The calculation method is dynamically determined, and the calculation method is as follows: ； in A region mask is used to distinguish between complex and simple regions. This indicates that the region contains obvious edge features and belongs to a complex region. A global perception strategy is used for feature extraction. This indicates that the region is flat, has low computational requirements, and can be computed quickly using a local CNN. Since road surface defects may cause deformation, it is necessary to adjust the position of the feature window to ensure that the network focuses on the key area. The calculation method is as follows: ； in The calculated offset matrix controls the adjustment direction of the feature window. This represents a learnable parameter matrix that is continuously optimized during training to adapt to the defect states of different road surfaces. Multi-scale information integration is performed, combining local CNNs and global perception strategies to ensure optimal computational strategies for different regions. The computation method is as follows: ； in This represents the characteristics of the final output. Represents global characteristics. This represents the features extracted by a local CNN.

2. The pavement defect detection method based on the improved YOLOv8 model according to claim 1, characterized in that, The multi-scale deformable attention module includes an offset prediction network and an attention weight prediction network.

3. The pavement defect detection method based on the improved YOLOv8 model according to claim 2, characterized in that, The step of embedding a multi-scale deformable attention module on the original CSPDarknet-53 structure to perform multi-scale feature extraction on the road surface image to be detected is as follows: The multi-scale deformable attention module adopts a multi-scale feature pyramid architecture, including... Four scale levels are used to capture defect features in parallel at different spatial resolutions; For each spatial location in the feature map : First, the predicted offset is superimposed on a uniformly distributed reference point grid to obtain the deformed sampling position. The calculation formula is as follows: ； in For a reference point network with a regular distribution, The predicted offset. An adjustable scaling factor controls the range of position adjustment; Then, features are sampled at each scale level using a bilinear interpolation algorithm to obtain geometrically adapted feature representations; Next, the multi-scale attention weighted sum is calculated. For each scale s and attention head h, the output feature is calculated as follows: ； in The attention weights are normalized. Features sampled at deformed locations using bilinear interpolation. It is a fixed parameter that represents the attention span designed by the model for each point. It is a loop variable used to process each specific sampling point sequentially during the calculation process; Finally, all attention heads and output features at all scales are concatenated along the channel dimension. The channel dimension is adjusted by a linear projection layer and added to the input features through residual connections to keep the output feature size consistent with the input.

4. The pavement defect detection method based on the improved YOLOv8 model according to claim 1, characterized in that, The steps for feature extraction using a global awareness strategy are as follows: A self-attention mechanism is used to extract remote information, enhancing the feature representation ability of defective regions. The calculation method is as follows: ； ； ； Calculate feature correlation: ； Calculate the weighted feature output: ； in, Given the input feature matrix, , , These represent the query, key, and value matrices, respectively. , , These represent the learnable weight matrix, The scaling factor represents the feature dimension and prevents the gradient from becoming too large. This represents the attention weight matrix, which is normalized using softmax and then used for weighted calculation. It is the final feature output, which contains remote dependency information.

5. The pavement defect detection method based on the improved YOLOv8 model according to claim 1, characterized in that, The step of using KANFusion-Neck for feature fusion processing, wherein KANFusion-Neck is configured with LinNonFusionBlock to replace the original DarknetBottleneck in PAFPN and enhance feature extraction capabilities, specifically includes: Input features are processed Convolution reduces dimensionality: ； in The input features are of size . , represent Convolution is used to reduce computation while preserving key information. These are the features after dimensionality reduction, with a size of , ; To improve feature fusion capabilities, the dimensionality-reduced features... The specific steps for performing WNLC transformation are as follows: ； in These are learnable weights used to control the contributions of different features. It is a nonlinear transformation function, defined as: ； in It is a learnable parameter matrix. It is a bias term. It is a non-linear activation function; Features after WNLC transformation pass Convolution expands back to the original number of channels: ； in It is the output feature, and the input feature. Having the same number of channels C, and in order to ensure smooth gradients and prevent information loss, residual connections are introduced: 。

Images