Pavement crack detection method based on star-yolol1

Abstract

The application relates to a Star-YOLO11-based road surface crack detection method, which comprises the following steps: acquiring a road surface image to be detected; inputting the road surface image to be detected into a Star-YOLO11 model to output a road surface crack detection result, wherein the Star-YOLO11 model is obtained by improving a YOLO11 model and training a training set, the improved YOLO11 model comprises introducing a StarBlock into a backbone network, introducing an attention mechanism into a feature refinement layer, and connecting feature information based on a pyramid structure in a neck network, and the training set comprises road surface crack images containing crack labels. The Star-YOLO11 model based on YOLO11n is designed in combination with a road surface crack detection scene, can provide algorithm support for light mobile devices, effectively improves the efficiency and precision of road surface crack detection, and provides effective support for device deployment and actual application of road surface detection.

Description

Technical Field

[0001] This invention relates to the field of pavement crack detection technology, and in particular to a pavement crack detection method based on Star-YOLO11. Background Technology

[0002] Currently, highway defect surveys still rely on manual methods, with a low degree of automation. The results are prone to errors and deviations due to human subjective factors, resulting in low reliability and efficiency, and making it impossible to maintain highway defects in a timely manner.

[0003] YOLO (YouOnlyLookOnce) is a single-stage object detection network framework. Compared to traditional multi-stage detection algorithms such as Faster R-CNN and Mask R-CNN, all computations are completed in a single forward propagation. This gives YOLO a significant advantage in real-time detection, maintaining high speed while ensuring accuracy is not much lower than multi-stage detection algorithms. Since its introduction, it has been enthusiastically received by researchers. After multiple iterations, YOLO has continuously optimized its detection accuracy and speed, making a significant contribution to the field of object detection. Su Weiguo et al. studied the application of the YOLOv3 deep learning algorithm to a road crack detection model, verifying its effectiveness. Qiu et al. compared the performance of YOLOv2, YOLOv3, and YOLOv4-tiny in road crack detection. Xing et al. proposed an improved YOLOv5 algorithm called EMG-YOLO for road crack detection on edge computing devices. By optimizing the model structure and loss function, they improved the accuracy and efficiency of crack detection. Zhang et al. proposed an improved YOLOv5 algorithm called USSC-YOLO, which improves the detection accuracy and robustness of multi-scale cracks in complex backgrounds by introducing ShuffleNetV2 blocks, coordinate attention (CA) mechanisms, and SwingTransformer blocks. Chen et al. improved the accuracy and efficiency of road crack detection by introducing the attention mechanism module CBAM, decoupled head, and improved αDIoU loss function into the YOLOv5s algorithm. Bai et al. improved AC-YOLO based on YOLOv8s for UAV aerial road crack detection, improving the accuracy and efficiency of road crack detection by introducing a dynamic large convolutional kernel attention mechanism, multi-scale feature fusion strategy, and improved loss function. Su et al. made significant improvements to the YOLO framework, proposing a new algorithm called MOD-YOLO, which improves the accuracy and real-time performance of crack detection in civil infrastructure by preserving original feature information, expanding the receptive field to the global level, and incorporating an attention mechanism. Youwai et al.'s YOLO9tr, which incorporates a partial attention mechanism, maintains high accuracy and fast inference in road damage detection despite a small number of model parameters. Meng et al. improved the lightweight nature of the YOLOv8 model in road crack detection through knowledge distillation and random learning strategies. Mulyanto et al. proposed a lightweight, fast ground crack detection method based on YOLOv8, which improved accuracy, F1 score, and FPS while reducing the number of model parameters.Pei et al. proposed the YOLO-RDD pavement defect detection model based on YOLOv8 by introducing novel feature extraction and fusion methods, a dynamic snake convolution (DSC-C2f) module, and a coordinate attention mechanism, which improved the detection accuracy and real-time performance of pavement cracks and other defects. Although Transformer-related models have great potential in the detection field, their large number of parameters remains a significant limitation.

[0004] In summary, applying deep learning technology to the field of pavement crack detection is significant. Current research mainly focuses on improving the detection accuracy of single pavement defects while neglecting the insufficient detection accuracy when multiple crack defects coexist. Furthermore, the parameters of the resulting models are becoming increasingly large, and many models cannot be effectively deployed on lightweight mobile devices for real-time detection. To improve the real-time and accurate detection of multiple crack defects, this invention provides a pavement crack detection method based on Star-YOLO11. Summary of the Invention

[0005] The purpose of this invention is to provide a road surface crack detection method based on Star-YOLO11, which effectively improves the efficiency and accuracy of road surface crack detection and provides effective support for the equipment deployment and practical application of road surface detection.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] The pavement crack detection method based on Star-YOLO11 includes:

[0008] Acquire the image of the road surface to be detected;

[0009] The road surface image to be detected is input into the Star-YOLO11 model, and the road surface crack detection result is output. The Star-YOLO11 model is obtained by improving the YOLO11 model and training it with a training set. The improved YOLO11 model includes introducing StarBlock in the backbone network, introducing an attention mechanism in the feature refinement layer, and connecting feature information based on a pyramid structure in the neck network. The training set includes road surface crack images containing crack labels.

[0010] Optionally, obtaining the training set includes: obtaining a dataset of original road surface crack images, performing traditional data augmentation and complex data augmentation on the original road surface crack images, and obtaining the augmented images as the training set. The traditional data augmentation includes center cropping, vertical flipping, and horizontal flipping operations, and the complex data augmentation includes adding noise, brightness and saturation, grayscale, and environmental factors.

[0011] Optionally, introducing StarBlock into the backbone network includes replacing the C3k2 module with StarBlock in the backbone network, wherein the third extraction block for extracting pavement crack features is stacked with three layers of StarBlock, and one layer of StarBlock is used in each of the remaining extraction blocks.

[0012] Optionally, introducing an attention mechanism in the feature refinement layer includes adding a CPCA attention mechanism to the end of the original C2PSA structure in the feature refinement layer, wherein the CPCA attention mechanism uses multi-scale depth-separable convolutional modules to form spatial attention.

[0013] Optionally, the CPCA attention mechanism employs multi-scale, depthwise separable convolutional modules to construct spatial attention, including:

[0014] By employing average pooling and max pooling operations through channel attention, spatial information is gathered from the feature map, input into a shared MLP, and summed to generate a channel attention map.

[0015] Channel priors are obtained by element-wise multiplication of input features and channel attention maps;

[0016] The channel prior is input into the deep convolution module, and after passing through three branches, the multi-scale features are added to generate a spatial attention map.

[0017] Optionally, the pyramid structure obtains information by adding two connection blocks and multiple branches to the PAFPN to connect the bottom and top layers, including several surface auxiliary fusion modules and several high-level auxiliary fusion modules;

[0018] The surface-level auxiliary fusion module is used to connect to the information transmitted from the backbone network to the neck network;

[0019] The advanced auxiliary fusion module is used to connect and fuse features of different scales within the neck network.

[0020] Optionally, the information transmitted from the backbone network to the neck network by the surface-level auxiliary fusion module includes:

[0021] The output feature map of the next layer and the output feature map of the next layer surface auxiliary fusion module of the backbone network are aligned and connected with the input feature map of the current layer of the backbone network through downsampling and upsampling, respectively.

[0022] The input feature map of the current layer of the backbone network, the output feature map of the previous layer in the aligned and connected backbone network, and the feature map of the next layer surface auxiliary fusion module are concatenated and fused through the Concat module to output the output feature map of the current layer surface auxiliary fusion module.

[0023] Optionally, the advanced auxiliary fusion module connects the fusion of features at different scales within the neck network, including:

[0024] The output feature maps of the previous layer surface auxiliary fusion module and the previous layer high-level auxiliary fusion module are downsampled, and the output feature map of the next layer surface auxiliary fusion module is upsampled and then aligned and connected with the output feature map of the current layer surface auxiliary fusion module.

[0025] The output feature maps of the previous layer surface auxiliary fusion module, the previous layer high-level auxiliary fusion module, the next layer surface auxiliary fusion module, and the current layer surface auxiliary fusion module are spliced ​​and fused using the Concat module after alignment and connection.

[0026] The beneficial effects of this invention are as follows: To achieve the task of road surface crack detection, and addressing the problems of numerous types of road surface defects, difficulty in extracting features from different types of defects, and slow defect recognition speed in road surface defect detection, a new method for road surface crack detection based on Star-YOLO11 is proposed. By replacing the C3k2 module with a multi-layer StarBlock in the feature extraction backbone network, the implicit dimension space of the features is expanded, enabling the capture of more effective information. Then, a channel prior convolutional attention mechanism is introduced into the feature refinement layer, combined with the PSA self-attention mechanism, to form a CPC-C2PSA structure. This strengthens the attention to crack defects with a small pixel proportion in the image at the weakest end. Finally, based on the idea of ​​a multi-auxiliary branch feature pyramid structure, a neck deep and shallow layer connection is constructed to ensure that feature fusion can simultaneously obtain information from deeper and shallower layers, thereby improving the detection effect of the model for small and medium-sized target cracks. In order to make the experimental dataset have good robustness and increase the generalization ability of the model, this paper designs a dataset augmentation method that combines traditional data augmentation with complex data augmentation. Through various comparative experiments and ablation experiments, the effectiveness of the dataset augmentation method and the Star-YOLO11 algorithm is verified. This invention can meet the actual road surface crack detection task and provide strong support for road inspection. Future work can focus on deploying the model to mobile devices such as mobile phones or drones for road surface crack detection tasks. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a structural diagram of the Star-YOLO11 according to an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of the dataset augmentation strategy according to an embodiment of the present invention;

[0030] Figure 3 This is a comparison diagram of different backbone network architectures in embodiments of the present invention;

[0031] Figure 4 This is a structural diagram illustrating the working principle of CPC-Attention in an embodiment of the present invention.

[0032] Figure 5 This is a structural diagram of the CPC-C2PSA fusion attention module according to an embodiment of the present invention;

[0033] Figure 6 The following are network structure diagrams of PAFPN, BiFPN, and SMAFPN according to embodiments of the present invention, wherein (a) is a PAFPN structure diagram, (b) is a BiFPN structure diagram, and (c) is an overall SMAFPN structure diagram;

[0034] Figure 7 This is a structural diagram of SAF and AAF according to an embodiment of the present invention;

[0035] Figure 8 The following are example images of five categories in the dataset of this invention: (a) longitudinal cracks, (b) transverse cracks, (c) mesh cracks, (d) pits, and (e) repairs.

[0036] Figure 9 The diagram illustrates some of the enhancement effects of this invention, where (a) is the original image, (b) is vertical flip + shadow, (c) is vertical flip + snowy weather, (d) is horizontal flip + brightness, and (e) is center crop + Gaussian blur.

[0037] Figure 10 This is a graph showing the dynamic change of the training loss value in an embodiment of the present invention.

[0038] Figure 11 This is a graph showing the dynamic change of the validation set loss value in an embodiment of the present invention.

[0039] Figure 12 This is a dynamic curve showing the average detection accuracy of an embodiment of the present invention.

[0040] Figure 13 These are the detection result indicators for each category of Star-YOLO11 in this embodiment of the invention;

[0041] Figure 14 The figure shows the experimental results comparing different algorithms in this embodiment of the invention;

[0042] Figure 15The following are heatmap comparison diagrams of different algorithms in embodiments of the present invention, wherein (a) is the original image, (b) is the YOLO11 heatmap, and (c) is the Star-YOLO11 heatmap.

[0043] Figure 16 The images shown are a comparison of detection images from different algorithms in this embodiment of the invention, where (a) is the original image, (b) is the YOLO11 detection image, and (c) is the Star-YOLO11 detection image. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] This embodiment provides a pavement crack detection method based on Star-YOLO11, including:

[0047] Acquire the image of the road surface to be detected;

[0048] The Star-YOLO11 model inputs the road surface image to be detected and outputs the road surface crack detection results. The Star-YOLO11 model is obtained by improving the YOLO11 model and training it on a training set. The improved YOLO11 model includes introducing StarBlock into the backbone network, introducing an attention mechanism into the feature refinement layer, and connecting feature information based on a pyramid structure in the neck network. The training set includes road surface crack images with crack labels. The structure of the Star-YOLO11 model is as follows: Figure 1 As shown.

[0049] Specifically, to ensure the dataset effectively simulates real-world environments, a dataset augmentation strategy was designed to provide the model with sufficient training data to improve generalization ability. To enhance the detection performance of road surface cracks, a Star-YOLO11 model was constructed. Addressing the issue of the large number of C3k2 modules used in the backbone network, resulting in mediocre detection performance and a large number of parameters, a proposal was made to replace the C3k2 modules with multiple StarBlocks, forming StarStage blocks with downsampling convolutions. This element-wise multiplication method improves the ability to extract image features while maintaining a lower parameter count. Considering the significant pixel proportion of road surface cracks and background, a third CPCA attention mechanism was added after the original two PSA attention layers in the feature refinement layer to strengthen the focus on small and inconspicuous cracks. To address the problem of the neck layer's inability to adaptively integrate high-level semantic information and low-level spatial information, SMAFPN was proposed, which adds two C3k2 modules and strengthens the connection between the neck and the backbone StarBlocks through multi-branch assistance.

[0050] Furthermore, obtaining the training set includes: obtaining the original pavement crack image dataset, performing traditional data augmentation and complex data augmentation on the original pavement crack images, and obtaining the augmented images as the training set. Among them, traditional data augmentation includes center cropping, vertical flipping and horizontal flipping operations, and complex data augmentation includes adding noise, brightness and saturation, grayscale and environmental factors.

[0051] Specifically, to ensure sufficient training volume for the model, a large and high-quality dataset is required. Data augmentation techniques reduce the time spent manually annotating images by transforming existing data, such as rotating, scaling, and cropping, to expand the training dataset. Data augmentation methods are divided into traditional data augmentation, which operates on image size and position, and complex data augmentation, which simulates environmental changes. However, existing research often uses only traditional or complex data augmentation methods, failing to effectively combine the advantages of both. Therefore, this embodiment proposes a new dataset augmentation strategy. It superimposes commonly used complex data augmentation elements such as Gaussian noise, ISO noise, brightness and saturation, and grayscale onto the traditional data augmentation operations of center cropping, vertical flipping, and horizontal flipping. Furthermore, considering the significant impact of weather on the model in practical applications, five additional environmental factors—fog, rain, snow, sunlight, and shadow—are added, resulting in a total of 12 augmentation methods. These can be combined to form up to 27 augmentation strategies, such as... Figure 2As shown, each image can be augmented with either a traditional data augmentation or a complex data augmentation, making each image in the dataset more complex, which can further enrich the dataset and enhance the model's generalization ability.

[0052] Furthermore, the introduction of StarBlock into the backbone network includes replacing the C3k2 module with StarBlock in the backbone network. Specifically, the third extraction block for extracting pavement crack features is stacked with three layers of StarBlock, and one layer of StarBlock is used in each of the remaining extraction blocks.

[0053] Specifically, the feature extraction backbone network, as a crucial stage in YOLO11 that transitions from lower-level to higher-level information, is designed to extract as much information as possible, leading to better subsequent detection results. However, due to model size limitations, the feature extraction block C3k2 used in the original YOLO11 cannot extract more information within the limited model size. Therefore, based on the element-wise multiplication concept in StarBlock, a stacked StarBlock structure is used to replace the original C3k2 module, increasing the model's detection accuracy. Depthwise separable convolutions are used before and after multiplication instead of ordinary 1*1 convolutions, reducing the computational parameters of the backbone network.

[0054] StarBlock has a very simple and efficient structure. Its core idea is element-wise multiplication, which is usually written as follows in a single-layer neural network: Where W represents the weight matrix and B represents the bias, for ease of representation, the weight matrix and bias are combined into a single entity W = [WB]. T , and X = [X 1] T Thus obtain By fusing the features of two linear transformations through element-wise multiplication, this method is applicable to single-output channel conversion and single-element input scenarios. Where d is the number of input channels, which extends to scenarios with multiple output channels and multiple element inputs. The following steps can be used to rewrite the code to obtain:

[0055]

[0056] Where i and j are channel indices, and α is the coefficient of each sub-item:

[0057]

[0058] After rewriting, we can obtain a combination of (d+2)(d+1) / 2 different sub-terms as shown in Equation 4, except for α. (d+1,:) x d+1Each term of x is non-linearly related to x, indicating that they are independent implicit dimensions, which can be obtained by element-wise multiplication in d-dimensional space. StarBlock has an implicit dimensional feature space, so it can significantly amplify the feature dimension without generating any additional computational overhead in a single layer. In the road crack detection task, it can obtain more dimensional information compared with C3k2.

[0059] This embodiment designs a stacked number of StarBlock layers in the backbone network, naming the improved backbone network Star-s50. Compared with the original YOLO11 backbone network and the Starnet-s1 and s2 backbone networks provided in StarBlock-related literature, Star-s50 has a lighter architecture and higher feature map extraction capability. It only stacks three StarBlock layers in the third extraction block, which focuses on extracting road crack features, while using one StarBlock layer each for image input and feature map output extraction to retain a larger receptive field. In subsequent experiments, this significantly improves the information observation capability of the backbone network and maintains the model's lightweight design. With the same number of parameters, it demonstrates better detection capabilities for complex scenes than the C3k2 module. Figure 3 As shown.

[0060] Furthermore, the introduction of an attention mechanism in the feature refinement layer includes adding a CPCA attention mechanism to the end of the original C2PSA structure in the feature refinement layer. The CPCA attention mechanism uses multi-scale depth-separable convolutional modules to form spatial attention.

[0061] Furthermore, the CPCA attention mechanism employs multi-scale, depthwise separable convolutional modules to construct spatial attention, including:

[0062] By employing average pooling and max pooling operations through channel attention, spatial information is gathered from feature maps, input into a shared MLP, and then summed to generate a channel attention map.

[0063] Channel priors are obtained by element-wise multiplication of input features and channel attention maps;

[0064] The channel prior is input into the deep convolution module, and after passing through three branches, the multi-scale features are added together to generate a spatial attention map.

[0065] Specifically, CPCA (Channel Prior Convolutional Attention) is a novel attention mechanism that combines channel attention and spatial attention. Compared to SE attention, which only focuses on the channel dimension and completely ignores spatial dimension information, and CBAM attention, which only generates single spatial information, CPCA uses multi-scale, depthwise separable convolutional modules to construct spatial attention. It can dynamically allocate attention weights on the channel and spatial dimensions, effectively extracting spatial relationships while preserving channel priors. It has a strong ability to focus on information channels and important regions. Its structure is as follows: Figure 4 As shown.

[0066] By employing a strategy of first channel attention and then spatial attention, as shown in equations (6)-(7), the channel attention part uses average pooling and max pooling operations to gather spatial information from the feature map, and then inputs it into a shared MLP (Multilayer Perceptron), which is then summed to generate a channel attention map, as shown in equation (8). Channel priors are obtained by multiplying the input features and the channel attention map element-wise. Subsequently, the channel priors are input into the deep convolution module, and after three branches, the multi-scale features are summed to generate a spatial attention map, as shown in equation (9), where F represents the feature map, F c This represents the channel prior refined feature map. The final feature map is represented by CA, SA, σ, and the three branches of spatial attention. Finally, the channel blending result and the channel prior are multiplied element-wise to obtain the final refined feature map.

[0067]

[0068] The original YOLOv11 introduced a C2PSA structure, nesting two layers of PSA partial self-attention mechanisms proposed in YOLOv10, which demonstrated significant global modeling capabilities. However, because self-attention cannot consider the local structure of the data, it cannot efficiently focus on target information with many local features, such as pavement cracks. Therefore, a CPCA attention mechanism was added to the end of the original C2PSA structure to form a CPC-C2PSA model structure, strengthening the weakest link in the data. Figure 5 As shown.

[0069] Furthermore, the pyramid structure obtains information between the bottom and top layers by adding two connection blocks and multiple branches on the basis of PAFPN, including several surface-level auxiliary fusion modules and several high-level auxiliary fusion modules.

[0070] The surface-level auxiliary fusion module is used to connect the information transmitted from the backbone network to the neck network.

[0071] Advanced auxiliary fusion module for fusing features at different scales within the neck network.

[0072] Furthermore, the information transmitted from the backbone network to the neck network by the surface-assisted fusion module includes:

[0073] The output feature map of the next layer and the output feature map of the next layer surface auxiliary fusion module of the backbone network are aligned and connected with the input feature map of the current layer of the backbone network through downsampling and upsampling, respectively.

[0074] The input feature map of the current layer of the backbone network, the output feature map of the previous layer in the aligned and connected backbone network, and the feature map of the next layer surface auxiliary fusion module are concatenated and fused through the Concat module to output the output feature map of the current layer surface auxiliary fusion module.

[0075] Furthermore, the advanced auxiliary fusion module connects the fusion of features at different scales within the neck network, including:

[0076] The output feature maps of the previous layer surface auxiliary fusion module and the previous layer high-level auxiliary fusion module are downsampled, and the output feature map of the next layer surface auxiliary fusion module is upsampled and then aligned and connected with the output feature map of the current layer surface auxiliary fusion module.

[0077] The output feature maps of the previous layer surface auxiliary fusion module, the previous layer high-level auxiliary fusion module, the next layer surface auxiliary fusion module, and the current layer surface auxiliary fusion module are spliced ​​and fused using the Concat module after alignment and connection.

[0078] Specifically, since the PAFPN used in the original YOLO11 neck tends to merge feature maps of the same scale, it lacks comprehensive processing and fusion of multi-scale information from different resolution layers, and cannot simultaneously and effectively adaptively integrate high-level semantic information and low-level spatial information, a Star Multi-Branch Auxiliary Feature Pyramid Network (SMAFPN) based on MAFPN and connected to StarBlock in the backbone network is proposed. SMAFPN does not use the RepHELAN module designed by MAFPN, but instead uses the C3k2 module in the original YOLO11 to suit the network structure of YOLO11. The connection is divided into two parts: the first part is Superficial Assisted Fusion (SAF), which is mainly responsible for connecting the information input from the backbone network to the neck, and the second part is Advanced Assisted Fusion (AAF), which mainly connects the fusion of features at different scales within the neck. Figure 6 (a)-(c) show the structural comparison of the improved feature fusion structure with the original PAFPN and the currently popular feature fusion module biFPN in the YOLO architecture. Based on PAFPN, biFPN adds an extraction block at the P3 layer, which introduces the same layer information in the backbone network into the extraction block of the previous stage of the detection head. SMAFPN, on the other hand, adds two connection blocks and multiple branches to connect the information between the bottom and top layers on the basis of PAFPN, thereby preserving the shallow information of StarBlock and fusing it with the deep information of the neck layer, thus improving the detection effect of small target road surface cracks.

[0079] like Figure 7 and Figure 6 As shown in (c), the connection of SAF mainly consists of four parts, including the input feature map P of the current layer of the backbone network. n The feature map P′ output by the current layer surface auxiliary fusion module n The feature map P output by the previous StarStage (each layer of the improved backbone network is called a StarStage) n-1 By downsampling and combining the input feature map P of the current layer of the backbone network n Perform alignment connections and output feature map P′ from the next layer surface auxiliary fusion module. n+1 After upsampling and combining with the current layer input feature map P of the backbone network n Align the connections, and then use the output feature map P of the current layer of the backbone network. n The StarStage output feature map P in the backbone network n-1 The next layer surface auxiliary fusion module outputs feature map P′n+1 The three inputs are concatenated and fused using the Concat module, integrating deep information with features from the same level in the backbone and high-resolution shallow layers. This preserves rich localization details to enhance the network's spatial representation and improves its ability to detect smaller targets. The output is shown in Equation (10), where U(·) represents the upsampling operation, Down represents a 3×3 downsampling convolution with batch normalization, δ represents the siLU activation function, and C represents a 1×1 convolution controlling the number of channels.

[0080] P′ n =Concat(δ(C(Down(P)) n-1 )),P n ,U(P′ n+1 )#(10)

[0081] AAF further enhances the interactive utilization of feature layer information by integrating multi-scale information in the neck region. It mainly consists of five parts, in addition to the feature map P′ output by the current layer surface auxiliary fusion module. n The current layer high-level auxiliary fusion module outputs feature map P″. n The feature map P″ output by the higher-level auxiliary fusion module n-1 In addition, a next-layer surface auxiliary fusion module was added to output feature map P′. n+1 The feature map P′ output by the surface auxiliary fusion module of the previous layer n-1 P′ n+1 The feature map P′ is output by upsampling and the current layer surface auxiliary fusion module. n Perform alignment and connection, P′ n-1 and P″ n-1 By downsampling and feature map P′ n Perform alignment connections, and then connect them with feature map P′. n They are spliced ​​and fused together using the Concat module, and the output result is shown in Equation (11). At this time, the Medium output can merge information from four different layers, which significantly enhances the performance of medium-sized targets.

[0082]

[0083] The specific experimental process of the method in this embodiment is provided below:

[0084] 1. Dataset preparation:

[0085] The experiment used the publicly available RDD2022 China dataset, a dataset of road surface cracks captured by drones and non-motorized vehicles equipped with smart cameras. This dataset includes five types of typical road surface cracks: longitudinal cracks, transverse cracks, alligator cracks, potholes, and repairs. Figure 8 As shown in (a)-(e).

[0086] The dataset was divided into training, validation, and test sets in an 8:1:1 ratio, comprising 3502 images for training, 437 images for validation, and 439 images for test. The training set was then augmented using a defined dataset augmentation strategy. Each image underwent one of the following enhancements: center cropping, vertical flipping, or horizontal flipping, plus one of the following: Gaussian noise, ISO noise, brightness / saturation adjustment, rain / fog / snow effect, shadow effect, sunlight effect, or grayscale effect. Some of the augmented images are illustrated below. Figure 9 As shown in (a)-(e).

[0087] The expanded training set contains 17,494 images, which can improve the quantity and quality of labels for different categories and enhance the generalization ability of the model. The label types and quantities in the enhanced dataset are shown in Table 1.

[0088] Table 1

[0089]

[0090] 2. Training environment:

[0091] To verify the effectiveness of the method proposed in this embodiment, PyCharm was selected as the programming environment, using the Windows 10 operating system, Python 3.10 as the programming language, PyTorch 2.3.0 as the deep learning framework, and CUDA version 11.8. The hardware testing environment for training the model consisted of an 11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz processor, 16GB of memory, and an NVIDIA RTX 4060Ti GPU (16GB of VRAM). Specific training hyperparameter settings are shown in Table 2.

[0092] Table 2

[0093]

[0094] 3. Evaluation indicators:

[0095] To comprehensively evaluate the model, the evaluation metrics used include precision (P), which reflects the model's ability to distinguish negative samples; recall (R), which reflects the model's ability to identify positive samples; mean average precision (mAP), which measures the model's accuracy, with mAP50 indicating that the intersection-union ratio of the ground truth bounding box and the predicted bounding box is greater than 50, indicating that the model's prediction is correct; F1 score, which measures the overall performance and stability of the model; parameters, which measures the size and complexity of the model; and gigabit-plus-float (GFLOPs), which represents billions of floating-point operations per second and measures the performance of computing devices when performing floating-point operations. The specific calculation formula is shown in equation (12):

[0096]

[0097] 4. Training process analysis:

[0098] The training process of deep learning models can be dynamically monitored using two core metrics: the loss function and the average detection accuracy (mAP). Lower loss values ​​and higher average detection accuracy generally indicate better model performance. For example... Figure 10 and Figure 11 As shown in the training and validation loss curves, the model can be roughly divided into three stages over a total of 200 iterations: rapid descent, asymptotic convergence, and steady-state convergence. In the early stages of training, the loss is high because the model struggles to accurately fit the probability distribution of the predicted boxes. As training progresses, the probability distribution of the predicted boxes gradually optimizes. In the initial rounds, the loss exhibits an exponentially rapid decline, followed by asymptotic convergence where the rate of decline gradually slows down, eventually leveling off at steady-state convergence, indicating that the model has reached a good training state.

[0099] Figure 12 The average detection accuracy curve validated during model training is shown. Overall, there are no significant fluctuations, and the curve has a smooth shape, which demonstrates the effectiveness of the algorithm architecture and the reliability of the dataset. The average detection accuracy curve eventually converges, proving that the model training is complete.

[0100] 5. Test Result Analysis:

[0101] To verify the effectiveness of the dataset augmentation strategy designed in this embodiment, the original YOLO11n model was trained using both the un-augmented and augmented training sets, and then tested and compared on the same test set. The comparison of the test results before and after training set augmentation is shown in Table 3. It can be seen that all indicators are improved. The dataset augmentation strategy proposed in this embodiment can significantly improve the generalization ability of the model and has high robustness.

[0102] Table 3

[0103]

[0104] To verify the effectiveness of the Star-s50 backbone network, experiments were conducted on YOLO11n with backbone network replacement. All tests were performed using the enhanced training set and the same test set. The comparison of test results for different backbone network replacements is shown in Table 4.

[0105] Table 4

[0106]

[0107] It can be seen that the Star-s50 backbone network proposed in this embodiment achieves 88.3% on mAP50, which is better than other backbone networks. Subsequently, experiments were conducted on YOLO11n using Star-s50 to compare different attention mechanisms to demonstrate the superiority of the CPC-C2PSA attention structure fused in this embodiment. The comparison of test results for different attention fusions is shown in Table 5.

[0108] Table 5

[0109]

[0110] As shown in Table 5, the model incorporating the CPCA attention mechanism achieved an mAP50 of 90%, far exceeding other attention mechanisms. To verify the effectiveness of the feature fusion neck structure improvement, controlled variable experiments were conducted on YOLO11n using star-s50 and CPC-C2PSA, comparing SMAFPN with PAFPN and biFPN. The test results for different feature fusion neck structures are shown in Table 6.

[0111] Table 6

[0112]

[0113] As shown in Table 6, the feature fusion structure of SMAFPN effectively improves the model's accuracy to 90.3%. When using biFPN, the accuracy is only slightly better than PAFPN, and far inferior to the SMAFPN structure used in this embodiment. Finally, to verify the effectiveness of Star-YOLO11, ablation experiments were conducted based on YOLO11n. "√" indicates the use of this module. The ablation experiment results are shown in Table 7. The precision, recall, and average detection accuracy of the method proposed in this embodiment in each category of detection results are as follows: Figure 13 As shown.

[0114] Table 7

[0115]

[0116] As shown in Table 7, all three modules effectively improve the detection accuracy of the model. The Star-s50 backbone network significantly reduces the number of YOLO11 model parameters and greatly improves mAP50, which is significantly better than the original YOLO11 backbone network. The introduction of the CPCA attention mechanism effectively improves the model's accuracy, deepens the focus on small target cracks and inconspicuous cracks, and, combined with Star-s50, can achieve an mAP50 of up to 90%, and still improves by nearly one point in mAP50-95. The introduction of SMAFPN allows the model to pay attention to the interaction between shallower and deeper information. Although it does not have a significant advantage compared to the combination of the other two models, the sum of the three can achieve a P-value of up to 89.9%, an mAP of up to 90.3%, and an mAP50-95 of up to 62.7%. Meanwhile, to verify the superiority of the improved model compared with other mainstream algorithms, Star-YOLO11 was compared with several other classic algorithms on the same dataset. The comparison test results of different algorithms are shown in the figure below. Figure 14 As shown in Table 8, Star-YOLO11 demonstrates superior performance across all key metrics compared to other network models of similar size, with the exception of recall, where all other metrics are optimal. Furthermore, in addition to improved detection performance, Star-YOLO11 also offers significant advantages over the original YOLO11 in terms of algorithm model size and real-time computation speed. A comparison of the model data is shown in Table 8.

[0117] Table 8

[0118]

[0119] After comparing the data during model testing, it was found that the improved Star-YOLO11 model has reduced its size and the total number of parameters has decreased by 18.8%. Theoretically, it can be well applied to the algorithm of lightweight mobile devices. The model's F1 score has also been slightly improved. Although its FPS is slightly lower than the original model, it still remains above 200. Among current models, its detection speed is far ahead and can well meet the actual detection needs.

[0120] 6. Visual Analysis:

[0121] Based on the experimental data analysis, to intuitively demonstrate the improvement effect of the proposed method and observe the degree of optimization of the model by each improvement, a visualization analysis was performed on the key experimental steps. To verify the model's information capture and feature extraction capabilities, as well as the optimization of the receptive field by channel prior convolutional attention, a heatmap comparison was conducted between the model before and after the improvement, such as... Figure 15 (a)-(c). Figure 15The color intensity differences in different regions visually reflect the significant difference in the attention paid to pavement cracks before and after the model improvement. The Star-YOLO11 model exhibits higher global coverage in crack perception and a significantly enhanced focus on crack targets. To verify the model's detection performance in practical engineering applications, the detection performance of the model before and after the improvement on the test set was compared. The detection results are as follows: Figure 16 (a)-(c) The improved algorithms significantly improve the issues of missed detections and detection accuracy of the original YOLO11 model.

[0122] By designing a Star-s50 feature extraction backbone formed by stacked StarBlock layers, replacing the C3K2 extraction block used in the original YOLO11 backbone, the accuracy of road crack detection is improved while increasing the model's lightweightness. Then, to address the issue that the original PSA attention of YOLO11 cannot efficiently focus on target information with many local features, such as road cracks, a channel prior convolutional attention mechanism, CPC-Attention, is introduced, which multiplies the input features element-wise with the channel attention map. Finally, to address the problem that the neck of the original YOLO11 cannot adaptively integrate high-level semantic information and low-level spatial information, a SMAFPN feature pyramid structure based on MAFPN is proposed, which is connected to Star-50 and uses C3k2 as the extraction block. A dataset augmentation strategy combining traditional data augmentation and complex data augmentation is designed, and the model is trained and tested on a dataset including five types of road cracks. Test results show that the Star-YOLO11 model achieves an accuracy of 89.9%, an improvement of 3.5% compared to the original model; mAP reaches 90.3%, an improvement of 2.6%; F1 score reaches 85.8%, an improvement of 0.5%; model size is reduced by 18.8%; and FPS reaches 225.73. It is very fast in detection and can be effectively applied to pavement crack detection.

[0123] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A pavement crack detection method based on Star-YOLO11, characterized in that, include: Acquire the image of the road surface to be detected; The road surface image to be detected is input into the Star-YOLO11 model, and the road surface crack detection result is output. The Star-YOLO11 model is obtained by improving the YOLO11 model and training it with a training set. The improved YOLO11 model includes introducing StarBlock in the backbone network, introducing an attention mechanism in the feature refinement layer, and connecting feature information based on a pyramid structure in the neck network. The training set includes road surface crack images with crack labels. Introducing an attention mechanism into the feature refinement layer includes adding a CPCA attention mechanism to the end of the original C2PSA structure in the feature refinement layer, wherein the CPCA attention mechanism uses multi-scale depth-separable convolutional modules to form spatial attention. The CPCA attention mechanism employs multi-scale, depthwise separable convolutional modules to construct spatial attention, including: By employing average pooling and max pooling operations through channel attention, spatial information is gathered from the feature map, input into a shared MLP, and summed to generate a channel attention map. Channel priors are obtained by element-wise multiplication of input features and channel attention maps; The channel prior is input into the deep convolution module, and after passing through three branches, the multi-scale features are added to generate a spatial attention map; The pyramid structure obtains information by adding two connection blocks and multiple branches to the PAFPN, and includes several surface-level auxiliary fusion modules and several high-level auxiliary fusion modules. The surface-level auxiliary fusion module is used to connect to the information transmitted from the backbone network to the neck network; The advanced auxiliary fusion module is used to connect and fuse features of different scales within the neck network.

2. The pavement crack detection method based on Star-YOLO11 according to claim 1, characterized in that, Obtaining the training set includes: obtaining a dataset of original road surface crack images; performing traditional data augmentation and complex data augmentation on the original road surface crack images; and obtaining the augmented images as the training set. The traditional data augmentation includes center cropping, vertical flipping, and horizontal flipping operations, while the complex data augmentation includes adding noise, brightness and saturation, grayscale, and environmental factors.

3. The pavement crack detection method based on Star-YOLO11 according to claim 1, characterized in that, Introducing StarBlock into the backbone network includes replacing the C3k2 module with StarBlock in the backbone network, wherein the third extraction block for extracting road surface crack features is stacked with three layers of StarBlock, and one layer of StarBlock is used in each of the remaining extraction blocks.

4. The pavement crack detection method based on Star-YOLO11 according to claim 1, characterized in that, The information transmitted from the backbone network to the neck network by the surface-level auxiliary fusion module includes: The output feature map of the next layer and the output feature map of the next layer surface auxiliary fusion module of the backbone network are aligned and connected with the input feature map of the current layer of the backbone network through downsampling and upsampling, respectively. The input feature map of the current layer of the backbone network, the output feature map of the previous layer in the backbone network after alignment and connection, and the feature map of the next layer surface auxiliary fusion module are concatenated and fused through the Concat module to output the output feature map of the current layer surface auxiliary fusion module.

5. The pavement crack detection method based on Star-YOLO11 according to claim 1, characterized in that, The advanced assisted fusion module connects the fusion of features at different scales within the neck network, including: The output feature maps of the previous layer surface auxiliary fusion module and the previous layer high-level auxiliary fusion module are downsampled, and the output feature map of the next layer surface auxiliary fusion module is upsampled and then aligned and connected with the output feature map of the current layer surface auxiliary fusion module. The output feature maps of the previous layer surface auxiliary fusion module, the previous layer high-level auxiliary fusion module, the next layer surface auxiliary fusion module, and the current layer surface auxiliary fusion module are spliced ​​and fused using the Concat module after alignment and connection.

Images