Road disease detection method based on scale difference

Abstract

The invention relates to a road disease detection method based on scale difference. The method comprises the steps that after road disease image data are cleaned, a Darklabel marking tool is used for carrying out detailed disease position and category marking on a cleaned image; a deformable convolutional network module DCNv3 is introduced into a network structure based on the preprocessed data, a small-scale detector is added, an attention mechanism CBAM is introduced, the effective receptive field of the model is expanded, the feature extraction capability of targets of different scales is enhanced, and a loss function is improved; and finally, evaluating the training model based on a road disease detection result of an actual engineering scene, verifying the generalization of the training model, and carrying out road disease detection through the evaluated model. Compared with the prior art, the method has the advantages that the influence of environmental interference on a detection result is reduced, so that the accuracy and recall rate of road disease target detection are comprehensively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of road damage target detection, and in particular to a road damage detection method based on scale difference. Background Art

[0002] In the field of road maintenance and management, timely and accurate detection of road defects is a key link in ensuring traffic safety and extending the service life of roads. However, road defect detection faces many challenges, including a wide variety of defects, significant scale differences, and complex and changeable road environment interference factors (such as vehicle occlusion, shadow interference, light changes, etc.). These factors not only increase the difficulty of defect detection, but also reduce the robustness and accuracy of the detection algorithm.

[0003] Currently, deep learning-based target detection algorithms, especially single-stage target detection algorithms such as YOLOv8, have attracted much attention due to their good balance between speed and accuracy. YOLOv8 achieves efficient prediction of the entire image through its unique network architecture design, including input image, backbone feature extraction network, neck network, and top detection network. Its mechanism is to divide the image into S×S grids, each grid unit is responsible for detecting the target in its coverage area, and generates a bounding box with confidence, and then removes redundant boxes through the non-maximum suppression algorithm to improve the detection quality. The basic network structure of YOLOv8 can be found in Figure 1 .

[0004] However, in actual road damage detection, due to the diversity of damage categories and significant scale differences, the features of small-scale targets are easily lost after multiple convolutions and pooling, affecting the detection accuracy. Secondly, the complex road environment interference factors make the existing algorithms insufficient in generalization ability, prone to false positive detection or missed recognition, and reduce the reliability of damage detection. Summary of the invention

[0005] The purpose of the present invention is to provide a road damage detection method based on scale difference in order to overcome the defects of the above-mentioned prior art. Starting from the problems of high model complexity and insufficient extraction of shallow target information and boundary information by the current benchmark network model, in-depth research is carried out on the optimization design of multi-scale target characteristics of road damage in actual scenarios. By optimizing training measures and improving network structure, the precision and recall rate of the model are improved.

[0006] During the conception process, the present invention focused on improving the accuracy of road damage target detection. Starting from optimizing the model structure, enhancing the ability to extract shallow and boundary information, and improving data preprocessing, a road damage detection optimization method based on scale difference is proposed. By adjusting the network architecture and optimizing the training strategy, the model's ability to recognize multi-scale targets is improved. At the same time, data cleaning, labeling and enhancement techniques, such as shadow weakening and other preprocessing methods, are used to reduce the impact of environmental interference on detection results, thereby comprehensively improving the accuracy and recall rate of road damage target detection, and providing more accurate and reliable damage information for road maintenance.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] The present invention provides a road damage detection method based on scale difference, comprising the following steps:

[0009] S1: Acquire asphalt pavement image data containing typical pavement diseases, perform data cleaning and disease marking on the acquired road disease data, obtain a pavement disease image data set containing different resolutions, and perform data preprocessing and enhancement;

[0010] S2: Based on the YOLOv8 basic network architecture, an improved road disease target detection algorithm is constructed, including: optimizing the network structure of the C2f module in the model backbone feature extraction network, replacing the original convolution module with the deformable convolution module DCNv3 module, increasing the effective prediction area, replacing the 1×1 convolution of the FPN module with a 3×3 deformable convolution DCNv3 module, increasing the model's multi-scale adaptability to features, setting a multi-scale detector in the top detection network of the YOLOv8 basic network architecture and introducing the CBAM module to optimize the model's target feature extraction process in scenes with complex interference factors;

[0011] S3: Deploy the model test environment and adjust the model parameters to adapt to the diversity of road diseases in images and their size differences. Input the preprocessed images into the constructed network model for training, obtain the model training weights, and finally conduct testing and verification.

[0012] S4: Construct performance evaluation indicators and set up ablation experiments to verify the effectiveness of the deformable convolution module, CBAM attention mechanism, and small-scale detector on the accuracy of the target detection model and the optimization of the model accuracy. Use the evaluated model to detect road damage.

[0013] Furthermore, in S1, the specific process includes:

[0014] S1-1: Acquire asphalt pavement images on multiple urban highways, wherein the asphalt pavement images contain typical pavement diseases, perform data cleaning on the images, exclude occluded or blurred images, ensure image data quality, and use DarkLabel software to accurately mark the diseases;

[0015] S1-2: Preprocess and enhance the labeled image dataset to improve the model's ability to identify diseases under different conditions, unify the image resolution and perform normalization processing, and prepare a high-quality dataset for model training;

[0016] S1-3: Divide the normalized image data into training set, validation set, and test set.

[0017] Furthermore, in S2, the specific process includes:

[0018] S2-1: Optimize the network structure of the C2f module in the backbone feature extraction network of the YOLOv8 model, replace the original convolution module with the deformable convolution module DCNv3 module, increase the effective prediction area, and use the improved CSPDarknet53 structure of the C2f module as the backbone feature extraction network of the model to enhance the transmission and fusion efficiency of features;

[0019] S2-2: Replace the 1×1 convolution of the FPN module in the YOLOv8 model with a 3×3 deformable convolution DCNv3 module;

[0020] S2-3: setting four feature detector structures including large, medium, small and micro in the top detection network of the YOLOv8 model;

[0021] S2-4: A CBAM module is introduced into the top detection network of the YOLOv8 model to judge the importance of different channel features through the channel attention mechanism, and then the key features in the target area are further refined based on the spatial attention mechanism;

[0022] S2-5: The target category loss (losso), confidence threshold loss (lossi) and bounding box prediction regression loss (lossc) are used as loss functions, and the VFL loss function is introduced as the target category loss, the EIOU loss function as the confidence threshold loss, and the DFL loss function as the bounding box prediction regression loss.

[0023] Furthermore, in S2-1, the specific process includes:

[0024] In the backbone feature extraction network of the YOLOv8 model, the network structure of the C2f module is optimized, and the deformable convolution module DCNv3 module replaces the original convolution module to increase the effective prediction area.

[0025] The CSPDarknet53 structure improved by the C2f module is used as the backbone feature extraction network of the model to enhance the efficiency of feature transmission and fusion;

[0026] The deformable convolution structure is an improved network structure of conventional regular convolution. When the input x∈R C×H×W , currently pixel p0, is expressed as:

[0027]

[0028] In the formula, K is the total number of sampling points, k is the number of sampling points, and w k ∈R C×C represents the projection weight of the kth sampling point, m k ∈R represents the modulation scalar of the kth sampling point. After normalization by the sigmoid function, it is used to adjust the weight of the sampling point. k Represented as the k-th position (-1, 1), (-1, 0), ..., (0, +1), ..., (+1, +1) sampled in the predefined grid in the regular convolution, Δp k is the offset corresponding to the kth grid sampling position, the offset Δp k Can interact with short-range or long-range features, sampling offset Δp k and the modulation scalar m k are all constrained by x;

[0029] The deformable convolution decomposes the convolution weights into depth direction and point direction. The depth direction part is responsible for adjusting the modulation scalar m of the original position perception. k , control different channels of the feature map, and the point direction part uses the projection weight w shared between the sampling points to reduce the number of model parameters. The group convolution is introduced to divide the spatial aggregation process into G groups, each with an independent sampling offset Δp gk and modulation scale m gk , which enables the model to learn richer information in different representation subspaces, and each group can show different spatial aggregation patterns;

[0030] The DCNv3 replaced in the C2f module can be expressed as:

[0031]

[0032] Where G is the number of groups of group convolution. For the Gth group, w g ∈R C×C represents the position-independent projection weight of the group, C = C / G represents the dimension; m gk ∈R represents the modulation scalar of the kth sampling point in the gth group; x gk ∈R C×H×W represents the input feature map of the slice; Δpgk is the sampling position p of the g-th group of grids k The corresponding offset.

[0033] Furthermore, in S2-2, the specific process includes:

[0034] The feature pyramid module in the neck network of the YOLOv8 model improves the detection performance of small target objects by fusing features of different scales. The 3×3 deformable convolution DCNv3 module is used to replace the fusion node in FPN and the standard convolution in the upsampling process, which increases the model's multi-scale adaptability to features and can more accurately capture and retain some lost shallow and boundary information when fusing high-level feature maps with low-level feature maps.

[0035] In the improved FPN structure, the input feature maps are set to R3, R4 and R5, and the output feature maps are F3, F4, F5, F6 and F7. The calculation formula of the output feature map is:

[0036]

[0037] Where R i Represents the feature map output of each layer of the network, Represents the feature map output after convolution, F i represents the feature map output of the feature pyramid module, Ti is the feature fusion processing function, which integrates features from different layers and produces an output feature map, and Ci is the 3×3 convolution operation applied to the feature map.

[0038] Furthermore, in S2-3, the specific process includes:

[0039] In the top detection network of the YOLOv8 model, four feature detector structures including large, medium, small and micro are designed. The multi-scale detector enables the model to effectively identify and locate targets of different scales, enhancing the applicability of the model for small-scale target detection.

[0040] Further, in S2-4, the CBAM module includes channel attention (CAM) and spatial attention (SAM), and the attention map is inferred along two independent dimensions of channel and spatial order, and the attention map is multiplied by the input feature map of adaptive feature refinement. The result of the convolution layer output is first focused on meaningful disease features through the channel attention (CAM), and the input image is transferred to the feature matrix by using the convolution layer. The number of channels of the obtained feature matrix is ​​the same as the number of cores, and the value of the number of cores is 1024 or 2048;

[0041] The CBAM module uses both maximum pooling and average pooling to adjust the weights of each channel in the feature map;

[0042] Spatial attention (SAM) is used to extract the location information of important objects. Average pooling and maximum pooling operations are used to calculate the spatial attention along the channel axis. The connection is made along the channel dimension, and the most important diseased parts in the network are focused on by spatial attention (SAM) for processing.

[0043] The calculation formula for processing the input feature map F in the CBAM module is as follows:

[0044]

[0045] Among them, F is the input feature map, Mc is the channel attention map, F' is the result of multiplying the feature map and the channel attention map, Ms is the spatial attention map, and F" is the result of the refined output;

[0046] Channel Attention (CAM) uses the inter-channel relationship of features to generate a channel attention map, compresses the feature map through global average pooling and maximum pooling operations to reduce data loss, and each channel is regarded as an independent feature detector response input;

[0047] Spatial Attention Array (SAM) generates a spatial attention map based on the spatial relationship of features, focusing on the location information of the features. It pays more attention to the specific location of the features in the image and can highlight the important feature areas in space.

[0048] Furthermore, in S2-5, the specific calculation formula of the loss function is:

[0049] Loss = a*L EIoU +b*DFL+c*VFL

[0050] a is XX, b is XX, C is XX;

[0051] The DFL loss function calculation formula is as follows:

[0052]

[0053] Where S i Represents the output of the Sigmod function, and y is the label value;

[0054] The VFL loss function is shown as follows:

[0055]

[0056] Where q is the predicted label and p is the true label;

[0057] The bounding box prediction loss uses EIoU, which takes into account the overlapping area, center point distance, aspect ratio, width and height size loss, convergence speed and accuracy of the predicted bounding box and the true bounding box, to improve the accuracy of bounding box positioning in the target detection task. The calculation formula is as follows:

[0058]

[0059] Among them, b and b gt Represents the bounding box and the ground-truth box, b and b gt represents the center point of the bounding box and the true box, ρ represents the Euclidean distance between the two center calculation points, c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box, w and h represent the length and width of the bounding box, w gt and h gt Represents the length and width of the real box, c w and c h is the minimum bounding box width and height.

[0060] Furthermore, in S3, the specific process includes:

[0061] A deep learning modeling environment based on PyTorch was built in the operating system. The settings included a learning rate of 0.01, a batch size of 64, and a training cycle (Epoch) of 300. For road disease detection, the model input image size was 640x640;

[0062] To adapt to the diversity and size differences of diseases, key detection parameters were adjusted: the confidence threshold was set to 0.4 to improve the recall rate, the NMS threshold was also set to 0.4 to reduce redundant detections, and the target box scaling value was set to 0.6 to adapt to the disease characteristics and ensure effective identification.

[0063] Furthermore, in S4, the specific process of constructing the performance evaluation index includes:

[0064] The main evaluation indicators introduced include recall, precision, average precision (mAP), F1-score, and frames per second (Fps) to evaluate the performance of the algorithm. Intersection over Union is introduced to evaluate the accuracy of the detection box. Four possible situations of the predicted sample are: predicting the true label as true, predicting the true label as false, predicting the false label as true, and predicting the false label as false.

[0065] The accuracy rates are:

[0066]

[0067] Among them, TP: is classified as a positive sample and the correct part;

[0068] FP: classified as positive samples and the wrong part;

[0069] The recall calculation formula is as follows:

[0070]

[0071] Among them, FN: is classified as a negative sample and the wrong part;

[0072] After the model training is completed, samples in the appropriate threshold classification data set are selected and the precision and recall are calculated. The PR curve is drawn by plotting the precision (Precision) and recall (Recall) under different thresholds to jointly evaluate the road disease detection performance of the model. The higher the value, the stronger the comprehensive detection ability of the model.

[0073] The mAP calculation formula is as follows:

[0074]

[0075] When the PR curves of multiple models intersect, it is difficult to judge the quality. The F1-score is calculated by the precision and recall rate to judge the quality of the model performance. The F1-score is the harmonic mean of the precision and recall rate. The F1-score of each category can be calculated by calculating the Precision and Recall under each category. The calculation method is as follows:

[0076]

[0077] Then calculate the average F1-score of each category to get the final result. The calculation method is as follows:

[0078]

[0079] Fps is the number of frames that the model can process in 1 second. The larger the Fps value, the shorter the average processing time, indicating that the model has a faster detection speed. It can be used to evaluate the real-time detection efficiency of the model. The calculation method is as follows:

[0080]

[0081] Where time is the average detection time of the model (s), frameNum represents the number of anchor frames for road crack detection, and elapsedtime is the time taken for target detection (s);

[0082] IoU is calculated as follows:

[0083]

[0084] Among them, area(D∩G) and area(D∪G) are the intersection and union of the predicted box and the true value box respectively;

[0085] The process of setting up an ablation experiment specifically includes:

[0086] The effectiveness of deformable convolution modules, CBAM attention mechanisms and small-scale detectors in improving the accuracy of target detection models is verified through ablation experiments. The experiments are divided into five groups, from the baseline model YOLOv8 to the complete network model integrating all the proposed modules. All experiments use the same hyperparameter configuration to compare Faster R-CNN, Cascade R-CNN, SSD, YOLOv8 and the trained models in S3 in terms of recall, precision, average accuracy (mAP), F1-score, frames per second (Fps) and intersection over union.

[0087] Compared with the prior art, the present invention has the following technical advantages:

[0088] 1) From the perspective of detection network structure, the application of DCNv3, CBAM module and the addition of small-scale detectors can improve the road detection accuracy, and the joint application of various strategies has no mutually exclusive effect. The network model optimized by the above three measures has improved the precision, recall and F1-score by 5.13%, 4.87% and 5.00%, and the loss curves of the training set and the validation set are consistent.

[0089] 2) From the perspective of disease categories, this algorithm model improves the detection accuracy of longitudinal cracks, transverse cracks, potholes and repair diseases, especially the detection accuracy of repair diseases, indicating that the model proposed in this invention has excellent detection capabilities when identifying multiple targets with large differences in pavement disease scales.

[0090] 3) From the perspective of data enhancement algorithms, the Mixup algorithm, CutMix algorithm and Mosaic algorithm can be effectively applied to road surface detection scenarios. The shadow weakening algorithm integrates the shadow weakened dataset into the road damage dataset, which improves the precision, recall and F1-score of the fitting model by 1.37%, 2.68% and 2.03% compared with the original dataset. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 This is the basic network structure diagram of YOLOv8;

[0092] Figure 2 is a standard data set diagram of the present invention;

[0093] Figure 3 It is a statistical diagram of pavement disease data of the present invention;

[0094] Figure 4 This is an example of the shadow weakening processing performed on a data set according to the present invention;

[0095] Figure 5 The dataset constructed for the present invention consists of:

[0096] Figure 6 This is the MixUp and CutMix data enhancement result diagram of the present invention;

[0097] Figure 7 This is the Mosaic data enhancement result diagram of the present invention;

[0098] Figure 8 This is a structural diagram of the variable convolution module introduced in the present invention;

[0099] Fig. 9 The fusion attention mechanism introduced by the present invention;

[0100] Fig.10 To improve the network structure diagram of the road damage target detection algorithm;

[0101] Fig.11 It is an iteration diagram of the algorithm in Example 1 and the loss functions and evaluation indicators of different models;

[0102] Fig.12 It is a Loss training iteration diagram of the algorithm in Example 1 and different models;

[0103] Fig.13 It is a comparison chart of PR curve and Fps results of the algorithm in Example 1 and different models;

[0104] Fig.14 This is a graph comparing the performance indicators of the algorithm in Example 1 with different models;

[0105] Fig.15 This is a comparison chart of the evaluation indicators mAP and F1-score obtained by the algorithm in Example 1;

[0106] Fig.16 This is a diagram of the disease type detection results obtained by the algorithm in Example 1. DETAILED DESCRIPTION

[0107] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The features such as the structural module name, control mode, algorithm, process or composition ratio not clearly described in this technical solution are regarded as common technical features disclosed in the prior art.

[0108] Example 1

[0109] This embodiment proposes a road damage detection optimization method based on scale difference, including the following steps:

[0110] S1. Constructing a pavement damage image dataset

[0111] The road disease dataset of this embodiment is derived from its own dataset. The acquired asphalt pavement disease data containing typical pavement diseases are annotated and cleaned to obtain a pavement disease image dataset containing different resolutions. The dataset is preprocessed and enhanced and divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0112] S2. Improved road damage target detection algorithm

[0113] An improved road disease target detection algorithm is proposed based on the YOLOv8 basic network architecture. The network structure of the C2f module is optimized in the model backbone feature extraction network. The deformable convolution module DCNv3 module replaces the original convolution module to increase the effective prediction area. The 1×1 convolution of the FPN module is replaced by a 3×3 deformable convolution DCNv3 module to increase the model's multi-scale adaptability to features. A multi-scale detector is set in the top detection network and the CBAM module is introduced to optimize the model's target feature extraction process in complex interference factor scenarios. At the same time, the loss function is improved.

[0114] S3: Deploy the model test environment to train and verify the model

[0115] Deploy the model test environment and adjust the model parameters to adapt to the diversity of road damage in images and their size differences. Input the preprocessed images into the constructed network model for training, obtain the model training weights, and finally conduct testing and verification.

[0116] S4. Introduce algorithm model performance evaluation indicators, qualitatively analyze the model's disease detection effect, and perform road disease detection using the evaluated model.

[0117] Relevant algorithm model performance evaluation indicators are introduced, and ablation experiments are set up to verify the effectiveness of deformable convolution modules, CBAM attention mechanisms, and small-scale detectors on the accuracy of target detection models and the optimization of model accuracy. The performance of the algorithm proposed in this invention is compared with different algorithms, and qualitative analysis of the detection effects of different types of diseases is performed.

[0118] In specific implementation, step S1 is implemented as follows:

[0119] (1) Asphalt pavements with typical pavement defects were selected from highway sections of different levels in multiple cities for data acquisition. Videos with a resolution of 1920×1080 and RGB images with 30 frames per second were obtained using forward-looking vehicle-mounted equipment.

[0120] (2) In order to ensure the accuracy of the model detection results and reduce problems such as poor labeling quality, sample imbalance, and noise interference, the constructed road damage dataset was systematically cleaned. First, the labeling of each sample was carefully reviewed to ensure that different types of road damage were clearly and accurately labeled. In addition, images containing occluded or blurred features were excluded to ensure that the data used could support the stability and effectiveness of the training model. The open source software DarkLabel was used to label the acquired video images. The road damage categories that needed to be labeled included six categories: transverse cracks, longitudinal cracks, mesh cracks, potholes, block repairs, and strip repairs. The labeled dataset is as follows: Figure 2 During the training process, different weights are assigned to samples of different categories, with higher weights given to samples of minority categories and lower weights given to samples of majority categories, thereby improving the model's ability to detect all types of road damage.

[0121] (3) After data cleaning, the labeled dataset contains 64,476 pavement disease images of different resolutions. 58,028 images were randomly selected as the training set (including 13,309 transverse cracks, 13,190 longitudinal cracks, 2,783 reticular cracks, 2,045 potholes, 14,025 block repairs, and 12,676 strip repairs), and 6,448 images were used as the test set (including 1,479 transverse cracks, 1,465 longitudinal cracks, 309 reticular cracks, 228 potholes, 1,558 block repairs, and 1,409 strip repairs). The 64,476 dataset images were compressed with a uniform resolution of 768×1024. The number of disease images in each category of the training set and the distribution of pavement disease pixel ratios are shown below. Figure 3 As shown, significant differences in the sizes of road damage in the dataset can be seen.

[0122] (4) To reduce the shadow interference in the image, the shadow weakening process is performed on the diseased image. Figure 4 This is an example of a shadow-weakened image. While maintaining the integrity of other key information in the image to the maximum extent, the shadow-weakened dataset and the shadow image in the original dataset are integrated into the road disease dataset in a 1:1 ratio. This setting increases the size of the dataset and optimizes the generalization performance of the model. On the other hand, it reduces false positive errors by allowing the model to learn the performance of disease features under different lighting conditions (with shadows and with weakened shadows). Figure 5 The composition of the data set has been increased.

[0123] (5) Use a variety of data enhancement techniques, such as flipping, scaling, blurring, color adjustment, and brightness and contrast changes, to simulate scenes under different lighting conditions. Combine advanced data enhancement algorithms such as CutMix, Mixup, and Mosaic to fuse image samples and enhance the model's ability to identify occluded objects and comprehensively understand image features, such as Figure 6 , 7 All images were uniformly adjusted to a resolution of 640 × 640 pixels, and the pixel values ​​were normalized to the range of 0 to 1 using the maximum normalization method to reduce numerical instability during model training.

[0124] In specific implementation, step S2 is implemented as follows:

[0125] (1) In the model backbone feature extraction network, the network structure of the C2f module is optimized, and the deformable convolution module DCNv3 module is used to replace the original convolution module to increase the effective prediction area. The CSPDarknet53 structure improved by the C2f module is used as the backbone feature extraction network of the model to enhance the efficiency of feature transfer and fusion. Figure 8 , is to add an offset to the regular grid sampling position in the standard convolution, so that the sampling grid can be deformed freely, and the function of the standard convolution layer is enhanced by introducing a learnable offset from the feature map of the previous layer through an additional convolution layer. This offset allows the network to dynamically adjust its receptive field when performing convolution operations, so that the network can extract features more effectively. The variable convolution structure is an improved network structure of the regular convolution. When a given input x∈R C×H×W , the current pixel is p0, which can be expressed as

[0126]

[0127] Where K is the total number of sampling points and k is the number of sampling points. k ∈R C×C represents the projection weight of the kth sampling point, m k ∈R represents the modulation scalar of the kth sampling point, which is used to adjust the weight of the sampling point after normalization by the sigmoid function. k Represented as the k-th position (-1, 1), (-1, 0), ..., (0, +1), ..., (+1, +1) sampled in the predefined grid in the regular convolution, Δp k is the offset corresponding to the kth grid sampling position. Offset Δp k Can interact with short-range or long-range features, sampling offset Δp k and the modulation scalar m k are all constrained by x.

[0128] The variable convolution adopts the strategy of separable convolution and decomposes the convolution weights into depth direction and point direction. The depth direction part is responsible for adjusting the modulation scalar m of the original position perception. k, controlling different channels of the feature map; the point direction part uses the projection weight w shared between the sampling points to reduce the number of model parameters. In addition, the introduction of group convolution divides the spatial aggregation process into G groups, each with an independent sampling offset Δp gk and modulation scale m gk , so that the model can learn richer information in different representation subspaces, and each group can show different spatial aggregation patterns. The original modulation scalar is normalized by the element-by-element Sigmoid function. Although it can ensure that the modulation scalar is in the range of [0,1], the gradient is unstable during large-scale parameter and data training. By adopting the Softmax normalization method along the sample points, the sum of all modulation scalars is constrained to 1, which improves the stability of the model under different scale training, so that the deformable convolution layer can maintain a more stable gradient when processing large-scale data and parameters, thereby improving the training efficiency and performance of the model. The DCNv3 replaced by the algorithm proposed in the present invention in the C2f module can be expressed as:

[0129]

[0130] Where G is the number of groups in the group convolution. For the Gth group, w g ∈R C×C represents the position-independent projection weight of the group, C = C / G represents the dimension; m gk ∈R represents the modulation scalar of the kth sampling point in the gth group; x gk ∈R C×H×W represents the input feature map of the slice; Δp gk is the sampling position p of the g-th group of grids k The corresponding offset.

[0131] (2) Use the 3×3 deformable convolution module DCNv3 to replace the fusion nodes in FPN and the standard convolution in the upsampling process, increase the model's multi-scale adaptability to features, and more accurately capture and retain some lost shallow and boundary information when fusing high-level feature maps with low-level feature maps, so that the target detection algorithm can extract richer and more accurate boundary information for the target, increase the accuracy of target classification and positioning, and enable the network to better aggregate image features of different scales and improve the detection accuracy of road disease targets with large scale differences. In actual road detection applications, it can focus more on the road surface status and reduce misidentification of the surrounding environment. In the improved FPN structure, the input feature maps are set to R3, R4 and R5, and the output feature maps are F3, F4, F5, F6 and F7. The calculation formula for the output feature map is:

[0132]

[0133] Among them, Ri represents the feature map output of each layer of the network, represents the feature map output after convolution, Fi represents the feature map output of the feature pyramid module, Ti is the feature fusion processing function, which integrates features from different layers and generates an output feature map, and Ci is the 3×3 convolution operation applied to the feature map.

[0134] (3) For the top detection network, the network is specially designed with four feature detector structures including large, medium, small and micro. The multi-scale detector enables the model to effectively identify and locate objects of different scales, enhancing the applicability of the model to small-scale object detection.

[0135] (4) The CBAM module is introduced into the top-level detection network. Its channel and spatial attention mechanisms strengthen the network’s representation of key features. The gradually refined attention mechanism not only improves the detection performance of the model, but is also more effective for the fusion of multi-scale features, especially in the detection of multiple targets with large size differences, providing richer and more accurate feature representation. The attention mechanism focuses on local features in the target feature map, strengthens feature information and suppresses useless information after weight allocation. Fig. 9 , the CBAM module includes channel attention (CAM) and spatial attention (SAM), which infer attention maps along two independent dimensions: channel and spatial order, and multiply the attention map by the input feature map for adaptive feature refinement. The output of the convolutional layer is first focused on meaningful disease features through CAM, and the input image can be transferred to the feature matrix by using the convolutional layer. The number of channels of the resulting feature matrix is ​​the same as the number of cores, and the value of the number of cores is usually 1024 or 2048. In order to solve the problem that many channel dimensions contain a lot of redundancy, the channel attention model applies a filter to selectively retain only the information most relevant to the desired result, thereby enhancing the attention to the target. CBAM also uses maximum pooling and average pooling to more appropriately adjust the weights of each channel in the feature map. The entire process after the SAM spatial attention module is completed is used to extract the positioning information of important objects. Both average pooling and maximum pooling operations are used to calculate the spatial attention along the channel axis and connect along the channel dimension. The SAM module focuses on the most important disease parts in the network for processing. The calculation formula for processing the input feature map F in CBAM is as follows:

[0136]

[0137] Among them, F is the input feature map, Mc is the channel attention map, F' is the result of multiplying the feature map and the channel attention map, Ms is the spatial attention map, and F" is the result of the refined output.

[0138] (5) The losses in the training process mainly include the following three types: target category loss (losso), confidence threshold loss (lossi) and bounding box prediction regression loss (lossc). A road disease hybrid loss function suitable for multi-scale target detection of road diseases is selected to increase the specificity of the improved network for the multi-scale road disease detection task and the road disease detection performance. The VFL loss function is introduced as the target category loss, the EIOU loss function is used as the confidence threshold loss, and the DFL loss function is used as the bounding box prediction regression loss, which are defined as:

[0139] Loss = λ1*loss o +λ2*loss c +λ3*loss i

[0140] Among them, λ1, λ2, and λ3 represent the weighted values ​​of the corresponding losses respectively.

[0141] In specific implementation, step S3 is implemented as follows:

[0142] (1) Environment configuration and hardware preparation: Ubuntu 20.04 operating system, Intel(R) Core(TM) i7-12800H CPU@GHz, NVIDIA GeForce GTX3060 graphics card, CUDA version 11.3, CUDNN version 8.1, and the deep learning framework based on Pytorch 1.3.1.

[0143] (2) Initial condition setting of the test environment: the learning rate is set to 0.01, the batch size is 64, the number of epochs in the experiment is defined as 300, and the initialization parameters are shown in Table 1.

[0144] Table 1 Test environment initial condition settings

[0145]

[0146]

[0147] (3) The input image size of the road damage detection model is uniformly set to 640x640. In order to adapt to the diversity of road damage in images and their size differences, specific adjustments are made to the key detection parameters of the model, including the confidence threshold, non-maximum suppression (NMS) threshold, and target box scaling value, while other parameters remain at the default settings. The confidence threshold is set to 0.4 to improve the recall rate of the model and reduce the possibility of missed detection. The threshold of non-maximum suppression (NMS) is also set to 0.4. Based on the fact that road damage targets usually do not overlap too much in multi-lane scenarios, such a setting helps to reduce redundant detection while maintaining target detection accuracy. The target box scaling value is set to 0.6. A smaller scaling value can better adapt to the characteristics of road damage and ensure that the model can effectively identify various types of damage.

[0148] (4) Input the preprocessed image into the constructed network model for training, obtain the model training weights, and finally perform testing and verification.

[0149] In specific implementation, step S4 is implemented as follows:

[0150] (1) Introduce algorithm model performance evaluation indicators to qualitatively analyze the model's disease detection effect. The main evaluation indicators include recall rate (Recall), precision rate (Precision), average precision rate (mAP, AveragePrecision), F1 score (F1-score) and frame per second (Frame Per Second, Fps) to evaluate the performance of the algorithm, and introduce intersection over union (IoU) to evaluate the accuracy of the detection box.

[0151] The accuracy P (Precision) indicates the proportion of correct predictions in a certain category of prediction targets to the total correct samples, and the recall R (Recall) indicates the proportion of correct predictions in the prediction targets to the total prediction samples. The definition formulas are:

[0152]

[0153] Among them, TP: is classified as a positive sample and the correct part;

[0154] FP: is classified as a positive sample and the wrong part.

[0155] The recall calculation formula is as follows:

[0156]

[0157] Among them, FN: is divided into negative samples and the wrong part.

[0158] After the model training is completed, it is necessary to select samples from the appropriate threshold classification data set to calculate the precision and recall. The PR curve is drawn by plotting the precision (Precision) and recall (Recall) under different thresholds to jointly evaluate the road disease detection performance of the model. The higher the value, the stronger the comprehensive detection ability of the model.

[0159] mAP is measured by the area under the PR curve and can be used to evaluate the performance of the model. If the PR curve of model A can completely contain the PR curve of model B, it means that the performance of model A is better than that of model B. mAP is the arithmetic mean of the average precision of all categories in the test set. It is calculated by taking the average of the average precision (AP) of each class in N categories. The larger the mAP value, the higher the detection accuracy of the model. According to the formula and confidence threshold, a series of precision and recall values ​​are calculated for each category. By drawing the PR curve, we can get mAP, which is the area under the PR curve. The size of the area can be used to evaluate the performance of the model. The calculation formula is as follows:

[0160]

[0161] When the PR curves of multiple models intersect, it is difficult to judge the quality. The F1-score is calculated by the precision and recall rate to judge the quality of the model performance. The F1-score is the harmonic mean of the precision and recall rate. The F1-score of each category can be calculated by calculating the Precision and Recall under each category. The calculation method is as follows:

[0162]

[0163] Then calculate the average F1-score of each category to get the final result. The calculation method is as follows:

[0164]

[0165] Fps is the number of frames that the model can process in 1 second. The larger the Fps value, the shorter the average processing time, indicating that the model has a faster detection speed. It can be used to evaluate the real-time detection efficiency of the model. The calculation method is as follows:

[0166]

[0167] Among them, time is the average detection time of the model (s), frameNum represents the number of anchor frames for road crack detection, and elapsedtime is the time taken for target detection (s).

[0168] In order to evaluate the performance of the target detection algorithm, the Intersection over Union (IoU) is often used to determine whether a target is successfully detected by the algorithm. Generally speaking, the larger the IoU value, the more accurate the target box prediction, and vice versa, the larger the prediction error. The IoU is calculated as follows:

[0169]

[0170] Among them, area(D∩G) and area(D∪G) are the intersection and union of the predicted box and the true value box, respectively.

[0171] (2) An ablation experiment is set up to verify the effectiveness of the deformable convolution module, CBAM attention mechanism, and small-scale detector on the accuracy of the target detection model and the optimization of the model accuracy.

[0172] The ablation experiments are divided into five groups: the first group uses YOLOv8 as the baseline model, the next three groups integrate deformable convolution modules, CBAM attention mechanism, and small-scale detectors respectively. The last group of experiments integrates all the proposed modules as a complete network model. All experiments use the same hyperparameter configuration during training. The hyperparameters used in the five groups of experiments are consistent during training.

[0173] (3) To verify the effectiveness of the network structure proposed in the present invention, the network is compared with the widely used two-stage target detection algorithms Faster R-CNN and Cascade R-CNN, the single-stage target detection algorithm SSD and the YOLOv8 benchmark model. The experimental results on the self-owned road disease dataset are shown in the table. In order to better evaluate the effectiveness of the model proposed in the present invention, five target detection algorithms, Faster R-CNN, Cascade R-CNN, SSD, YOLOv8 and the algorithm of the present invention are compared in terms of P, R, mAP, F1-score, Fps, IOU, etc.

[0174] (4) In order to qualitatively evaluate the detection effect of the optimized model on specific road disease types, the effectiveness of the optimized model is qualitatively evaluated based on the evaluation indicators and detection effects, starting from the detection effect of each disease type image.

[0175] Application Example 1

[0176] In this application example, a road damage detection optimization method based on scale difference includes the following steps:

[0177] (1) Construct a self-owned road dataset and divide the dataset into training set, validation set and test set according to the ratio of 8:1:1. The implementation is as follows:

[0178] Firstly, the asphalt pavement with typical pavement diseases was photographed using the vehicle-mounted equipment to obtain a video with a resolution of 1920×1080 and an RGB image with 30 frames per second.

[0179] Then, the constructed road damage dataset was systematically cleaned. First, the annotations of each sample were carefully reviewed to ensure that different types of road damage were clearly and accurately marked. In addition, images containing occluded or blurred features were excluded to ensure that the data used could support the stability and effectiveness of the training model. The open source software DarkLabel was used to mark the acquired video images for damage. The road damage categories that needed to be marked included six categories: transverse cracks, longitudinal cracks, mesh cracks, potholes, block repairs, and strip repairs. Different weights were assigned to samples of different categories during the training process, with higher weights given to samples of minority categories and lower weights given to samples of majority categories, thereby improving the model's ability to detect all types of road damage.

[0180] After data cleaning, the labeled dataset contains 64,476 pavement disease images of different resolutions. 58,028 images were randomly selected as the training set (including 13,309 transverse cracks, 13,190 longitudinal cracks, 2,783 reticular cracks, 2,045 potholes, 14,025 block repairs, and 12,676 strip repairs), and 6,448 images were randomly selected as the test set (including 1,479 transverse cracks, 1,465 longitudinal cracks, 309 reticular cracks, 228 potholes, 1,558 block repairs, and 1,409 strip repairs).

[0181] In order to reduce the shadow interference in the image, the shadow of the diseased image is weakened, while the integrity of other key information in the image is maintained to the maximum extent. The shadow-weakened dataset and the shadow image in the original dataset are integrated into the road disease dataset in a 1:1 ratio. This setting, on the one hand, increases the scale of the dataset and optimizes the generalization performance of the model. On the other hand, by allowing the model to learn the performance of disease features under different lighting (with shadows and weakened shadows), false positive errors can be reduced.

[0182] A variety of data enhancement techniques are used, such as flipping, scaling, blurring, color adjustment, and brightness contrast changes, to simulate scenes under different lighting conditions. Advanced data enhancement algorithms such as CutMix, Mixup, and Mosaic are combined to fuse image samples to enhance the model's ability to identify occluded objects and understand image features comprehensively. All images are uniformly adjusted to a resolution of 640×640 pixels, and the pixel values ​​are normalized to a range of 0 to 1 through the maximum normalization method to reduce numerical instability during model training. The data set is divided into training, validation, and test sets in a ratio of 8:1:1.

[0183] (2) An improved road damage target detection algorithm is proposed based on the YOLOv8 basic network architecture, and its implementation is as follows:

[0184] First, the network structure of the C2f module in the model backbone feature extraction network is optimized, and the deformable convolution module DCNv3 module replaces the original convolution module to increase the effective prediction area. The CSPDarknet53 structure improved by the C2f module is used as the backbone feature extraction network of the model to enhance the efficiency of feature transmission and fusion.

[0185] Secondly, the 3×3 deformable convolution module DCNv3 is used to replace the fusion nodes in FPN and the standard convolution in the upsampling process, which increases the model's multi-scale adaptability to features. When fusing high-level feature maps with low-level feature maps, it can more accurately capture and retain some lost shallow and boundary information, so that the target detection algorithm can extract richer and more accurate boundary information for the target, increase the accuracy of target classification and positioning, and enable the network to better aggregate image features of different scales and improve the detection accuracy of road disease targets with large scale differences. In actual road detection applications, it can focus more on the road surface status and reduce misidentification of the surrounding environment.

[0186] For the top detection network, the network is specially designed with four feature detector structures including large, medium, small and micro. The multi-scale detector enables the model to effectively identify and locate targets of different scales, enhancing the applicability of the model to small-scale target detection. The CBAM module is introduced in the top-level detection network, and the network's representation of key features is strengthened through its channel and spatial attention mechanism. The gradually refined attention mechanism not only improves the detection performance of the model, but is also more effective for the fusion of multi-scale features, especially in the detection of multiple targets with large size differences, providing richer and more accurate feature representation. The attention mechanism focuses on local features in the target feature map, strengthens feature information and suppresses useless information after weight allocation.

[0187] Select a road disease hybrid loss function suitable for multi-scale road disease target detection to increase the specificity of the improved network for multi-scale road disease detection tasks and road disease detection performance. The losses in the training process mainly include the following three types: target category loss (losso), confidence threshold loss (lossi) and bounding box prediction regression loss (lossc). The VFL loss function is introduced as the target category loss, the EIOU loss function is used as the confidence threshold loss, and the DFL loss function is used as the bounding box prediction regression loss, which are defined as:

[0188] Loss = λ1*loss o +λ2*loss c +λ3*loss i

[0189] Among them, λ1, λ2, and λ3 represent the weighted values ​​of the corresponding losses respectively.

[0190] (3) Deploy the model and build the modeling environment on the Ubuntu 20.04 operating system, set the initial conditions of the test environment, and perform verification tests. The implementation is as follows:

[0191] The model testing environment proposed in this invention is Intel(R) Core(TM) i7-12800H CPU@GHz, NVIDIAGeForce GTX3060 graphics card, CUDA version 11.3, CUDNN version 8.1, and the deep learning framework based on Pytorch1.3.1. The learning rate is set to 0.01, the batch size is 64, the number of epochs in the experiment is defined as 300, and the initialization parameters are set as shown in Table 1.

[0192] The input image size of the road damage detection model is uniformly set to 640x640. In order to adapt to the diversity of road damage in images and their size differences, specific adjustments are made to the key detection parameters of the model, including the confidence threshold, non-maximum suppression (NMS) threshold, and target box scaling value, while other parameters remain at the default settings. The confidence threshold is set to 0.4 to improve the recall rate of the model to reduce the possibility of missed detection. The threshold of non-maximum suppression (NMS) is also set to 0.4. Based on the fact that road damage targets usually do not overlap too much in multi-lane scenarios, such a setting helps to reduce redundant detection while maintaining target detection accuracy. The target box scaling value is set to 0.6. A smaller scaling value can better adapt to the characteristics of road damage and ensure that the model can effectively identify various types of damage.

[0193] (4) Introduce relevant algorithm model performance evaluation indicators, set up ablation experiments to verify the effectiveness of deformable convolution module, CBAM attention mechanism, and small-scale detector on the accuracy of target detection model and the optimization of model accuracy; compare the performance of the algorithm proposed in this invention with different algorithms, and conduct qualitative analysis on the detection effect of different types of diseases, which is implemented as follows:

[0194] In order to evaluate the performance of the algorithm, the main evaluation indicators include recall rate (Recall), precision rate (Precision), average precision rate (mAP, Average Precision), F1 score (F1-score) and frame rate per second (Frame Per Second, Fps) to evaluate the performance of the algorithm, and the intersection over union (Intersection over Union) is introduced to evaluate the accuracy of the detection box.

[0195] Ablation experiments are set up to verify the effectiveness of the deformable convolution module, CBAM attention mechanism, and small-scale detector on the accuracy of the target detection model and the optimization of the model accuracy. The ablation experiments are divided into five groups: the first group uses YOLOv8 as the baseline model, and the next three groups integrate the deformable convolution module, CBAM attention mechanism, and small-scale detector respectively. The last group of experiments integrates all the proposed modules as a complete network model. All experiments use the same hyperparameter configuration during training. The hyperparameters used in the five groups of experiments during training are consistent. The iterative trends of P and R during training are shown in Figure 2. Fig.11 As shown in the figure, according to the iterative trend chart during the training process, it can be observed that the three improvement ideas all have different degrees of improvement on the YOLOv8 model in the complex road interference scene, and the deformable convolution module has the highest improvement on the model. At the same time, the comprehensive improvement of the model algorithm proposed by the present invention has achieved a good fitting effect. In addition, as the loss curve gradually fits during the training, the loss function values ​​of the deformable convolution module, the CBAM attention mechanism, and the small-scale detector are all lower than those of YOLOv8, indicating that the model has a better degree of data fitting and has improved accuracy in locating road diseases. The model of this embodiment that applies the three measures has achieved the lowest and best loss function value, as shown in Figure 1. Fig.12 In order to explore the performance differences of different models, a model indicator verification experiment was conducted on the test set, and the verification results are shown in Table 2 below:

[0196] Table 2 Performance index comparison test results

[0197]

[0198] Note: √ means the improvement point is added in the network, - means the improvement point is not added in the network

[0199] According to the data analysis in the table, the deformable convolution module improves the model detection accuracy by 4.67%. After using 3×3 deformable convolution instead of 1×1 convolution, the model can understand more boundary information of small targets and effectively integrate shallow information of objects into high-level feature maps. Deformable convolution dynamically adjusts the convolution grid according to the target contour, which helps to understand the complex detail feature information of various crack diseases. At the same time, the network structure modification of the feature pyramid can effectively solve the problem of reduced target accuracy caused by insufficient extraction of target boundary information in the feature pyramid, so that the target detection algorithm can extract richer and more accurate boundary information for the target, thereby improving the detection accuracy of the target. The CBAM attention mechanism module can improve the accuracy by 3.85%, which may be because CBAM adaptively strengthens the network's representation of key features through its channel and spatial attention mechanisms, enhancing the ability to extract road disease features.

[0200] The model proposed in this invention has a high F1-score, which reflects the effectiveness of the improved network structure in the task of detecting road damage targets. The original YOLOv8 model may lose some target boundary information during the feature mapping and fusion process, while the model in this chapter expands the information receiving range by applying 3×3 deformable convolution, thereby obtaining more target boundary information, resulting in overall performance that is better than the baseline YOLOv8 model in terms of accuracy (P), recall (R), and F1-score. In addition, the network structure proposed in this invention and the original YOLOv8 model have little difference in the number of frames processed per second (fps) when detecting the same test video, indicating that the network model does not significantly increase the computational complexity of the network while improving the detection accuracy, and maintains a real-time processing capability similar to the original model, proving the effectiveness of the model optimization strategy. The PR curve and Fps results are compared as shown in the figure. Fig.13 shown.

[0201] In order to verify the effectiveness of the network structure proposed in the present invention, the network is compared with the widely used two-stage target detection algorithms Faster R-CNN and Cascade R-CNN, the single-stage target detection algorithm SSD and the YOLOv8 benchmark model, and the experimental results on the self-owned road disease dataset are shown in the table. In order to better evaluate the effectiveness of the model proposed in the present invention, five target detection algorithms, Faster R-CNN, Cascade R-CNN, SSD, YOLOv8 and the algorithm of this embodiment are compared in terms of P, R, mAP, F1-score, Fps, IOU, etc. The comparison results are shown in Table 3.

[0202] Table 3 Comparison of detection algorithm performance indicators

[0203]

[0204] From the data in Table 3, it can be seen that the network model proposed in this invention has a great advantage in terms of average detection accuracy index. Compared with Faster R-CNN, Cascade R-CNN, SSD and YOLOv8 target detection algorithms, the accuracy rates are 5.13%, 5.55%, 6.90% and 2.24% higher respectively. Among them, AP 50 The accuracy rates of the proposed algorithm are 5.97%, 7.92%, 10.16% and 3.67% higher respectively, indicating that the proposed algorithm can effectively solve the problem of reduced target accuracy caused by insufficient extraction of target boundary information and shallow information in the feature pyramid, extract richer and more accurate boundary information, strengthen the network's representation of key features, and enhance the ability to extract road damage features.

[0205] The accuracy of the algorithm proposed in this invention is good (0.921), indicating that the correct proportion of its predicted positive samples is high; while Faster R-CNN has the lowest accuracy, indicating that it predicts more false positive results. It also performs best in F1-score (0.928), which means that it has a good balance between precision and recall. At the same time, the IOU value is not much different from that of YOLOv8, both close to 0.83, indicating that these two models have strong target positioning capabilities.

[0206] The performance indicators of different models are compared as follows: Fig.14 As shown in the figure, compared with Faster R-CNN, Cascade R-CNN, SSD and YOLOv8 algorithms, the algorithm proposed in this invention has better performance in P, R and AP. 50 ,AP 75 , F1-score, and IOU performance are all relatively advantageous. The DCNv3 module and CBAM in the YOLOv8 model network structure can achieve significant improvements in performance evaluation indicators. The algorithm of the present invention is superior to other target detection algorithms in terms of comprehensive performance. It not only has higher precision and recall rate, but also can locate targets more accurately.

[0207] There are various types of road diseases and their forms vary. The network model shows different detection accuracy according to the different characteristics of the disease. In order to qualitatively evaluate the detection effect of the optimized model on specific road disease types, the effectiveness of the optimized model is qualitatively evaluated based on the evaluation indicators and detection effect, starting from the detection effect of each disease type image. The comparison of the evaluation indicators mAP and F1-score of each disease type on the test set between the optimized model of this embodiment and the YOLOv8 model is shown in Figure 2. Fig.15As shown in the figure, by comparing the evaluation indicators of the model of this embodiment and YOLOv8 on the test set, the recall rate and precision rate of each disease type of the algorithm of this embodiment have been improved. The mAP of transverse cracks, longitudinal cracks, mesh cracks, potholes and repair diseases are all improved by 4.9%, 1.7%, 2.6%, 0.1%, 3.0% and 1.5% compared with the original model, and the F1-score is improved by 5.04%, 1.84%, 2.13%, 0.53%, 3.09% and 2.32%, respectively, which is significantly improved for small-scale targets. Longitudinal and transverse cracks maintain a high recall rate and precision rate in both the original model and the model of this embodiment, while the recall rate and precision rate of potholes in the model are lower than the average level of disease detection, and they are difficult to identify samples. The detection accuracy of mesh cracks and potholes on different models is lower than the average detection accuracy of the model, indicating that mesh cracks and potholes are difficult to identify samples in the road surface detection task. Considering the pavement disease data set, the number of data samples for potholes is only 15.5% of that for longitudinal cracks. The disadvantage of labeled data volume is also the reason why the accuracy of mesh cracks is not high enough. In summary, the optimized model has a high recognition accuracy for pavement diseases and improves the recognition ability for smaller-sized diseases. The specific disease type detection results are shown in the figure below: Fig.16 shown.

[0208] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A road damage detection method based on scale difference, characterized in that: The following steps are involved: S1: Acquire asphalt pavement image data containing typical pavement diseases, perform data cleaning and disease marking on the acquired road disease data, obtain a pavement disease image data set containing different resolutions, and perform data preprocessing and enhancement; S2: Based on the YOLOv8 basic network architecture, an improved road disease target detection algorithm is constructed, including: optimizing the network structure of the C2f module in the model backbone feature extraction network, replacing the original convolution module with the deformable convolution module DCNv3 module, increasing the effective prediction area, replacing the 1×1 convolution of the FPN module with a 3×3 deformable convolution DCNv3 module, increasing the model's multi-scale adaptability to features, setting a multi-scale detector in the top detection network of the YOLOv8 basic network architecture and introducing the CBAM module to optimize the model's target feature extraction process in scenes with complex interference factors; S3: Deploy the model test environment and adjust the model parameters to adapt to the diversity of road diseases in images and their size differences. Input the preprocessed images into the constructed network model for training, obtain the model training weights, and finally conduct testing and verification. S4: Construct performance evaluation indicators and set up ablation experiments to verify the effectiveness of the deformable convolution module, CBAM attention mechanism, and small-scale detector on the accuracy of the target detection model and the optimization of the model accuracy. Use the evaluated model to detect road damage.

2. The road damage detection method based on scale difference according to claim 1 is characterized in that: In S1, the specific process includes: S1-1: Acquire asphalt pavement images on multiple urban highways, wherein the asphalt pavement images contain typical pavement diseases, perform data cleaning on the images, exclude occluded or blurred images, ensure image data quality, and use DarkLabel software to accurately mark the diseases; S1-2: Preprocess and enhance the labeled image dataset to improve the model's ability to identify diseases under different conditions, unify the image resolution and perform normalization processing, and prepare a high-quality dataset for model training; S1-3: Divide the normalized image data into training set, validation set, and test set.

3. The road damage detection method based on scale difference according to claim 1, characterized in that: In S2, the specific process includes: S2-1: Optimize the network structure of the C2f module in the backbone feature extraction network of the YOLOv8 model, replace the original convolution module with the deformable convolution module DCNv3 module, increase the effective prediction area, and use the improved CSPDarknet53 structure of the C2f module as the backbone feature extraction network of the model to enhance the transmission and fusion efficiency of features; S2-2: Replace the 1×1 convolution of the FPN module in the YOLOv8 model with a 3×3 deformable convolution DCNv3 module; S2-3: setting four feature detector structures including large, medium, small and micro in the top detection network of the YOLOv8 model; S2-4: A CBAM module is introduced into the top detection network of the YOLOv8 model to judge the importance of different channel features through the channel attention mechanism, and then the key features in the target area are further refined based on the spatial attention mechanism; S2-5: The target category loss (losso), confidence threshold loss (lossi) and bounding box prediction regression loss (lossc) are used as loss functions, and the VFL loss function is introduced as the target category loss, the EIOU loss function as the confidence threshold loss, and the DFL loss function as the bounding box prediction regression loss.

4. The road damage detection method based on scale difference according to claim 3 is characterized in that: In S2-1, the specific process includes: In the backbone feature extraction network of the YOLOv8 model, the network structure of the C2f module is optimized, and the deformable convolution module DCNv3 module replaces the original convolution module to increase the effective prediction area. The CSPDarknet53 structure improved by the C2f module is used as the backbone feature extraction network of the model to enhance the efficiency of feature transmission and fusion; The deformable convolution structure is an improved network structure of conventional regular convolution. When the input x∈R C×H×W , currently pixel p0, is expressed as: In the formula, K is the total number of sampling points, k is the number of sampling points, and w k ∈R C×C represents the projection weight of the kth sampling point, m k ∈R represents the modulation scalar of the kth sampling point. After normalization by the sigmoid function, it is used to adjust the weight of the sampling point. k Represented as the k-th position (-1, 1), (-1, 0), ..., (0, +1), ..., (+1, +1) sampled in the predefined grid in the regular convolution, Δp k is the offset corresponding to the kth grid sampling position, the offset Δp k Can interact with short-range or long-range features, sampling offset Δp k and the modulation scalar m k are all constrained by x; The deformable convolution decomposes the convolution weights into depth direction and point direction. The depth direction part is responsible for adjusting the modulation scalar m of the original position perception. k , control different channels of the feature map, and the point direction part uses the projection weight w shared between the sampling points to reduce the number of model parameters. The group convolution is introduced to divide the spatial aggregation process into G groups, each with an independent sampling offset Δp gk and modulation scale m gk , which enables the model to learn richer information in different representation subspaces, and each group exhibits different spatial aggregation patterns; The DCNv3 replaced in the C2f module can be expressed as: Where G is the number of groups of group convolution. For the Gth group, w g ∈R C×C represents the position-independent projection weight of the group, C = C / G represents the dimension; m gk ∈R represents the modulation scalar of the kth sampling point in the gth group; x gk ∈R C×H×W represents the input feature map of the slice; Δp gk is the sampling position p of the g-th group of grids k The corresponding offset.

5. The road damage detection method based on scale difference according to claim 3 is characterized in that: In S2-2, the specific process includes: The feature pyramid module in the neck network of the YOLOv8 model improves the detection performance of small target objects by fusing features of different scales. The 3×3 deformable convolution DCNv3 module is used to replace the fusion node in FPN and the standard convolution in the upsampling process, which increases the model's multi-scale adaptability to features and can more accurately capture and retain some lost shallow and boundary information when fusing high-level feature maps with low-level feature maps. In the improved FPN structure, the input feature maps are set to R3, R4 and R5, and the output feature maps are F3, F4, F5, F6 and F7. The calculation formula of the output feature map is: Where R i Represents the feature map output of each layer of the network, Represents the feature map output after convolution, F i represents the feature map output of the feature pyramid module, Ti is the feature fusion processing function, which integrates features from different layers and produces an output feature map, and Ci is the 3×3 convolution operation applied to the feature map.

6. The road damage detection method based on scale difference according to claim 3 is characterized in that: In S2-3, the specific process includes: In the top detection network of the YOLOv8 model, four feature detector structures including large, medium, small and micro are designed. The multi-scale detector enables the model to effectively identify and locate targets of different scales, enhancing the applicability of the model for small-scale target detection.

7. The road damage detection method based on scale difference according to claim 3 is characterized in that: In S2-4, the CBAM module includes channel attention (CAM) and spatial attention (SAM), and the attention map is inferred along two independent dimensions of channel and spatial order, and the attention map is multiplied by the input feature map of adaptive feature refinement. The result of the convolution layer output is first focused on meaningful disease features through the channel attention (CAM). The input image is transferred to the feature matrix by using the convolution layer, and the number of channels of the obtained feature matrix is ​​the same as the number of cores, and the value of the number of cores is 1024 or 2048; The CBAM module uses both maximum pooling and average pooling to adjust the weights of each channel in the feature map; Spatial attention (SAM) is used to extract the location information of important objects. Average pooling and maximum pooling operations are used to calculate the spatial attention along the channel axis. The connection is made along the channel dimension, and the most important diseased parts in the network are focused on by spatial attention (SAM) for processing. The calculation formula for processing the input feature map F in the CBAM module is as follows: Among them, F is the input feature map, Mc is the channel attention map, F' is the result of multiplying the feature map and the channel attention map, Ms is the spatial attention map, and F" is the result of the refined output; Channel Attention (CAM) uses the inter-channel relationship of features to generate a channel attention map, compresses the feature map through global average pooling and maximum pooling operations to reduce data loss, and each channel is regarded as an independent feature detector response input; Spatial Attention Array (SAM) generates a spatial attention map based on the spatial relationship of features, focusing on the location information of the features. It pays more attention to the specific location of the features in the image and can highlight the important feature areas in space.

8. The road damage detection method based on scale difference according to claim 3 is characterized in that: In S2-5, the specific calculation formula of the loss function is: Loss=a*L EIoU +b*DFL+c*VFL a is XX, b is XX, C is XX; The DFL loss function calculation formula is as follows: Where S i Represents the output of the Sigmod function, and y is the label value; The VFL loss function is shown as follows: Where q is the predicted label and p is the true label; The bounding box prediction loss uses EIoU, which takes into account the overlapping area, center point distance, aspect ratio, width and height size loss, convergence speed and accuracy of the predicted bounding box and the true bounding box, to improve the accuracy of bounding box positioning in the target detection task. The calculation formula is as follows: Among them, b and b gt Represents the bounding box and the ground-truth box, b and b gt represents the center point of the bounding box and the true box, ρ represents the Euclidean distance between the two center calculation points, c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box, w and h represent the length and width of the bounding box, w gt and h gt Represents the length and width of the real box, c w and c h is the minimum bounding box width and height.

9. The road damage detection method based on scale difference according to claim 1, characterized in that: In S3, the specific process includes: A deep learning modeling environment based on PyTorch was built in the operating system. The settings included a learning rate of 0.01, a batch size of 64, and a training cycle (Epoch) of 300. For road disease detection, the model input image size was 640x640; To adapt to the diversity and size differences of diseases, key detection parameters were adjusted: the confidence threshold was set to 0.4 to improve the recall rate, the NMS threshold was also set to 0.4 to reduce redundant detections, and the target box scaling value was set to 0.6 to adapt to the disease characteristics and ensure effective identification.

10. The road damage detection method based on scale difference according to claim 1, characterized in that: In S4, the specific process of constructing performance evaluation indicators includes: The main evaluation indicators introduced include recall, precision, average precision (mAP), F1-score, and frames per second (Fps) to evaluate the performance of the algorithm. Intersection over Union is introduced to evaluate the accuracy of the detection box. Four possible situations of the predicted sample are: predicting the true label as true, predicting the true label as false, predicting the false label as true, and predicting the false label as false. The accuracy rates are: Among them, TP: is classified as a positive sample and the correct part; FP: classified as positive samples and the wrong part; The recall calculation formula is as follows: Among them, FN: is classified as a negative sample and the wrong part; After the model training is completed, samples in the appropriate threshold classification data set are selected and the precision and recall are calculated. The PR curve is drawn by plotting the precision (Precision) and recall (Recall) under different thresholds to jointly evaluate the road disease detection performance of the model. The higher the value, the stronger the comprehensive detection ability of the model. The mAP calculation formula is as follows: When the PR curves of multiple models intersect, it is difficult to judge the quality. The F1-score is calculated by the precision and recall rate to judge the quality of the model performance. The F1-score is the harmonic mean of the precision and recall rate. The F1-score of each category can be calculated by calculating the Precision and Recall under each category. The calculation method is as follows: Then calculate the average F1-score of each category to get the final result. The calculation method is as follows: Fps is the number of frames that the model can process in 1 second. The larger the Fps value, the shorter the average processing time, indicating that the model has a faster detection speed. It is used to evaluate the real-time detection efficiency of the model. The calculation method is as follows: Where time is the average detection time of the model (s), frameNum represents the number of anchor frames for road crack detection, and elapsedtime is the time taken for target detection (s); IoU is calculated as follows: Among them, area(D∩G) and area(D∪G) are the intersection and union of the predicted box and the true value box respectively; The process of setting up an ablation experiment specifically includes: The effectiveness of deformable convolution modules, CBAM attention mechanisms and small-scale detectors in improving the accuracy of target detection models is verified through ablation experiments. The experiments are divided into five groups, from the baseline model YOLOv8 to the complete network model integrating all the proposed modules. All experiments use the same hyperparameter configuration to compare Faster R-CNN, Cascade R-CNN, SSD, YOLOv8 and the trained models in S3 in terms of recall, precision, average accuracy (mAP), F1-score, frames per second (Fps) and intersection over union.

Images