Road defect detection method based on improved YOLOv10 network

Abstract

The invention discloses a road defect detection method based on an improved YOLOv10 network, and belongs to the field of defect image processing. The core of the method is that a C2f-MCA module is introduced into a trunk part in the YOLOv10 network, modeling can be carried out on the attention of the width, the height and the channel at the same time, a multi-dimensional collaborative attention mechanism is established, and the accuracy and efficiency of a target recognition task are improved; a multi-level feature fusion module BDI is used to replace a traditional feature connection operation module, and semantic information and detail information in an image are enhanced by integrating multi-level feature fusion generated by an encoder; a loss function is optimized, and a SlideLoss loss function is designed, so that the model pays more attention to difficult samples, and bounding box regression is more accurate. Data training and test experiment results show that the average detection precision mAP (at) 0.5 of the new method can reach 86.6%, and the new method is superior to an existing road defect detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of defect image processing, and more specifically, to a road defect detection method based on an improved YOLOv10 network. Background Art

[0002] In the mid-20th century, some mechanical equipment began to be used for road inspection. For example, equipment such as road surface roughness meters and surface friction coefficient meters were used to detect the condition of road surfaces. These mechanical inspection methods are more efficient than manual inspection, but they still have certain limitations, such as limited coverage and slow inspection speed. After entering the 21st century, image processing and computer vision technologies have developed rapidly, providing new means for road defect detection. Through high-resolution cameras installed on vehicles or drones, road images and videos are captured in real time. Combined with image processing algorithms, road defects can be automatically detected and classified. Image preprocessing techniques such as graying, filtering, edge detection, and morphological processing are widely used to improve image quality and highlight road defect features. These technologies help solve the problems of noise and illumination changes in images, allowing subsequent detection algorithms to identify defects more accurately.

[0003] In recent years, the successful application of machine learning and deep learning algorithms in image classification and object detection has greatly promoted the development of road defect detection technology. Traditional machine learning methods such as support vector machines (SVM) and random forests are used to extract features from preprocessed images and perform classification. Deep learning methods such as convolutional neural networks (CNN) can automatically learn and extract features under large amounts of data training to improve detection accuracy. With the improvement of computing power and the generation of large-scale data, data-driven learning algorithms have shown excellent performance in road defect detection. Real-time detection and processing technologies have achieved real-time road defect detection by developing efficient algorithms and hardware acceleration technologies. For example, CNN based on deep learning can run on hardware acceleration devices such as GPUs, significantly improving detection speed and meeting the needs of real-time applications. Deep learning technology, especially generative adversarial networks (GANs), has shown great potential in road defect detection. GANs can be used for data enhancement and image generation to improve the robustness and generalization ability of the model. In addition, transfer learning technology is also widely used. By using pre-trained models, fine-tuning on small sample data sets can improve detection results. Moreover, road defect detection technology can be combined with autonomous driving and intelligent transportation systems to further improve the safety and efficiency of transportation systems. Autonomous driving vehicles are equipped with high-precision sensors and advanced detection algorithms, which can monitor road conditions in real time, avoid or slow down in time, and reduce the occurrence of traffic accidents.

[0004] Among the numerous deep learning target detection algorithms, the YOLO series of networks excels in road defect detection due to its unique design and excellent performance. A significant advantage of YOLO is that its detection speed is very fast. Unlike the traditional method of extracting regions and then classifying them, YOLO directly detects and classifies them on the entire image, avoiding multiple redundant calculations. YOLO uses an end-to-end training method, that is, the entire process from input image to output detection results is completed by a neural network. Since YOLO has a relatively simple model structure, a small number of parameters, and high computational efficiency, it is very convenient in actual deployment. However, in the task of road defect detection, due to factors such as small defect targets, complex road backgrounds, complex lighting, and unbalanced categories, directly using the basic version of the YOLO network will lead to problems such as low detection accuracy, high false positive rate, and poor generalization ability.

[0005] After searching, the Chinese patent application No. 202410617362.9 discloses a road defect detection method combining YOLOv8 and RTDETR. The method uses the YOLOv8 feature extraction network to realize multi-scale feature extraction of road radar images, and uses the RTDETR defect detection network to identify and segment the defect area with the extracted features. In the Backbone network of the YOLOv8 feature extraction network, the CAGM module is used to interact with the feature information obtained by the Conv layer and the GFNet layer multiple times to enrich the gradient flow information, thereby improving the accuracy and recall rate. However, due to the expensiveness of LiDAR equipment, the cost is higher than that of pure image detection, and the amount of data generated by LiDAR is large, making it difficult to process data, but it can avoid the impact of weather on detection accuracy and the limitation of viewing angle. The Chinese patent application with number 202410279380.0 discloses a road defect detection technology based on unsupervised full attention recognition. It adds a full attention network and introduces an unsupervised learning network to correctly and intuitively reflect the location of road crack defects. It does not require manual labeling of data, which greatly saves manpower and time costs. However, if the data set is too large, it requires very high computing and storage resources.

[0006] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0007] 1. Technical problem to be solved by the invention

[0008] In order to overcome the problems of low detection accuracy, high false positive rate and poor generalization ability in road defect detection tasks, the present invention improves on the existing YOLOv10 network and provides a method dedicated to road defect detection, which is called a road defect detection method based on an improved YOLOv10 network. The network of the present invention is based on the original structure, adds an MCA attention mechanism to the C2f module of the backbone network, uses SDI combined with the idea of ​​bidirectional feature fusion to replace the original feature link Concat layer, and adds a SlideLoss loss function, which improves performance while having almost no additional computational burden, more effectively combines features of different scales, improves the problem of sample imbalance, and improves the detection accuracy of the image.

[0009] 2. Technical solution

[0010] In order to achieve the above object, the technical solution provided by the present invention is:

[0011] The road defect detection method based on the improved YOLOv10 network of the present invention comprises the following steps:

[0012] Step 1: Collect high-quality road defect image data, mark the collected data for defects, randomly select a portion as a test set, and then divide the remaining images into training sets and validation sets according to the data categories of road defects;

[0013] Step 2: Use the C2f-MCA module to replace the C2f module in the YOLOv10 network backbone. The C2f-MCA module is obtained by adding the MCA self-attention mechanism to the Bottleneck layer in the original C2f module to achieve multi-dimensional collaborative attention.

[0014] Step 3: Introduce the BDI module into the backbone of the YOLOv10 network to replace the Concat layer in the original network. The BDI module is a multi-level feature fusion module to enhance the semantic information and detail information in the image.

[0015] Step 4: Construct the SlideLoss loss function and use it to improve the attention of difficult samples with unbalanced samples and make the bounding box regression more accurate.

[0016] Step 5: Use the training set and the validation set to train the improved network and obtain the optimized model parameters to ensure the effectiveness and practicality of the model in road defect detection;

[0017] Step 6: Use the trained model to predict the test set and output the detected road image.

[0018] As a further improvement of the present invention, in step 2, the C2f-MCA module first includes the first convolution layer cv1 to receive data; then the output of cv1 is divided into two equal parts according to the channel dimension, one of which is directly passed to the Concat layer and the other is passed to Split, and the output of Split is also divided into two equal parts according to the channel dimension, one of which is directly passed to the Concat layer and the other is processed by two Bottleneck modules; C2f-MCA is repeated 3 times in the network, using 1 Bottleneck module, and the C2f-MCA module is repeated 6 times in total; after receiving the data, the Bottleneck module is divided into two equal parts according to the channel dimension, one of which is directly passed to the Concat layer, and the other is passed to the MCA module for calculation after convolution of two convolution layers, and then passed to the Concat layer; next, the Concat layer splices the received results in the channel dimension; finally, the output is obtained through the second convolution layer cv2. This partial improvement can make the spliced ​​features come from information from different branches to enrich and improve the expressive power of the features.

[0019] As a further improvement of the present invention, in step 2, the MCA module includes: first, performing residual extraction on the input data; then performing three-branch processing on the residual extracted data, and the three branches respectively model the attention in three dimensions of height, width and number of channels, and the modeling process successively includes dimensional transformation operation, squeezing transformation, average pooling and standard deviation pooling, convolution, excitation transformation, convolution, dimensional transformation, and weight multiplication operation; next, performing dimensional transformation on the data of different branches to unify the dimensions; finally, performing weighted summation on the data output by the branches and obtaining the output of the final MCA module.

[0020] As a further improvement of the present invention, in step 3, the BDI module uses a multi-level feature fusion module SDI to replace the Concat layer of the original model, and adds bidirectional feature fusion therein; specifically: first, a list channels containing channels of each layer is received, the number of convolutional layers is determined according to the number of elements of channels, and the fusion weights, swish activation function and convolutional layer convs are initialized; then, a feature map list xs is received, the weights are calculated, and forward feature fusion and backward feature fusion are performed, and the feature map sequence obtained by the forward feature fusion is element-by-element multiplied and the feature map series after the backward feature fusion is element-by-element multiplied; finally, the two multiplied feature maps are weightedly accumulated to obtain the final feature map.

[0021] As a further improvement of the present invention, in step 3, the process of forward feature fusion includes: forward feature fusion is a process from low layer to high layer, that is, starting from the second feature map, layer by layer, first adjusting each feature map to the target size; if the size of the current feature map is larger than the target size, using an adaptive average pooling layer to downsample it to the target size; if the size of the current feature map is smaller than the target size, using a bilinear interpolation method to upsample it to the target size; then each feature map is multiplied by the corresponding weight, and finally the product result is input into the corresponding convolution layer to obtain the final high-level feature map; the purpose of this is to fuse feature maps at different levels to obtain richer feature representations.

[0022] As a further improvement of the present invention, in step 3, the process of backward feature fusion includes: backward feature fusion is a process from high layer to low layer, that is, starting from the second to last feature map, processing layer by layer; firstly, the size of the feature map is adjusted and multiplied with the corresponding weight; then, it is converted through the convolution layer, and the high-level information can be further integrated in the entire feature map pyramid; finally, all the feature maps that have undergone the convolution operation are cumulatively multiplied element by element to obtain the final low-level feature map.

[0023] As a further improvement of the present invention, in step 4, a SlideLoss loss function is designed to improve the weight of the difficult sample allocation in view of the sparsity of the difficult samples. The SlideLoss loss function calculation includes: first using BCEWithLogitsLoss to calculate the binary cross entropy loss, and then passing the calculated loss result to the SlideLoss loss function. The Slideloss loss function adjusts the loss value of each sample through a weighted mechanism, and finally outputs the aggregated loss value.

[0024] As a further improvement of the present invention, in step 5, the Adam algorithm is used to iteratively update the network model parameters during network training, and the training cycle is 300 epochs.

[0025] As a further improvement of the present invention, in step 6, the image outputted by the test data includes the category of the data, the bounding box and the prediction accuracy.

[0026] 3. Beneficial effects

[0027] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0028] (1) The road defect detection method based on the improved YOLOv10 network of the present invention introduces the MCA multi-dimensional collaborative attention mechanism in the C2f module of the backbone network, and uses a new method of simultaneously inferring the channel, height and width dimension attention using a three-branch architecture, with almost no additional overhead. While improving the performance, there is almost no additional computational burden, and the mAP50 is improved by 0.6% compared with the original model.

[0029] (2) The road defect detection method based on the improved YOLOv10 network of the present invention uses a multi-level feature fusion module SDI to replace the Concat layer of the original model to enhance the semantic information and detail information in the image, and adds bidirectional feature fusion thereto. The bidirectional feature fusion is used to more effectively combine features of different scales to improve the accuracy. Compared with the original model, the mAP50 is improved by 2.5% and the accuracy is improved by 1.7%.

[0030] (3) The road defect detection method based on the improved YOLOv10 network of the present invention adds the SlideLoss loss function on the basis of the original classification loss, and attempts to assign higher weights to difficult samples to improve the problem of sample imbalance and make the classification more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is an overall flow chart of an embodiment of the present invention;

[0032] Figure 2 This is an improved YOLOv10 network diagram of an embodiment of the present invention;

[0033] Figure 3 is an architecture diagram of a C2f-MCA module according to an embodiment of the present invention;

[0034] Figure 4 is an architecture diagram of the MCA according to an embodiment of the present invention;

[0035] Figure 5 This is an architecture diagram of a feature fusion module BDI according to an embodiment of the present invention;

[0036] Figure 6 It is a classification weight curve diagram of SlideLoss of an embodiment of the present invention;

[0037] Figure 7 Graphs showing the detection results of different sample graphs in the embodiment;

[0038] Figure 8 It is the mAP50 (B) curve of different methods in the embodiment. DETAILED DESCRIPTION

[0039] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings.

[0040] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0041] The present invention will be further described below in conjunction with the embodiments.

[0042] Example

[0043] The road defect detection method based on the improved YOLOv10 network of this embodiment includes the following steps:

[0044] Step 1: Collect high-quality road defect image data, mark the collected data for defects, and randomly select 20% as the test set. Then divide the remaining images into training set and validation set according to the data categories of road defects; the data ratio of training set and validation set is 7:3;

[0045] Step 2: Replace the C2f module with the C2f-MCA module in the backbone of the YOLOv10 network. The C2f-MCA module is obtained by adding the MCA self-attention mechanism to the Bottleneck layer in the original C2f module to achieve multi-dimensional collaborative attention. This embodiment improves the performance of the model by adding the MCA multi-dimensional collaborative attention module to the C2f layer and focusing on the more important parts of the input data.

[0046] The specific process of step 2 is as follows:

[0047] Step 21, first transfer the input data to the first convolutional layer cv1, receive the input feature map and generate an intermediate feature map;

[0048] Step 22, then split the generated intermediate feature map into two parts, one part is transmitted to the final Concat layer, and the other part is passed to two Bottleneck modules for further processing. C2f-MCA is repeated 3 times in the network, using 1 Bottleneck module, and a total of 6 C2f-MCA modules are repeated;

[0049] Step 23, after the Bottleneck module receives the data, it passes the data to the MCA module after convolution of two convolutional layers. The MCA module first extracts the residual of the input data; then the residual extracted data is processed in three branches, and the three branches model the attention of the three dimensions of height, width and number of channels respectively. The modeling process includes dimension transformation operation, squeezing transformation, average pooling and standard deviation pooling, convolution, excitation transformation, convolution, dimension transformation, and weight multiplication operation; next, the dimension of the data of different branches is transformed to make the dimension unified; then the generated feature map is passed to the Concat layer for splicing;

[0050] like Figure 4 The MCA module consists of three parallel branches, where the H and W branches are responsible for capturing the interdependence of features in the spatial dimension, and the C branch is mainly used to capture the interdependence between channels. First, an F∈R C×H×W The feature map of , where C is the number of channels, H and W are the height and width of the spatial feature map, respectively. In the middle branch, the feature map is rotated 90 degrees counterclockwise to obtain the transformed feature map F W ∈R W ×H×C , F W Input into the squeeze transformation to get the aggregated feature map Then aggregate the feature map is passed to the excitation transformation to capture the interaction between features in the spatial dimension, and the resulting width-wise feature weights are expressed as Next, the attention weight A is generated through the Sigmoid activation function W ∈R W×1×1 , and then through element-by-element multiplication, we get the enhanced feature map F' W ∈R W×H×C Finally, the feature map F is obtained after dimension transformation W ∈R C×H×W , which is consistent with the dimension of the original input.

[0051] Similarly, the leftmost branch also first rotates the feature map 90 degrees counterclockwise to obtain the rotated feature map F H ∈R H×C×W ; Then the feature map is sequentially sent to the squeeze transformation and the excitation transformation to obtain the aggregated feature map and the feature weights in terms of height F H ∈R 1×1×H ; Next, a specific attention weight A is generated through the activation function H ∈R H×1×1 ; Finally, element-by-element multiplication and dimension transformation are used to obtain the feature map F"H ∈R C×H×W , which is consistent with the feature map dimension of the original input.

[0052] The design concept of the rightmost branch is similar to some representative channel attention mechanisms, which is mainly responsible for modeling spatial interdependencies and capturing interactions between channels. First, the same feature map F is generated by mapping C ∈R C×H×W ; Then it is input into the squeeze transformation and the excitation transformation in turn, and the aggregate features and channel feature weights F are continuously inferred C ∈R C×1×1 , F C ∈R C×1×1 ; Next, the attention weight A of a specific channel is calculated through the Sigmoid activation function C ∈R C×1×1 , and generate the feature map F' through dimension transformation C ∈R C×H×W ;Finally, the feature map F is obtained by element-by-element multiplication mapping. C ∈R C×H×W , which is consistent with the dimension of the original input feature map. In the integration stage, the branch output data is weighted and summed to obtain the output of the final MCA module to obtain the final feature map.

[0053] Step 24: Input the feature map output by the cv1 layer, the feature map output by the Split layer, and the feature map output by each Bottleneck module into the Concat layer for feature fusion, which can comprehensively utilize the scale and multi-level information of multiple features;

[0054] Step 25: Convolve the feature map output by the Concat layer through the second convolutional layer cv2 to obtain the C2f-MCA output. This improvement allows the concatenated features to come from information from different branches, thereby enriching and improving the expressiveness of the features.

[0055] like Figure 3 As shown in the figure, in the C2f-MCA module, the input data first enters the first convolutional layer cv1, part of it is directly passed to the final Concat block, and the other part is sequentially passed to Split, which converts the features into Figure 1Part of it is directly passed to the final Concat block, and the other part is passed to each Bootleneck module in sequence. The Bootleneck module passes part of the data to the final Concat block, and the remaining data is processed by the first and second Conv layers, and then input into the MCA self-attention module to generate the processed feature map. Next, the feature map passed by cv1, Split, Bootleneck and the processed feature map are spliced ​​through the Concat layer, and finally the output is obtained through the second convolution layer cv2 to generate the final feature map.

[0056] Step 3: Introduce the BDI module into the backbone of the YOLOv10 network to replace the Concat layer in the original network. The BDI module is a multi-level feature fusion module to enhance the semantic information and detail information in the image.

[0057] The specific process of step 3 is as follows:

[0058] Step 31, receive a list channels containing channels of each layer, determine the number of convolutional layers according to the number of elements in channels, and initialize the fusion weights, swish activation function and convolutional layer convs;

[0059] Step 32, the SDI module receives a feature map list xs and calculates the weights. Specifically, the weights are first activated using the swish activation function, then the sum of the weights after activation is calculated, and finally the original weights are divided by the sum of the weights after activation so that the sum of the normalized weights is 1, and the weight corresponding to each element in the feature map list xs is obtained;

[0060] Step 33, traverse the feature map list, and perform forward and backward feature fusion on each feature map. Forward feature fusion is a process from low layer to high layer, that is, starting from the second feature map, layer by layer, and first adjust each feature map to the target size.

[0061] If the size of the current feature map is larger than the target size, it is downsampled to the target size using an adaptive average pooling layer; adaptive average pooling is a pooling operation that adjusts the input feature map to a specified output size. The specific process is as follows:

[0062] Assume that the size of the input feature map is (H in , W in ), the size of the output feature map is (H out , W out ).

[0063] First, we calculate the high k of each pooling window size h and wide k w :

[0064]

[0065] here Both represent rounding down operations, and then calculate the step length s of each pooling window height h and width step length s w :

[0066]

[0067] Then calculate the starting position of the pooling window (h start ,w start ) and the end position (h end ,w end ), i,j are each input unit input (i,j) index:

[0068]

[0069] Finally, each input unit input(i,j) is averaged and pooled to obtain the output unit output(i,j):

[0070]

[0071] If the size of the current feature map is smaller than the target size, use bilinear interpolation to upsample it to the target size. Bilinear interpolation is a common interpolation method used to adjust the size of an image in an image. It calculates the pixel value of the new position by linear interpolation between four known adjacent pixels. The steps are as follows:

[0072] Suppose a pixel is to be adjusted to the target size, and the positions of the four adjacent pixels are known to be (x1, y1), (x1, y2), (x2, y1), (x2, y2). The values ​​of these pixels are I1, I2, I3, and I4 respectively.

[0073] First, calculate the distance ratio dx and dy of the target position (x, y) relative to the adjacent pixels:

[0074] dx=(x-x1) / (x2-x1)

[0075] dy=(y-y1) / (y2-y1)

[0076] Then, linear interpolation is performed in the horizontal and vertical directions according to the distance ratio:

[0077] I interp (x,y)=(1-dx)·(1-dy)·I1+dx·(1-dy)·I2+(1-dx)·dy·I3+dx·dy·I4

[0078] Among them I interp (x,y) is the interpolated pixel value at position (x,y).

[0079] Each feature map is then multiplied by the corresponding weight, and the product is finally input into the corresponding convolution layer to obtain the final high-level feature map. The purpose of this is to fuse feature maps at different levels to obtain a richer feature representation. Backward feature fusion is a process from high to low layers, that is, starting from the second to last feature map and processing layer by layer. First, the feature map size is adjusted and multiplied by the corresponding weight; then it is transformed through the convolution layer to further fuse high-level information in the entire feature map pyramid; finally, all feature maps that have undergone convolution operations are cumulatively multiplied element by element to obtain the final low-level feature map. The purpose of this process is to further fuse high-level information in the entire feature map pyramid;

[0080] Step 34: Perform weighted accumulation on the feature maps processed by the forward feature fusion and the backward feature fusion to obtain a final feature map;

[0081] like Figure 3 As shown, first, a list channels containing channels of each layer is received, the number of convolutional layers is determined according to the number of elements in channels, and the fusion weights, swish activation function and convolutional layer convs are initialized. Then, a feature map list xs is received, and the weights are calculated. The feature map sequence obtained by forward feature fusion is element-by-element multiplied, and the feature map series after backward feature fusion is element-by-element multiplied. Finally, the two multiplied feature maps are weighted and accumulated to obtain the final feature map.

[0082] Step 4: Construct the SlideLoss loss function and use it to improve the attention of difficult samples with unbalanced samples and make the bounding box regression more accurate.

[0083] The specific process is as follows:

[0084] Step 41, pass the true value and the predicted value to the binary classification loss function BCEWithLogitsLoss to calculate the basic loss. BCEWithLogitsLoss is a loss function for binary classification that combines the Sigmoid function and the binary cross entropy loss. The loss value Lx is calculated as follows:

[0085]

[0086] Where I represents the total number of categories, σ(·) is the Sigmoid function, which maps the output of the model to the interval [0,1], and p i Represents the predicted probability value of the i-th sample, y iis the true label of the i-th sample, which takes the value of 0 or 1. i ) as input for probability estimation and loss calculation. i Represents sample x i The real label is used to calculate the binary cross loss, which helps the model learn the ability to match the predicted results with the real label. i = 1, BCEWithLogitsLoss will penalize the degree to which the model's predicted value deviates from 1; when y i = 0, BCEWithLogitsLoss will penalize the degree to which the model's predicted values ​​deviate from 0.

[0087] Step 42, then dynamically adjust the loss value according to the true label value sc and the preset threshold u:

[0088] For samples with sc below u-0.1, the weight is fixed to 1.0 to enforce the loss in these regions.

[0089] For samples where sc is between u-0.1 and u, the weight is e 1-u , so that the loss of these samples is moderately considered.

[0090] For samples whose sc is higher than u, a gradually weakening weight e is given 1-sc , in order to reduce the impact of losses on the prediction results in these areas.

[0091] like Figure 6 As shown in the figure, the samples are divided into positive samples and negative samples by the parameter u, and then different samples are weighted. The weighting function is as follows:

[0092]

[0093] Step 43: By multiplying the sc values ​​of these samples with the corresponding weights f(sc,u), a modulation weight modulating_weight is obtained, and then the loss of each sample is weighted adjusted Lxw=Lx*modulating_weight to obtain a tensor Lxw containing the weighted loss value.

[0094] Step 44: Calculate the preset reduction value according to the loss function to return the loss value in different forms:

[0095] If reduction is 'mean', returns the mean loss over all samples.

[0096] If reduction is 'sum', the total loss of all samples is returned.

[0097] If reduction is 'none', returns the single sample loss.

[0098] Step 5: Use the training set and validation set to train the improved network to obtain optimized model parameters;

[0099] Step 6: Use the trained model to predict the test set and output the detected road image.

[0100] Effectiveness Evaluation Criteria

[0101] In the embodiment, the RDD2022 public data set is used, which is provided by the Global Road Damage Detection Challenge; since the format of some pictures cannot be directly used by YOLO and a large number of pictures do not have the required annotations, the data format is converted and cleaned. After cleaning, there are a total of 36,366 pictures, with a total of 8 categories, namely D00 (longitudinal cracks), D10 (transverse cracks), D20 (cracking), D40 (potholes), D43 (blurred intersections), D44 (blurred white lines), D50 (manhole covers), and Repair. These pictures are divided into 7:3 according to categories, with 22,683 pictures in the training set, 13,684 pictures in the validation set, and a separate test set with 9,035 pictures. The training round is set to 300, the batch size is set to 48, the initial learning rate is 0.01, and the input picture size is 640×640.

[0102] In this embodiment, in order to better evaluate the performance of the model, accuracy, recall, mAP50, and mAP50-95 are used as evaluation indicators. mAP50 is the average detection accuracy mAP of all categories when the IoU threshold is 0.5, and mAP50-95 is the average detection accuracy mAP with a step size of 0.05 and a threshold from 0.5 to 0.95. The specific formula is as follows:

[0103]

[0104] TP represents the positive sample predicted as a positive example, FP represents the negative sample predicted as a positive example, and FN represents the positive sample predicted as a negative example. The calculation formula of detection accuracy AP is as follows:

[0105]

[0106] Where N represents the total number of categories, and mAP is the average value of AP of all categories. The formula is as follows:

[0107]

[0108] Processing result evaluation

[0109] As shown in Table 1 below, it can be seen that the improved model is superior to the YOLOv8n-R method in the patent application No. 202410617362.9 and the traditional YOLOv10n method in terms of road defect detection recognition effect. The precision rate of the present invention is 85.2%, the recall rate is 81.0%, and the mAP50 is 87.2%. Compared with the original model YOLOv10n, although the memory usage has increased, other indicators have improved and mAP50 has increased by about 5.1%. Figure 7 This is the prediction result image of the test set, including the defect category, bounding box and score. Figure 8 It can also be seen from the curve that the curve of the model does not tend to be smooth. With the increase of the number of training times, mAP50 will be improved.

[0110] Table 1 Quantitative evaluation values ​​of processing results of different models

[0111]

[0112] Therefore, in summary, the method of the present invention can effectively improve the detection accuracy of road defects and has certain application prospects.

[0113] The above is a schematic description of the present invention and its implementation mode, which is not restrictive and is only one of the implementation modes of the present invention, and is not actually limited thereto. Therefore, if a person skilled in the art is inspired by it and, without departing from the purpose of the invention, designs a structure and an implementation mode similar to the technical solution without creativity, they shall all fall within the protection scope of the present invention.

Claims

1. A road defect detection method based on an improved YOLOv10 network, characterized in that: include: Step 1: Collect road defect image data, mark the collected data for defects, randomly select a portion as a test set, and then divide the remaining images into training sets and validation sets according to the data categories of road defects; Step 2: Use the C2f-MCA module to replace the C2f module in the YOLOv10 network backbone. The C2f-MCA module is obtained by adding the MCA self-attention mechanism to the Bottleneck layer in the original C2f module to achieve multi-dimensional collaborative attention. Step 3: Introduce the BDI module into the backbone of the YOLOv10 network to replace the Concat layer in the original network. The BDI module is a multi-level feature fusion module to enhance the semantic information and detail information in the image. Step 4: Construct the SlideLoss loss function and use it to improve the attention of difficult samples with unbalanced samples and make the bounding box regression more accurate. Step 5: Use the training set and validation set to train the improved network to obtain optimized model parameters; Step 6: Use the trained model to predict the test set and output the detected road image.

2. The road defect detection method based on the improved YOLOv10 network according to claim 1, characterized in that: In the step 2, the C2f-MCA module first includes the first convolution layer cv1 to receive data; then the output of cv1 is divided into two equal parts according to the channel dimension, one of which is directly passed to the Concat layer, and the other is passed to the Split, and the output of the Split is also divided into two equal parts according to the channel dimension, one of which is directly passed to the Concat layer, and the other is processed by two Bottleneck modules; C2f-MCA is repeated 3 times in the network, using 1 Bottleneck module, and the C2f-MCA module is repeated 6 times in total; after receiving the data, the Bottleneck module is divided into two equal parts according to the channel dimension, one of which is directly passed to the Concat layer, and the other is passed to the MCA module for calculation after being convolved by two convolutional layers, and then passed to the Concat layer; Next, the Concat layer concatenates the received results in the channel dimension; finally, the output is obtained through the second convolutional layer cv2.

3. The road defect detection method based on the improved YOLOv10 network according to claim 2, characterized in that: In the step 2, the MCA module includes: first, performing residual extraction on the input data; then performing three-branch processing on the residual extraction data, and the three branches respectively model the attention in three dimensions of height, width and number of channels, and the modeling process successively includes dimension transformation operation, squeezing transformation, average pooling and standard deviation pooling, convolution, excitation transformation, convolution, dimension transformation, and weight multiplication operation; next, performing dimension transformation on the data of different branches to unify the dimensions; finally, performing weighted summation on the data output by the branches and obtaining the output of the final MCA module.

4. The road defect detection method based on the improved YOLOv10 network according to claim 1, characterized in that: In the step 3, the BDI module uses a multi-level feature fusion module SDI to replace the Concat layer of the original model, and adds bidirectional feature fusion thereto; specifically: first, receive a list channels containing channels of each layer, determine the number of convolutional layers according to the number of elements of channels, and initialize the fusion weights, swish activation function and convolutional layer convs; then, receive a feature map list xs, calculate the weights, and perform forward feature fusion and backward feature fusion, perform element-by-element multiplication on the feature map sequence obtained by the forward feature fusion and perform element-by-element multiplication on the feature map series after the backward feature fusion processing; finally, perform weighted accumulation on the two multiplied feature maps to obtain the final feature map.

5. The road defect detection method based on the improved YOLOv10 network according to claim 4 is characterized in that: In the step 3, the process of forward feature fusion includes: forward feature fusion is a process from low layer to high layer, that is, starting from the second feature map, layer by layer, first adjusting each feature map to the target size; if the size of the current feature map is larger than the target size, using an adaptive average pooling layer to downsample it to the target size; if the size of the current feature map is smaller than the target size, using a bilinear interpolation method to upsample it to the target size; then multiplying each feature map with the corresponding weight, and finally inputting the product result into the corresponding convolution layer to obtain the final high-level feature map.

6. The road defect detection method based on the improved YOLOv10 network according to claim 4, characterized in that: In step 3, the process of backward feature fusion includes: backward feature fusion is a process from high layer to low layer, that is, starting from the second to last feature map, processing layer by layer; firstly, the size of the feature map is adjusted and multiplied with the corresponding weight; then, it is converted through the convolution layer, and the high-level information can be further integrated in the entire feature map pyramid; finally, all the feature maps that have undergone the convolution operation are cumulatively multiplied element by element to obtain the final low-level feature map.

7. The road defect detection method based on the improved YOLOv10 network according to claim 1, characterized in that: In step 4, the SlideLoss loss function calculation includes: first using BCEWithLogitsLoss to calculate the binary cross entropy loss, and then passing the calculated loss result to the SlideLoss loss function. The Slideloss loss function adjusts the loss value of each sample through a weighted mechanism, and finally outputs the aggregated loss value.

8. The road defect detection method based on the improved YOLOv10 network according to claim 1, characterized in that: In step 5, the Adam algorithm is used to iteratively update the network model parameters during network training, and the training cycle is 300 epochs.

9. The road defect detection method based on the improved YOLOv10 network according to claim 1, characterized in that: In step 6, the image outputted by the test data includes the category of the data, the bounding box and the prediction accuracy.

Images