A method and system for detecting road defects based on UAV optical remote sensing images

Abstract

This invention provides a method and system for road defect detection based on UAV optical remote sensing imagery, belonging to the field of road defect detection. To address the shortcomings of existing road defect detection methods in small target recognition and feature extraction under complex background interference, this invention utilizes a CSP-SMAM module, enabling the model to efficiently extract multi-scale features, particularly enhancing the perception of targets of different scales in complex scenes. Combined with a CAFPN structure, the model further improves feature fusion and localization accuracy. The Wise-MPDIoU loss function optimizes the target localization process, making it particularly suitable for small target detection. Finally, the CA attention mechanism enhances the model's localization ability through direction and position awareness information, improving the model's recognition accuracy in complex backgrounds.

Description

Technical Field

[0001] This invention relates to the field of road defect detection technology, and more specifically, to a method and system for road defect detection based on UAV optical remote sensing images. Background Technology

[0002] As a crucial foundation for socio-economic development, the health of road networks directly impacts public travel safety and the sustained stability of economic activities. Traditional inspection methods primarily rely on manual patrols, ground monitoring, and recording by fixed cameras. However, these methods often have limitations when dealing with large-scale and complex road environments. Especially in situations where road defects are widespread, hidden, and diverse, manual patrols are inefficient and susceptible to factors such as terrain and weather, making it difficult to detect and repair road defects in a timely and accurate manner. To address these issues, with the continuous development of drone technology, it has been widely applied in urban management, traffic monitoring, and other fields. Drones, with their small size, maneuverability, lack of terrain limitations, wide field of view, and high stability, have demonstrated significant advantages in high-risk, high-intensity road defect detection tasks.

[0003] By equipping drones with optical remote sensing image sensors, comprehensive and efficient monitoring and location of road defects can be achieved from the air. From a high-altitude perspective, drones can quickly scan large road areas, identifying the specific locations of surface defects, thus significantly improving the efficiency and accuracy of road defect detection. However, achieving this goal requires combining drone flight technology with image processing technology, utilizing object detection algorithms from deep learning to analyze and identify remote sensing images. Most existing object detection technologies rely on training with general datasets (such as COCO and Pascal VOC), but these datasets differ significantly from road defect images in scale, category, and complexity, making direct transfer and application difficult.

[0004] In UAV-based road defect detection, remote sensing images typically contain a large amount of complex background information. Defect targets are often small objects, and are significantly affected by factors such as terrain and weather, resulting in a lack of clear image features. Therefore, to address this issue, researchers need to construct targeted training datasets and combine them with suitable deep learning models, taking into account practical application contexts, to effectively improve the accuracy and speed of road defect detection. Furthermore, traditional image processing methods have certain bottlenecks in defect detection, classification, and localization. It is necessary to combine algorithms such as deep convolutional neural networks (CNNs) and region convolutional neural networks (R-CNNs) to enhance the model's learning ability, overcome problems such as complex backgrounds and insufficient target features, and thus achieve accurate and efficient road defect detection.

[0005] In summary, road defect detection methods based on UAV optical remote sensing imagery can significantly improve inspection efficiency and achieve precise location of hard-to-detect defect areas. With the continuous development of UAVs and deep learning technologies, future road defect detection will become more intelligent and efficient, ensuring the safety and smooth flow of urban traffic on a larger scale. Summary of the Invention

[0006] The technical problem to be solved by this invention is:

[0007] To address the shortcomings of existing road defect detection methods in feature extraction under conditions of small target recognition and complex background interference.

[0008] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0009] This invention provides a method for detecting road defects based on UAV optical remote sensing imagery, comprising the following steps:

[0010] S100. High-resolution road damage images are collected using UAV equipment, a raw database is constructed and preprocessed, and the preprocessed dataset is divided into training set, validation set and test set.

[0011] S200: Construct a YOLOv8-MSCA model, consisting of a backbone network, a neck network, and a head network connected in sequence; employ the CSP-SMAM module and introduce a partial multi-scale feature extraction strategy FDPN to extract feature information at different scales across channels; design a coordinate attention mechanism to encode the feature map in both horizontal and vertical directions, capturing short-range and long-range dependency information; utilize a feature fusion mechanism combined with the channel attention module to improve the understanding of image details and enhance the model's ability to focus on key information; design a CAFPN structure for the neck network, considering feature focusing and diffusion mechanisms to aggregate and transfer multi-scale disease features to the detection stage;

[0012] S300. The YOLOv8-MSCA model is trained using the training set, the model is evaluated using the test set, and the model parameters are adjusted based on the evaluation results to finally obtain the trained road defect detection model.

[0013] S400: Acquire road damage images taken by the drone that need to be processed, and input them into the trained YOLOv8-MSCA model to obtain classification and detection results.

[0014] Furthermore, the backbone network includes a first Conv module, a second Conv module, a first CSP-SMAM module, a third Conv module, a second CSP-SMAM module, a fourth Conv module, a third CSP-SMAM module, a fifth Conv module, a fourth CSP-SMAM module, an SPPF module, and a C_Attention module connected in sequence.

[0015] The neck network includes a first FDPN module, a first Upsample module, a first concat module, a fifth CSP-SMAM module, a second concat module, and a sixth CSP-SMAM module connected in sequence; simultaneously, the first FDPN module, the sixth Conv module, the third concat module, the seventh CSP-SMAM module, the fourth concat module, and the eighth CSP-SMAM module are connected; the second CSP-SMAM module is connected to the first FDPN module, and the C_Attention module is connected to the first FDPN module; the first FDPN module is connected to the second FDPN module, and the second FDPN module, the second Upsample module, and the second concat module are connected. Then, the second FDPN module, the sixth Conv module, and the fourth concat module are connected; the third concat module is connected to the C_Attention module; the first FDPN module is connected to the CAttention module; the first FDPN module is connected to the third CSP-SMAM module; the first FDPN module is connected to the second CSP-SMAM module; the first concat module and the second CSP-SMAM module; the first Upsample module is connected to the second concat module; the sixth Conv module is connected to the fourth concat module; the fifth CSP-SMAM module is connected to the second FDPN module; and the seventh CSP-SMAM module is connected to the second FDPN module.

[0016] The head network includes a first Detect module, a second Detect module, and a third Detect module; the first Detect module is connected to the sixth CSP-SMAM module, the second Detect module is connected to the second FDPN module, and the third Detect module is connected to the eighth CSP-SMAM module; each Detect module extracts features and performs target detection through convolution operations.

[0017] Furthermore, the CSP-SMAM module includes,

[0018] Input feature map F in The feature map F is of size H×W×C. First, a 3×3 convolution operation (Conv3×3) is used to extract features from all channels, resulting in the feature map F.3×3 The calculation formula is:

[0019] F 3×3 =Conv3×3(F in )

[0020] Then, the input feature map F 3×3 The process is divided into two parts. One part undergoes further feature extraction via a 5×5 convolution operation (Conv5×5) to obtain the feature map F. 5×5 The formula is as follows:

[0021] F 5×5 =Conv5×5(F 3×3 )

[0022] The other part remains unchanged, resulting in feature map F. 3×3_remain The formula is as follows:

[0023] F 3×3_remain =F 3×3

[0024] Next, the F part after 5×5 convolution processing 5×5 It is divided into two parts again. One part is further processed by a 7×7 convolution operation (Conv7×7) to extract features, resulting in feature map F. 7×7 The calculation formula is:

[0025] F 7×7 =Conv7×7(F 5×5 )

[0026] The other part remains unchanged, resulting in feature map F. 5×5_remain2 The formula is:

[0027] F 5×5_remain2 =F 5×5

[0028] The obtained F 7×7 Feature map and invariant part F 5×5_remain2 and F 3×3_remain The feature maps are concatenated to obtain a fused multi-scale feature map F. concat The formula is:

[0029] F concat =[F 3×3_remain ,F 5×5_remain2 ,F 7×7 ]

[0030] Then, the spliced ​​feature map F concat Perform a 1×1 convolution operation Conv1×1 to reduce the number of channels and further refine feature information to obtain the feature map F. conv1×1 The formula is:

[0031] F conv1×1 =Conv1×1(F concat )

[0032] Finally, F conv1×1 With input feature map F in Perform addition operations to form the final output feature map F. out The formula is:

[0033] F out =F conv1×1 +F in .

[0034] Furthermore, in the CAFPN module, the three scale feature maps P3, P4, and P5 output from the backbone network are used as input. First, the feature maps at each scale are uniformly processed by three lightweight downsampling modules Adown to obtain feature representations of the same resolution.

[0035]

[0036] The three downsampled feature maps are concatenated along the channel dimension to obtain the fused feature map F. CAFPN :

[0037]

[0038] F fused feature map F CAFPN The input is fed into the feature focusing module, where features are extracted using depthwise separable convolutions of different scales to enhance its context modeling capabilities. Specifically, this includes:

[0039]

[0040] F id =F CAFPN

[0041] Where DWConv(·) is the depthwise separable convolution operation, K 5×5 K 7×7 K 9×9 K 11×11 Convolutional kernels of sizes 5×5, 7×7, 9×9, and 11×11;

[0042] By summing and fusing the five branches pixel by pixel, a multi-scale context-aware feature map is obtained:

[0043]

[0044] Fusion Feature Map F sum The convolution output F is obtained by compression and processing using standard convolution.conv1 The formula is:

[0045] F conv1 =Conv(F sum )

[0046] Finally, a residual enhancement mechanism is introduced to improve F CAFPN The output of the convolution is summed with the output of the convolution to form the final CAFPN module output:

[0047] F out =F conv1 +F CAFPN .

[0048] Furthermore, in the CAFPN module of the neck network, the feature focusing mechanism FFM and the feature diffusion mechanism FDM are introduced.

[0049] The input feature map first passes through the first FDPN module, which transforms the input feature map into feature maps of different scales through multi-scale feature extraction; then, the first Upsample module upsamples the low-resolution feature map; and finally, the first concat module concatenates the feature maps of different scales.

[0050] The feature focusing mechanism extracts information from multiple scales through multiple parallel convolution operations; the input feature map is processed by the CSP-SMAM module, which adopts a partial multi-scale convolution strategy to extract features at different scales and enhances the expression of detailed features through feature focusing.

[0051] The feature diffusion mechanism uses the second FDPN module to diffuse the focused features, allowing features at different scales to influence and merge with each other. During the diffusion process, the feature map after feature focusing is expanded by a 7×7 convolution operation, while simultaneously propagating the feature map from one scale to other scales. The diffused feature map is then fused with feature maps at other scales and finally input into the detection module for target detection.

[0052] Furthermore, let the feature map be... After feature focusing and diffusion operations, the generated enhanced feature map is X. enhanced The feature diffusion mechanism is implemented through the following formula:

[0053] X enhanced =Upsample(X)·Conv(X);

[0054] Where Upsample(X) represents scale upsampling, Conv represents convolution operation, and Fusion represents feature fusion operation.

[0055] Further, step S300 specifically includes: inputting the training set into the constructed road defect detection model; using the backbone network to extract features from the input UAV images; using the neck network to enhance and fuse the features extracted by the backbone network; using the head network to output predicted bounding boxes based on the fused features; calculating the loss between the predicted bounding boxes and the ground truth bounding boxes in the dataset according to the MPDIoU loss function; updating the model parameters using the stochastic gradient descent algorithm; using the test set to evaluate the performance of the road defect detection model after each round of training; and adjusting the parameters according to the evaluation results until the predetermined number of training rounds is reached or the model converges, ultimately obtaining the trained detection model.

[0056] Furthermore, the formula for calculating the Wise-MPDIoU loss function is as follows:

[0057]

[0058] in, These represent the top-left and bottom-right corners of the predicted bounding box and the ground truth bounding box, respectively. This indicates that the distance between corresponding points is calculated, and w and h represent the width and height of the bounding box.

[0059] This invention provides a road defect detection system based on UAV optical remote sensing imagery. The system has program modules corresponding to the steps described above, and executes the steps in the above-described road defect detection method based on UAV optical remote sensing imagery during operation.

[0060] Compared with the prior art, the beneficial effects of the present invention are:

[0061] This invention utilizes the CSP-SMAM module, enabling the model to efficiently extract multi-scale features, particularly enhancing its perception of targets at different scales in complex scenes. Combined with the CAFPN structure, the model further improves feature fusion and localization accuracy. The Wise-MPDIoU loss function optimizes the target localization process, making it particularly suitable for small target detection. Finally, the CA attention mechanism strengthens the model's localization capability through direction and position awareness information, improving recognition accuracy in complex backgrounds. This invention is applicable to large-scale automated road defect detection, enabling real-time detection on resource-constrained equipment, and possesses significant practical application value. Attached Figure Description

[0062] Figure 1 This is a flowchart of a road defect detection method based on UAV optical remote sensing imagery in an embodiment of the present invention;

[0063] Figure 2 This is a structural diagram of the selective multi-scale aggregation module CSP-SMAM in an embodiment of the present invention;

[0064] Figure 3 This is a structural diagram of the feature extraction strategy FDPN in an embodiment of the present invention;

[0065] Figure 4 This is a structural diagram of the feature focusing and feature diffusion mechanism CAFPN in an embodiment of the present invention;

[0066] Figure 5 This is a diagram illustrating the effect of road defect detection in an embodiment of the present invention. Detailed Implementation

[0067] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0068] Specific Implementation Plan 1: Combining Figures 1 to 4 As shown, this invention provides a method for detecting road defects based on UAV optical remote sensing imagery, comprising the following steps:

[0069] S100. High-resolution road defect images are collected using UAV equipment to construct a raw database; considering the specific characteristics of road defect detection, the raw data undergoes multi-stage processing, including:

[0070] First, remove blurry images caused by equipment shaking, uneven lighting, or extreme weather; remove samples with excessive mixing between the target and background that cannot be clearly labeled; exclude images of atypical road scenes (such as construction areas or foreign objects obstructing the view) to ensure that the data is consistent with the actual detection scene.

[0071] Secondly, the LabelImg annotation tool is used to perform high-precision bounding box annotation on defects such as cracks and pits. In view of the characteristics of small target size and dense distribution in UAV imagery, an overlapping slicing strategy (such as 640×640 pixel slices with an overlap rate of 30%) is adopted to preserve the integrity of defects and avoid loss of slice edge information.

[0072] Finally, considering the characteristics of road defects, multi-scale rotation (±30°), random occlusion simulation (simulating fallen leaves and water stains), brightness jitter (±20%), and HSV color gamut perturbation were adopted to improve the robustness of the model to complex lighting and occlusion scenarios.

[0073] S200. Construct a YOLOv8-MSCA network. The detection model of this network structure includes a Backbone, Neck, and Head connected in sequence. A CSP-SMAM module is used, and a partial multi-scale feature extraction strategy FDPN is introduced to efficiently extract feature information at different scales in some channels, improving the ability to perceive information at different scales during detection. A coordinate attention mechanism is designed to encode the feature map in both horizontal and vertical directions, capturing short-range and long-range dependency information, thereby improving the accurate localization of the target area. A feature fusion mechanism combined with a channel attention module is used to improve the understanding of image details and enhance the model's ability to focus on key information. The neck network is designed with a CAFPN structure, comprehensively considering feature focusing and diffusion mechanisms, enabling multi-scale disease features to be effectively aggregated and transmitted to the detection stage.

[0074] The backbone network includes a first Conv module, a second Conv module, a first CSP-SMAM module, a third Conv module, a second CSP-SMAM module, a fourth Conv module, a third CSP-SMAM module, a fifth Conv module, a fourth CSP-SMAM module, an SPPF module, and a C_Attention module connected in sequence.

[0075] The neck network includes a first FDPN module, a first Upsample module, a first concat module, a fifth CSP-SMAM module, a second concat module, and a sixth CSP-SMAM module connected in sequence; simultaneously, the first FDPN module, the sixth Conv module, the third concat module, the seventh CSP-SMAM module, the fourth concat module, and the eighth CSP-SMAM module are connected; the second CSP-SMAM module is connected to the first FDPN module, and the C_Attention module is connected to the first FDPN module; the first FDPN module is connected to the second FDPN module, and the second FDPN module, the second Upsample module, and the second concat module are connected. Then, the second FDPN module, the sixth Conv module, and the fourth concat module are connected; the third concat module is connected to the C_Attention module; the first FDPN module is connected to the CAttention module; the first FDPN module is connected to the third CSP-SMAM module; the first FDPN module is connected to the second CSP-SMAM module; the first concat module and the second CSP-SMAM module; the first Upsample module is connected to the second concat module; the sixth Conv module is connected to the fourth concat module; the fifth CSP-SMAM module is connected to the second FDPN module; and the seventh CSP-SMAM module is connected to the second FDPN module.

[0076] The head network includes a first Detect module, a second Detect module, and a third Detect module; the first Detect module is connected to the sixth CSP-SMAM module, the second Detect module is connected to the second FDPN module, and the third Detect module is connected to the eighth CSP-SMAM module; each Detect module further extracts features and performs target detection through convolution operations;

[0077] In the CSP-SMAM module, a partial multi-scale feature extraction strategy is adopted. Through parallel convolution operations, feature information of different scales is efficiently extracted from some channels, thereby enhancing the model's ability to detect road defects, especially its adaptability to targets of different sizes. Specifically, this includes:

[0078] Input feature map F in The feature map F is of size H×W×C. First, a 3×3 convolution operation (Conv3×3) is used to extract features from all channels, resulting in the feature map F. 3×3 The calculation formula is:

[0079] F 3×3 =Conv3×3(F in )

[0080] Then, the input feature map F 3×3 The process is divided into two parts. One part undergoes further feature extraction via a 5×5 convolution operation (Conv5×5) to obtain the feature map F. 5×5 The formula is as follows:

[0081] F 5×5 =Conv5×5(F 3×3 )

[0082] The other part remains unchanged, resulting in feature map F. 3×3_remain The formula is as follows:

[0083] F 3×3_remain =F 3×3

[0084] Next, the F part after 5×5 convolution processing 5×5 It is divided into two parts again. One part is further processed by a 7×7 convolution operation (Conv7×7) to extract features, resulting in feature map F. 7×7 The calculation formula is:

[0085] F 7×7 =Conv7×7(F 5×5 )

[0086] The other part remains unchanged, resulting in feature map F. 5×5_remain2The formula is:

[0087] F 5×5_remain2 =F 5×5

[0088] The obtained F 7×7 Feature map and invariant part F 5×5_remain2 and F 3×3_remain The feature maps are concatenated to obtain a fused multi-scale feature map F. concat The formula is:

[0089] F concat =[F 3×3_remain ,F 5×5_remain2 ,F 7×7 ]

[0090] Then, the spliced ​​feature map F concat Perform a 1×1 convolution operation Conv1×1 to reduce the number of channels and further refine feature information to obtain the feature map F. conv1×1 The formula is:

[0091] F conv1×1 =Conv1×1(F concat )

[0092] Finally, F conv1×1 With input feature map F in Perform an addition operation (i.e., residual connection) to form the final output feature map F. out The formula is:

[0093] F out =F conv1×1 +F in

[0094] Through this design, the CSP-SMAM module can achieve multi-scale feature extraction on some channels, further enhancing the model's ability to perceive road defects, especially when dealing with defects of different sizes, it shows good adaptability.

[0095] In the CAFPN module, based on the combination of feature focusing and feature diffusion mechanisms, the model's ability to perceive targets at different scales is effectively improved by uniformly aligning multi-layer features, performing multi-scale convolutional modeling, and residual fusion. Specifically, this includes:

[0096] Using the three scale feature maps P3, P4, and P5 output from the backbone network as input, the feature maps at each scale are first processed uniformly through three lightweight downsampling modules (Adown) to obtain feature representations of the same resolution.

[0097]

[0098] The three downsampled feature maps are concatenated along the channel dimension to obtain the fused feature map F. CAFPN :

[0099]

[0100] F fused feature map F CAFPN The input is fed into the feature focusing module, where features are extracted using depthwise separable convolutions of different scales to enhance its context modeling capabilities. Specifically, this includes:

[0101]

[0102] F id =F CAFPN

[0103] Where DWConv(·) is the depthwise separable convolution operation, K 5×5 K 7×7 K 9×9 K 11×11 Convolutional kernels of sizes 5×5, 7×7, 9×9, and 11×11;

[0104] By summing and fusing the five branches pixel by pixel, a multi-scale context-aware feature map is obtained:

[0105]

[0106] Fusion Feature Map F sum Further compression and processing using standard convolution yields the convolutional output F. conv1 The formula is:

[0107] F conv1 =Conv(F sum )

[0108] Finally, a residual enhancement mechanism is introduced to improve F CAFPN The output of the convolution is summed with the output of the convolution to form the final CAFPN module output:

[0109] F out =F conv1 +F CAFPN

[0110] Through the above operations, the CAFPN module can not only fully integrate semantic information from different receptive fields, but also enhance the model's ability to express multi-scale targets, significantly improve detection accuracy, and show stronger robustness and adaptability, especially when dealing with disease targets of different types and sizes.

[0111] To improve model inference speed and reduce computational overhead, this invention introduces a lightweight downsampling module inspired by the Adown module in YOLOv9. In the CAFPN module, the lightweight downsampling module Adown first processes the feature maps at three scales output by the backbone network. Each scale feature map is downsampled and unified to the same resolution for subsequent stitching and fusion. This Adown module optimizes the computation process while preserving key features. Specifically, through global average pooling (GAP), it significantly reduces computation while effectively preserving key information in the feature maps. The module's structure is inspired by YOLOv9, and its design effectively reduces computational complexity and enhances feature expressiveness. Its structure is as follows:

[0112] For the input feature map P i Performing a global average pooling operation (AvgPool) reduces the image spatial size and computational cost, yielding the feature map F of the global average pooling operation. avgpool :

[0113] F avgpool =AvgPool(P i )

[0114] Next, the channel dimension is divided into two branches, with each branch having half the number of channels as the original. The first branch is processed by a 3×3 convolution kernel K with a stride S1 of 2. 3×3 Perform downsampling to obtain preliminary downsampling features F down1 :

[0115] F down1 =Conv(F avgpool ,K 3×3 (S1 = 2)

[0116] The second branch first uses max pooling (MaxPool) and then extracts features through convolution to obtain the second part of downsampled features F. down2 :

[0117] F down2 =Conv(MaxPool(F avgpool ),K 1×1 )

[0118] Finally, the feature maps of the two branches are merged through a channel concatenation operation to obtain the final downsampled feature F. final :

[0119] F final =[F down1 ,F down2 ]

[0120] In the CAFPN module of the neck network, a Feature Focusing Mechanism (FFM) and a Feature Diffusion Mechanism (FDM) are introduced to optimize the multi-scale feature fusion and representation process, thereby improving the accuracy and robustness of object detection. By combining feature focusing and diffusion paths, the model can efficiently process feature information from different scales and enhance its perception of local details and global context, specifically including:

[0121] The input feature map first passes through the first FDPN module, which transforms the input feature map into feature maps of different scales through multi-scale feature extraction. These feature maps capture the diversity and details in the image. Next, the first Upsample module upsamples the low-resolution feature map to retain more detail information. The subsequent first concat module concatenates the feature maps of different scales to further enrich the feature representation.

[0122] The feature focusing mechanism extracts important information from multiple scales through multiple parallel convolution operations; the input feature map is processed by the CSP-SMAM module, which adopts a partial multi-scale convolution strategy to efficiently extract features at different scales and enhances the expression of detailed features through feature focusing.

[0123] The feature diffusion mechanism is responsible for expanding the feature map from a local range to a wider area, enhancing the network's ability to integrate information across different scales. Through the second FDPN module, the network diffuses the focused features, allowing features at different scales to influence and merge with each other. The diffusion operation not only enhances the network's perception of details but also enhances the propagation of contextual information, enabling the network to integrate feature information from different scales to optimize detection results.

[0124] During the diffusion process, the feature map after feature focusing is expanded through a 7×7 convolution operation, while the feature map is propagated from one scale to other scales to enhance the network's global perception capability. The diffused feature map is then fused with feature maps of other scales to generate multi-scale feature maps with richer information. These feature maps are finally input into the detection module for target detection.

[0125] Let the feature map be After feature focusing and diffusion operations, the generated enhanced feature map is X. enhanced The feature diffusion mechanism is implemented through the following formula:

[0126] X enhanced =Fusion(Upsample(X)·Conv(X))

[0127] Here, Upsample(X) represents scale upsampling, Conv represents convolution operation, and Fusion represents feature fusion operation (such as addition, concatenation, etc.). This mechanism is performed across multiple scales, effectively improving the spatial coverage and context awareness of features. Finally, the enhanced multi-scale feature map is input into the detection module (Detect) for object detection tasks.

[0128] S300. The road defect detection model is trained using the training set, its performance is evaluated using the test set, and the model parameters are adjusted based on the evaluation results to obtain the finally trained road defect detection model; specifically including,

[0129] The training set is input into the constructed road defect detection model. The backbone network is used to extract features from the input UAV images. The neck network is used to enhance and fuse the features extracted by the backbone network. The head network is used to output predicted bounding boxes based on the fused features. The loss between the predicted bounding box and the ground truth bounding box in the dataset is calculated according to the MPDIoU loss function. The stochastic gradient descent (SGD) algorithm is used to update the model parameters. The performance of the road defect detection model after each round of training is evaluated using the test set. The parameters are adjusted according to the evaluation results until the predetermined number of training rounds is reached or the model converges, and finally the trained detection model is obtained.

[0130] The performance evaluation uses precision, recall, and mean average precision (mAP) to verify the model's detection performance, frames per second (FPS) to verify the model's detection speed, and floating-point operations to verify the model's computational complexity.

[0131] When training the road damage detection network model, the size of the input images was uniformly adjusted to 640×640×3; the SGD optimizer was used for training for a total of 300 epochs; in addition, to enhance the detection capability, Mosaic data augmentation technology was used in the last 10 training epochs; this adjustment aims to improve the robustness of the model in detecting road damage.

[0132] The main task of the head network is to further process the features from the neck network, generate the final predicted bounding box, and compare it with the ground truth bounding boxes in the dataset to optimize the model parameters. The head network transforms the feature maps extracted by the neck network into specific information required for target detection, including the target's category, location, and confidence level. To improve detection accuracy and robustness, the head network includes three Detect modules, which are used to detect target objects of different sizes.

[0133] Each Detect module internally includes convolutional layers, activation functions, prediction layers, and non-maximum suppression (NMS). The convolutional layers further process the feature maps from the neck network to extract more deep-level features. The activation functions introduce non-linearity, enhancing the expressive power of the features.

[0134] Non-maximum suppression (NMS) is used to process the final detection results, remove redundant bounding boxes, and retain the predicted box with the highest confidence. Specifically, after sorting according to confidence, NMS removes other boxes with high overlap with the current box (IoU greater than the threshold), ensuring that each target is detected only once and reducing false alarms.

[0135] In deep learning object detection tasks, loss functions measure the difference between predicted and ground truth boxes, and algorithms optimize their parameters. IoU (Intersection over Union) represents the overlap between predicted and ground truth boxes and is an important metric for object detection tasks. YOLOv8n uses the CIoU loss function, which considers the shape information and diagonal distance of the target box (i.e., the predicted box), as well as multiple factors such as position, shape, and orientation. This makes it easier for the model to capture the accurate shape of the target and improve the accuracy of target localization. However, during model training, if the center points of the predicted and ground truth boxes overlap and their aspect ratios are the same, the CIoU loss function cannot accurately express the actual relationship between the predicted and ground truth boxes.

[0136] To address the issue of inconsistent image quality across object detection datasets, the original model's CIoU loss function is replaced with the WIoU loss function. This not only considers the overlap between the predicted and ground truth bounding boxes but also incorporates the center distance and aspect ratio. The calculation formula is as follows:

[0137]

[0138] Where υ measures the aspect ratio, α represents a dynamic balance parameter that automatically adjusts the influence of the aspect ratio term υ based on the IoU of the current sample; ρ 2 B pred B gt B represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. pred B represents the predicted bounding box. gt Let c represent the true bounding box, and ω represent the normalization factor. gt and h gt ω represents the width and height of the actual bounding box. pred and h predCIoU represents the width and height of the predicted bounding box; however, CIoU is somewhat vague in its definition of aspect ratio, making it difficult to further optimize high-quality regression samples, leading to an imbalance between positive and negative samples; Wise-MPDIoU is used to replace CIoU to alleviate the problem of imbalance between positive and negative samples.

[0139] The WIoU loss function is a bounding box regression loss function used in object detectors. It combines an IoU metric and a smoothing term as the bounding box regression loss, incorporates a dynamic non-monotonic mechanism, and employs a reasonable gradient gain allocation strategy to reduce large or harmful gradients in extreme samples. WIoU addresses the potential bias problem of IoU by weighting the region between the predicted and ground truth boxes. Its calculation formula is as follows:

[0140]

[0141] Among them, b i Let g represent the predicted bounding box coordinates of the i-th target. i ω represents the coordinates of the true bounding box of the i-th object. i Indicates the weight value;

[0142] MPDIoU is an improved algorithm that directly minimizes the distance between the top-left and bottom-right corners of the predicted bounding box and the ground truth bounding box. It is more accurate in detecting and locating small targets and has better robustness. Its calculation formula is as follows:

[0143]

[0144] in, These represent the top-left and bottom-right corners of the predicted bounding box and the ground truth bounding box, respectively. This indicates the calculation of the distance between corresponding points, where w and h represent the width and height of the bounding box. This invention improves MPDIoU using the idea of ​​the WIoU loss function, replacing CIoU with Wise-MPDIoU. This helps the model better learn the accurate location of the target and improves the model's detection accuracy. The calculation formula is as follows:

[0145]

[0146] S400: Acquire road defect images captured by the UAV that need to be processed and input them into the trained YOLOv8n model to obtain classification and detection results; perform real-time slicing and resolution normalization on the road images captured by the UAV to adapt to the model input requirements; integrate the road defect detection algorithm and input / output processing module to form a complete road defect detection process; address the inconsistency in image quality and size captured by different devices, as well as the differences between these images and the algorithm's preset input, the input preprocessing module generates target image sub-images with preset sizes and groups them; simultaneously, the output processing module extracts positioning information, shooting information, etc. from the original images and labels the detected defect areas, thereby intuitively reflecting the detection results and providing data support for subsequent road maintenance and repair work.

[0147] Specific Implementation Scheme 2: The present invention provides a road defect detection system based on UAV optical remote sensing imagery. This system has a program module corresponding to the above steps, and executes the steps in the above-mentioned road defect detection method based on UAV optical remote sensing imagery when running.

[0148] The other combinations and connections in this implementation scheme are the same as in Specific Implementation Scheme 1.

[0149] Simulation Experiment

[0150] Compared to the YOLOv8n algorithm, the method of this invention significantly improves the accuracy of road defect detection. The model described in this invention is primarily developed using Python, employing the open-source deep learning framework PyTorch as the neural network training framework, and utilizing CUDA 11.8 for hardware-accelerated training. The model was trained and tested in the following hardware environment: [Processor details missing]. Core TM The system uses an i7-12700KF CPU and an NVIDIA RTX 4060Ti GPU (equipped with 16GB of VRAM). During model training, the input image size was set to 640×640 pixels, and the Stochastic Gradient Descent (SGD) algorithm was used for parameter optimization. The training epochs were set to 300, the batch size to 64, and the initial learning rate to 0.01. As shown in Table 1, a complete end-to-end processing system was formed, demonstrating significant improvements in both accuracy and efficiency. This system has important application value and provides a new method for road defect detection. Figure 5 This image shows a comparison of the target detection performance of the proposed detection method in a simulated environment. The left side of the image is the original simulated image, and the right side is the detection result after introducing the Wise-MPDIoU loss function in this invention. It can be seen that the detection accuracy of this invention is significantly improved in target edge regions and overlapping regions, and it can still maintain a good bounding box fitting effect, especially for targets with partial occlusion or pose changes.

[0151] Table 1

[0152]

[0153]

[0154] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1.A method for detecting road disease based on unmanned aerial vehicle (UAV) optical remote sensing image, characterized in that, The method comprises the following steps: S100, collecting high-resolution road disease images based on a UAV device, constructing an original database and performing preprocessing, and dividing the preprocessed data set into a training set, a validation set and a test set; S200, constructing a YOLOv8-MSCA model, including a backbone network, a neck network and a head network connected in sequence; adopting a CSP-SMAM module, introducing a partial multi-scale feature extraction strategy FDPN to extract feature information of different scales on the channel; designing a coordinate attention mechanism to encode the feature map in the horizontal and vertical directions, capturing short-range and long-range dependency information; Using a feature fusion mechanism combined with a channel attention module to improve the understanding of image details and enhance the focusing ability of the model on key information; the neck network is designed with a CAFPN structure, considering the feature focusing and diffusion mechanism, so that the multi-scale disease features are aggregated and transmitted to the detection stage; In the CSP-SMAM module, including, Input feature map F in with size HxWxC, is first processed by a 3x3 convolution operation Feature extraction is performed on all channels to obtain a feature map The calculation formula is: Then, the input feature map is divided into two parts, one of which continues to extract features through a 5x5 convolution operation to obtain a feature map , as shown in the following formula: Another part remains unchanged, resulting in the feature map The formula is as follows: Next, the portion processed by 5x5 convolution Again, one portion continues to pass through the 7x7 convolution operation Feature extraction is performed to obtain a feature map The calculation formula is: Another part remains unchanged, resulting in the feature map The formula is: The obtained Feature maps and the part that remains unchanged And The feature maps are spliced to obtain a fused multi-scale feature map The formula is: Then, the spliced feature map is subjected to a 1x1 convolution operation to reduce the number of channels and further refine the feature information, to obtain a feature map , and the formula is: Finally, the is added to the input feature map to form the final output feature map , which is given by: S300, training the constructed YOLOv8-MSCA model using the training set, evaluating the performance of the model using the test set, and adjusting the model parameters according to the evaluation results, and finally obtaining a trained road disease detection model; S400, obtaining the road disease image taken by the UAV that needs to be processed, and inputting it into the trained YOLOv8-MSCA model to obtain the classification detection result. 2.The road disease detection method based on the UAV optical remote sensing image according to claim 1, characterized in that: The backbone network comprises a first Conv module, a second Conv module, a first CSP-SMAM module, a third Conv module, a second CSP-SMAM module, a fourth Conv module, a third CSP-SMAM module, a fifth Conv module, a fourth CSP-SMAM module, an SPPF module and a C_Attention module connected in sequence; The neck network comprises a first FDPN module, a first Upsample module, a first concat module, a fifth CSP-SMAM module, a second concat module and a sixth CSP-SMAM module connected in sequence; meanwhile, the first FDPN module, a sixth Conv module, a third concat module, a seventh CSP-SMAM module, a fourth concat module and an eighth CSP-SMAM module are connected; the second CSP-SMAM module is connected with the first FDPN module, and a C_Attention module is connected with the first FDPN module; the first FDPN module is connected with a second FDPN module, the second FDPN module, a second Upsample module and a second concat module are connected, and the second FDPN module, a sixth Conv module and a fourth concat module are connected; the third concat module is connected with the C_Attention module, the first FDPN module and the CAttention module are connected, the first FDPN module and the third CSP-SMAM module are connected, the first FDPN module and the second CSP-SMAM module are connected, and the first concat module and the second CSP-SMAM module are connected; the first Upsample module is connected with the second concat module, and the sixth Conv module is connected with the fourth concat module; the fifth CSP-SMAM module is connected with the second FDPN module, and the seventh CSP-SMAM module is connected with the second FDPN module; The head network comprises a first Detect module, a second Detect module and a third Detect module; the first Detect module is connected with the sixth CSP-SMAM module, the second Detect module is connected with the second FDPN module, and the third Detect module is connected with the eighth CSP-SMAM module; each Detect module extracts features through convolution operation and performs target detection. 3.The road disease detection method based on the UAV optical remote sensing image of claim 2, characterized in that: In the CAFPN module, the feature maps at three scales output by the backbone network are used. , , As input, the feature maps at different scales are first processed uniformly using three lightweight downsampling modules, Adown, to obtain feature representations with the same resolution. : The three down-sampled feature maps are spliced through a channel dimension splicing operation to obtain a fused feature map : Fuse feature maps The input is fed into the feature focusing module, where features are extracted using depthwise separable convolutions of different scales to enhance its context modeling capabilities. Specifically, this includes: wherein, (·) is a depthwise separable convolution operation, K 5×5 , K 7×7 , K 9×9 , K 11×11 is a 5x5, 7x7, 9x9, and 11x11 size kernel; The above five branches are added and fused pixel by pixel to obtain a multi-scale context perception feature map: Fused feature maps Compressed and processed by standard convolution to obtain a convolution output The formula is: Finally, a residual enhancement mechanism is introduced to combine with the convolution output to form the final CAFPN module output: 。 4.The road disease detection method based on the UAV optical remote sensing image of claim 2, characterized in that: In the CAFPN module in the neck network, a feature focusing mechanism FFM and a feature diffusion mechanism FDM are introduced, The input feature map is first subjected to the first FDPN module, which converts the input feature map into feature maps of different scales through multi-scale feature extraction; then, the first Upsample module performs up-sampling processing on the low-resolution feature map; the first concat module splices the feature maps of different scales thereafter; The feature focusing mechanism extracts information from multiple scales through multiple parallel convolution operations; the input feature map is processed through the CSP-SMAM module, which adopts a partial multi-scale convolution strategy to extract features of different scales and enhance the expression of detailed features through feature focusing; The feature diffusion mechanism diffuses the focused features through the second FDPN module, so that features of different scales can interact and fuse with each other; in the diffusion process, the feature map after feature focusing is expanded through a 7*7 convolution operation, and the feature map is propagated from one scale to other scales; the feature map after diffusion is fused with the feature map of other scales; and finally input to the detection module detect for target detection. 5.The road disease detection method based on UAV optical remote sensing image according to claim 4, characterized in that: Let the feature map be After the feature focusing and diffusion operation, the generated enhanced feature map is ; The feature diffusion mechanism is performed by the following formula: ； wherein, denotes a scale up-sampling, denotes a convolution operation, is a feature fusion operation. 6.The road disease detection method based on UAV optical remote sensing image according to claim 2, characterized in that: In step S300, specifically, the training set is input into the constructed road disease detection model, the input unmanned aerial vehicle image is subjected to feature extraction by the backbone network, the features extracted by the backbone network are enhanced and fused by the neck network, and the prediction box is output according to the fused features by the head network; the loss between the prediction box and the real box in the data set is calculated according to the MPDIoU loss function, and the model parameters are updated by using the stochastic gradient descent algorithm; the performance of the road disease detection model after each round of training is evaluated by using the test set, and the parameters are adjusted according to the evaluation result until the predetermined training round or the model converges, and finally the trained detection model is obtained. 7.The method of claim 6, wherein the method further comprises: The calculation formula of the Wise-MPDIoU loss function is where, denotes the points of the upper left corner and the lower right corner of the prediction box and the ground truth box, denotes the distance between the corresponding points, and w and h denote the width and height of the bounding box. 8.A road disease detection system based on unmanned aerial vehicle optical remote sensing images, characterized in that: The system has program modules corresponding to the steps of any one of claims 1-7, and when running, the steps of the above method for detecting road diseases based on unmanned aerial vehicle optical remote sensing images are executed.

Images