Unmanned aerial vehicle road disease detection and analysis system and method based on YOLO algorithm

Abstract

The invention discloses an unmanned aerial vehicle road disease detection and analysis system and method based on a YOLO algorithm, and the method comprises the following steps: synchronously collecting a road image and geographic coordinates through an unmanned aerial vehicle, and constructing an image data set with geographic reference; constructing an initial YOLO target detection model based on the data set; introducing an additional detection layer for detecting a pixel area into the model, and embedding a channel attention mechanism into the feature extraction network to obtain an improved YOLO target detection model; reasoning a data set by using the model, and outputting a disease category, a bounding box coordinate and a confidence coefficient; pixel coordinates are converted into geographic coordinates through a coordinate conversion algorithm, and disease records with accurate geographic coordinates are generated; and optimizing and updating the model based on the record as an incremental training sample. According to the method, the disease detection precision and the small target identification capability are effectively improved, and the long-term adaptability and generalization performance of the model are enhanced through a closed-loop incremental learning mechanism.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and unmanned aerial vehicle remote sensing application, and particularly relates to an unmanned aerial vehicle road disease detection and analysis system and method based on a YOLO algorithm. BACKGROUND

[0002] Roads, as important infrastructure, their health directly affects the safety and efficiency of transportation. Traditional manual inspection methods have inherent defects such as low efficiency, high risk, limited coverage, etc., and are difficult to meet the large-scale, high-frequency, and fine needs of modern road maintenance management. With the development of unmanned aerial vehicle technology and computer vision technology, using unmanned aerial vehicle aerial images for automatic road disease detection has become the mainstream technical direction.

[0003] In the development process of road disease detection technology based on unmanned aerial vehicle images, the key technical route mainly focuses on image acquisition, target recognition and positioning analysis. In terms of image acquisition, through the unmanned aerial vehicle platform equipped with high-resolution visible light cameras, image data of large-area road surfaces can be efficiently acquired, and geographic reference information of the images can be recorded with the help of the global navigation satellite system. In terms of target recognition, deep learning models, especially target detection algorithms based on convolutional neural networks, are widely used due to their superior feature learning and recognition capabilities. Among them, one-stage detection algorithms represented by the YOLO series have become the mainstream model architecture in this field due to their good balance between detection speed and accuracy.

[0004] However, the existing road disease detection schemes based on general target detection frameworks such as YOLO still face several technical bottlenecks in practical applications. The general detection model has insufficient feature extraction and fusion capabilities for road diseases, which are significantly different in scale and shape and have complex backgrounds, leading to missed detection and false detection, especially for small-sized disease recognition accuracy. Existing methods are mostly limited to outputting two-dimensional image coordinates and category information of diseases, lacking the ability to quantify the actual physical size of diseases, while pixel area is a key indicator for evaluating the severity of diseases.

[0005] Therefore, how to provide a YOLO algorithm-based unmanned aerial vehicle road disease detection and analysis system and method with high detection accuracy, disease area quantification and precise geographic positioning capabilities, and model self-iterative optimization is a problem that needs to be solved by those skilled in the art. SUMMARY

[0006] One purpose of the present application is to propose a UAV road disease detection and analysis system and method based on YOLO algorithm, aiming at the technical problems of insufficient detection accuracy of road diseases, lack of disease area quantification, inaccurate geographic positioning, and lack of continuous self-optimization ability of the model in the prior art, a closed-loop technical solution is proposed, which includes synchronous acquisition of image and geographic information, construction of improved YOLO model with fusion pixel area detection and channel attention mechanism, accurate conversion of geographic coordinates, and incremental updating of the model using the detection results. The present application has the beneficial effects of improving the detection sensitivity and accuracy of small target diseases, realizing direct quantitative output of disease pixel area, achieving centimeter-level geographic positioning accuracy through multi-coordinate system conversion, and continuously improving the adaptability and generalization ability of the model through an automatic incremental learning mechanism.

[0007] The UAV road disease detection and analysis system and method based on YOLO algorithm according to the embodiments of the present application include the following steps: S1. Collecting images of the road surface by a UAV, and synchronously recording the geographic coordinate information at each frame of image acquisition time to form an image dataset with geographic reference; S2. Building an initial target detection model based on the image dataset with geographic reference, with YOLO model as the architecture; S3. Based on the initial target detection model, introducing an additional detection layer for detecting pixel area, and embedding a channel attention mechanism in the feature extraction network of the initial target detection model to obtain an optimized improved YOLO target detection model; S4. Inputting the image dataset with geographic reference into the improved YOLO target detection model for inference, and outputting recognition results containing disease category labels, bounding box coordinates, and confidence scores; S5. Converting the pixel coordinates in the recognition results into coordinates in the geographic coordinate system through a coordinate transformation algorithm, and performing correlation calculation with the geographic coordinate information of the corresponding frame of image to generate road disease target records with accurate geographic coordinates; S6. Based on the road disease target records with accurate geographic coordinates, as incremental training samples, optimizing and updating the parameters of the improved YOLO target detection model.

[0008] Optionally, the S1 specifically includes: S1.1 Configuring a UAV flight platform, and collecting digital images of the road surface at fixed time intervals through a high-resolution visible light camera during flight; S1.2 Synchronously recording the longitude coordinate, latitude coordinate, and elevation coordinate of the digital image acquisition time while collecting the digital image; S1.3 timestamp matching the digital images with corresponding longitude coordinates, latitude coordinates and elevation coordinates to form a georeferenced image dataset.

[0009] Optionally, the S2 specifically comprises: S2.1 extracting all digital images based on the georeferenced image dataset to form an original training image set; S2.2 preprocessing each digital image in the original training image set to obtain a preprocessed image set; S2.3 selecting a YOLOv5 network architecture as the benchmark structure of the initial target detection model, the YOLOv5 network architecture comprising an input end, a backbone network, a neck network and a detection head; S2.2 configuring the backbone network of the YOLOv5 network architecture as a CSPDarknet structure for extracting multi-scale features from the preprocessed image set; S2.3 configuring the neck network of the YOLOv5 network architecture as a path aggregation network structure for realizing multi-level feature fusion of the feature pyramid; S2.4 configuring the detection head of the YOLOv5 network architecture as three detection layer structures of different scales for detecting large, medium and small sized targets respectively S2.5 obtaining the initial target detection model with the YOLO model as the architecture.

[0010] Optionally, the S3 specifically comprises: S3.1 adding an additional detection layer for calculating pixel area after the three different scale detection layer structures of the detection head of the YOLOv5 network architecture; the additional detection layer comprising a convolution layer, a batch normalization layer and an activation function layer; S3.2 concatenating the output feature maps of the three different scale detection layer structures with the output feature map of the additional detection layer in the channel dimension to form a fusion feature map containing pixel area information; S3.3 calculating the pixel area of each detected target by pixel value summation operation based on the fusion feature map of the additional detection layer; S3.4 embedding a channel attention mechanism module after each cross-stage local network module of the backbone network CSPDarknet structure; the channel attention mechanism module performing global average pooling on the fusion feature map to generate a channel statistic vector; S3.5 sequentially performing a first fully connected layer transformation, an activation function processing and a second fully connected layer transformation on the channel statistic vector; S3.6 mapping the output of the second fully connected layer transformation through an activation function to generate a channel attention weight vector; S3.7 The channel attention weight vector is multiplied with the original feature map output by the cross-stage local network module to obtain a channel attention weighted feature map, thereby obtaining the improved YOLO target detection model.

[0011] Optionally, the S4 specifically includes: S4.1 Inputting the digital images in the geographic reference image dataset into the improved YOLO target detection model, and performing forward propagation calculation by the improved YOLO target detection model; S4.2 The improved YOLO target detection model outputs a bounding box prediction parameter set at each position of the three different scale detection layer structures, and the bounding box prediction parameter set includes a horizontal center point coordinate, a vertical center point coordinate, a bounding box width, and a bounding box height; S4.3 The improved YOLO target detection model outputs a class prediction probability value set at each position of the three different scale detection layer structures; S4.4 The improved YOLO target detection model outputs a pixel area value corresponding to each predicted bounding box through the fusion feature map of the additional detection layer and pixel value summation operation; S4.5 For each bounding box prediction parameter set, a corresponding confidence score is calculated, and the confidence score is the product of a target existence probability value and a highest class prediction probability value; S4.6 Based on a preset confidence threshold, a bounding box prediction parameter set with a confidence score greater than the confidence threshold is screened to form an initial recognition result set; S4.7 A non-maximum suppression algorithm is applied to the initial recognition result set to eliminate overlapping redundant bounding boxes, thereby obtaining a final recognition result, and the recognition result includes a disease class label, a bounding box coordinate, a pixel area, and a confidence score.

[0012] Optionally, the S5 specifically includes: S5.1 Based on the longitude coordinate, the latitude coordinate, and the elevation coordinate of the digital image acquisition time recorded in the geographic reference image dataset, the geographic coordinates of the center point of each digital image corresponding to the unmanned aerial vehicle image are obtained; S5.2 The bounding box coordinates of each road disease target are extracted from the final recognition result, and the bounding box coordinates are represented in a pixel coordinate system with the upper left corner of the image as the origin; S5.3 The pixel center point position of each disease target in the digital image is calculated according to the bounding box coordinates of each disease target; S5.4 The conversion relationship between the pixel coordinate system of the digital image and the image plane coordinate system with the image center as the origin is established according to the imaging parameters of the high-resolution visible light camera, and the pixel center point position of the disease target is converted into the image plane coordinate. S5.5 Based on the height data recorded during the flight of the unmanned aerial vehicle and the camera attitude angle data, a projection relationship between the image plane coordinate system and the local ground coordinate system centered on the unmanned aerial vehicle is established, the image plane coordinates of the disease target are projected into the local ground coordinate system, and the planar position of the disease target in the local ground coordinate system is obtained; S5.6 The spatial conversion relationship between the local ground coordinate system and the geodetic coordinate system is established by using the geographic coordinates of the center point of the unmanned aerial vehicle image and the attitude data of the unmanned aerial vehicle, and the planar position of the disease target in the local ground coordinate system is converted into the longitude coordinate, latitude coordinate and elevation coordinate in the geodetic coordinate system; S5.7 The disease class label, bounding box coordinates, pixel area, confidence score of each disease target are associated with the longitude coordinate, latitude coordinate and elevation coordinate in the geodetic coordinate system, and the road disease target record with accurate geographic coordinates is generated.

[0013] Optionally, the S6 specifically comprises: S6.1 Extract the disease class label, bounding box coordinates, pixel area, confidence score, longitude coordinate, latitude coordinate and elevation coordinate from the road disease target record with accurate geographic coordinates to form an incremental training sample data set; S6.2 Label the bounding box coordinates, disease class label and pixel area of each disease target in the incremental training sample data set to the corresponding original digital image to form a labeled training image; S6.3 Perform data enhancement processing on the labeled training image, and the data enhancement processing includes random horizontal flip, random rotation and random brightness adjustment; S6.4 Merge the labeled training image after data enhancement processing and the original training image set to form an updated training image set; S6.5 Use the updated training image set to train the improved YOLO target detection model for a new round, and the training uses the back propagation algorithm to optimize the model parameters; S6.6 In the training process, calculate the loss function value between the model prediction result and the labeled result, and the loss function includes the bounding box loss, the class prediction loss and the pixel area loss; S6.7 When the number of training iterations reaches the preset number, stop training, save the optimized model parameters, and complete the parameter optimization and update of the improved YOLO target detection model.

[0014] Optionally, the following modules are included: An unmanned aerial vehicle image acquisition device is configured to acquire digital images of the road surface and simultaneously record the longitude coordinate, latitude coordinate and elevation coordinate at each image acquisition time; The data preprocessing device is configured to receive digital images and geographic coordinate information output by the unmanned aerial vehicle image acquisition device, perform timestamp matching, and form an image dataset with geographic reference. The model training device is configured to construct an initial target detection model with a YOLO model as an architecture based on the image dataset with geographic reference, introduce an additional detection layer for detecting pixel area in the initial target detection model, and embed a channel attention mechanism in a feature extraction network, thereby obtaining an improved YOLO target detection model. The target detection inference device is configured to input the image dataset with geographic reference into the improved YOLO target detection model and output an identification result containing a disease category label, a bounding box coordinate, and a confidence score. The geographic coordinate conversion device is configured to convert pixel coordinates in the identification result into coordinates in a geographic coordinate system and perform associated calculation with geographic coordinate information of a corresponding frame image to generate a road disease target record with accurate geographic coordinates. The model incremental updating device is configured to construct an incremental training sample dataset based on the road disease target record with accurate geographic coordinates and perform parameter optimization and updating on the improved YOLO target detection model.

[0015] The present application has the following advantages: The present application introduces an additional detection layer for detecting pixel area, realizes direct quantitative output of the physical size of the disease target, effectively improves the objectivity and refinement level of disease severity assessment, embeds a channel attention mechanism in the feature extraction network of the initial target detection model, enhances the feature extraction and focusing ability of the multi-scale disease target in a complex background, and significantly improves the detection accuracy and robustness of small-sized diseases. The present application establishes a multi-level coordinate conversion algorithm from pixel coordinates to image plane coordinates, local ground coordinates, and finally to the geodetic coordinate system, realizes cm-level accurate positioning of the disease target, and greatly improves the direct applicability of the detection result in subsequent maintenance and repair operations. The present application continuously optimizes the model by using the detection result with accurate geographic coordinates as an incremental training sample, constructs a technical closed loop from data acquisition, intelligent identification to model self-updating, and effectively enhances the long-term adaptability and generalization ability of the system in different road environments and disease forms. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, illustrate the application, and are used to explain the application without limiting the application. In the drawings:

[0017] Fig. 1A flowchart of the UAV road disease detection and analysis system and method based on the YOLO algorithm according to the present application is shown in the figure. Fig. 2 A technical architecture diagram of the core model optimization step in the UAV road disease detection and analysis system and method based on the YOLO algorithm according to the present application is shown in the figure. DETAILED DESCRIPTION

[0018] In combination Figs. 1-2 The present application will be further described in detail. These drawings are simplified schematic diagrams and only schematically show the basic structure of the present application, showing the main components related to the present application. According to Fig. 1 and Fig. 2 the UAV road disease detection and analysis system and method based on the YOLO algorithm according to the present application:

[0019] S1. Collect images of the road surface by a UAV, and simultaneously record the time of each frame image collection and the geographic coordinate information to form a geographic reference image dataset; S2. Based on the geographic reference image dataset, an initial target detection model with a YOLO model as the architecture is constructed; S3. Based on the initial target detection model, an additional detection layer for detecting pixel area is introduced, and a channel attention mechanism is embedded in the feature extraction network of the initial target detection model to obtain an improved YOLO target detection model; S4. Based on the improved YOLO target detection model, the geographic reference image dataset is input for inference to output recognition results containing disease category labels, bounding box coordinates, and confidence scores; S5. The pixel coordinates in the recognition results are converted into coordinates in the geographic coordinate system by a coordinate transformation algorithm, and associated calculations are performed with the geographic coordinate information of the corresponding frame image to generate road disease target records with accurate geographic coordinates; S6. Based on the road disease target records with accurate geographic coordinates as incremental training samples, the improved YOLO target detection model is parameter-optimized and updated.

[0020] In this embodiment, the S1 specifically includes: S1.1 Configure a UAV flight platform, and collect digital images of the road surface at fixed time intervals by a high-resolution visible light camera during flight; S1.2 Simultaneously record the longitude coordinates, latitude coordinates, and elevation coordinates of the time of digital image collection while collecting the digital images; S1.3 Time stamp match the digital images with the corresponding longitude coordinates, latitude coordinates, and elevation coordinates to form a geographic reference image dataset.

[0021] In this embodiment, S2 specifically includes: S2.1 Based on the georeferenced image dataset, extract all digital images to form the original training image set; S2.2 Preprocess each digital image in the original training image set to obtain the preprocessed image set; S2.3 Select the YOLOv5 network architecture as the baseline structure for the initial object detection model. The YOLOv5 network architecture includes an input end, a backbone network, a neck network, and a detection head. S2.2 Configure the backbone network of the YOLOv5 network architecture as the CSPDarknet structure, which is used to extract multi-scale features from the preprocessed image set; S2.3 Configure the neck network of the YOLOv5 network architecture as a path aggregation network structure to achieve multi-level feature fusion of the feature pyramid; S2.4 Configures the YOLOv5 network architecture with a detection header consisting of three detection layers at different scales, used for detecting large, medium, and small targets respectively. S2.5 yields an initial object detection model based on the YOLO model architecture.

[0022] In this embodiment, S3 specifically includes: S3.1 After the three different scale detection layer structures in the detection head of the YOLOv5 network architecture, an additional detection layer for calculating pixel area is added; the additional detection layer includes a convolutional layer, a batch normalization layer, and an activation function layer; S3.2 The output feature maps of the three detection layer structures at different scales are concatenated with the output feature map of the additional detection layer in the channel dimension to form a fused feature map containing pixel area information; S3.3 Based on the fusion feature map of the additional detection layer, the pixel area of ​​each detected target is calculated by summing the pixel values. S3.4 After each cross-stage local network module of the backbone network CSPDarknet structure, a channel attention mechanism module is embedded; the channel attention mechanism module performs global average pooling on the fused feature map to generate channel statistical vectors; S3.5 Perform the first fully connected layer transformation, activation function processing, and second fully connected layer transformation sequentially on the channel statistical vector; S3.6 The output of the second fully connected layer transformation is mapped through an activation function to generate a channel attention weight vector; S3.7 Multiply the channel attention weight vector with the original feature map output by the cross-stage local network module channel by channel to obtain the channel attention weighted feature map, thus obtaining the improved YOLO object detection model.

[0023] In this embodiment, S4 specifically includes: S4.1 Input digital images from the georeferenced image dataset into the improved YOLO object detection model, and the improved YOLO object detection model performs forward propagation computation; S4.2 The improved YOLO target detection model outputs a set of bounding box prediction parameters at each position of the three different scale detection layer structures. The set of bounding box prediction parameters includes the x-coordinate of the center point, the y-coordinate of the center point, the width of the bounding box, and the height of the bounding box. S4.3 Improve the YOLO object detection model by outputting a set of class prediction probability values ​​at each position of the detection layer structure at three different scales; S4.4 The improved YOLO object detection model outputs the pixel area value corresponding to each predicted bounding box by fusing the feature map of the additional detection layer and performing pixel-wise value summation. S4.5 For each bounding box prediction parameter set, calculate the corresponding confidence score, where the confidence score is the product of the target existence probability value and the highest category prediction probability value; S4.6 Based on a preset confidence threshold, filter the set of bounding box prediction parameters with confidence scores greater than the confidence threshold to form an initial recognition result set; S4.7 Apply the non-maximum suppression algorithm to the initial recognition result set to eliminate overlapping redundant bounding boxes and obtain the final recognition result, which includes disease category label, bounding box coordinates, pixel area and confidence score.

[0024] In this embodiment, S5 specifically includes: S5.1 Based on the longitude, latitude, and elevation coordinates of the digital image acquisition time recorded in the image dataset with geographic reference, obtain the geographic coordinates of the UAV image center point corresponding to each digital image; S5.2 From the final recognition results, extract the bounding box coordinates of each road defect target. The bounding box coordinates are represented by the pixel coordinate system with the upper left corner of the image as the origin. S5.3 Calculate the pixel center position of each disease target in the digital image based on the bounding box coordinates of each disease target; S5.4 Based on the imaging parameters of the high-resolution visible light camera, establish the transformation relationship between the pixel coordinate system of the digital image and the image plane coordinate system with the image center as the origin, and convert the pixel center point position of the disease target into image plane coordinates. S5.5 Based on the flight altitude data and camera attitude angle data recorded during UAV flight, establish the projection relationship between the image plane coordinate system and the ground local coordinate system centered on the UAV, project the image plane coordinates of the disease target onto the ground local coordinate system, and obtain the planar position of the disease target in the ground local coordinate system. S5.6 Using the geographic coordinates of the center point of the UAV image and the UAV attitude data, establish the spatial transformation relationship between the local ground coordinate system and the geodetic coordinate system, and convert the planar position of the disease target in the local ground coordinate system into the longitude coordinates, latitude coordinates and elevation coordinates in the geodetic coordinate system. S5.7 associates the disease category label, bounding box coordinates, pixel area, confidence score of each disease target with the longitude, latitude and elevation coordinates in the geodetic coordinate system to generate a road disease target record with accurate geographic coordinates.

[0025] In this embodiment, S6 specifically includes: S6.1 Extract the disease category label, bounding box coordinates, pixel area, confidence score, longitude coordinates, latitude coordinates and elevation coordinates from the road disease target records with precise geographic coordinates to form an incremental training sample dataset; S6.2 Label the bounding box coordinates, disease category label, and pixel area of ​​each disease target in the incremental training sample dataset onto the corresponding original digital image to form the labeled training image; S6.3 Perform data augmentation processing on the labeled training images, the data augmentation processing including random horizontal flipping, random rotation and random brightness adjustment; S6.4 Merge the data-augmented labeled training images with the original training image set to form an updated training image set; S6.5 Using the updated training image set, a new round of training is performed on the improved YOLO object detection model, wherein the training uses the backpropagation algorithm to optimize the model parameters; S6.6 During training, the loss function value between the model prediction result and the annotation result is calculated. The loss function includes bounding box loss, category prediction loss and pixel region area loss. S6.7 When the number of training iterations reaches the preset number, stop training, save the optimized model parameters, and complete the parameter optimization and update of the improved YOLO object detection model.

[0026] This embodiment includes the following modules: The drone image acquisition device is configured to acquire digital images of the road surface and simultaneously record the longitude, latitude, and elevation coordinates at the moment of acquisition for each frame of the image. The data preprocessing device is configured to receive digital images and geographic coordinate information output by the UAV image acquisition device, perform timestamp matching, and form an image dataset with geographic reference. The model training device is configured to construct an initial object detection model based on the georeferenced image dataset, and to introduce an additional detection layer for detecting pixel area into the initial object detection model, and to embed a channel attention mechanism in the feature extraction network, thereby obtaining an improved YOLO object detection model. The target detection inference device is configured to input the georeferenced image dataset into the improved YOLO target detection model and output a recognition result including disease category labels, bounding box coordinates, and confidence scores. The geographic coordinate transformation device is configured to convert the pixel coordinates in the recognition result into coordinates in the geographic coordinate system, and perform correlation calculation with the geographic coordinate information of the corresponding frame image to generate a road defect target record with accurate geographic coordinates. The model incremental update device is configured to construct an incremental training sample dataset based on the road disease target records with accurate geographic coordinates, and to optimize and update the parameters of the improved YOLO target detection model.

[0027] Example 1: To verify the feasibility of this invention in practice, a section of urban main road was selected as the application scenario. This section is approximately 10 kilometers long and exhibits various typical road surface defects, including cracks, potholes, network cracks, repairs, and ruts. Traffic volume on this section is high, making manual inspection risky and difficult to cover the entire road segment. Traditional detection methods rely on vehicle-mounted or handheld devices to collect images segment by segment, followed by manual visual inspection or identification based on general object detection algorithms. This approach suffers from low efficiency, high missed detection rate for small defects, poor positioning accuracy, and inability to quantify the defect area. This invention aims to achieve automated, high-precision, and quantifiable full-coverage detection and analysis of road defects using an unmanned aerial vehicle (UAV) platform and an improved YOLO algorithm.

[0028] During implementation, an unmanned aerial vehicle (UAV) flight platform equipped with a high-resolution visible light camera and a global navigation satellite system (GNSS) receiver module was used. The UAV flew along a predetermined route at a fixed altitude and speed, while the camera automatically captured digital images of the road surface at set time intervals. At the same time as each image exposure, its precise timestamp and corresponding longitude, latitude, and elevation coordinates were recorded. After the flight mission, each digital image was precisely matched with its geographic coordinates at the time of acquisition using the timestamp, forming a georeferenced image dataset. More than 5,000 valid images were acquired in total.

[0029] Based on this dataset, an initial object detection model was constructed. Specifically, all digital images were extracted from the dataset as the original training image set, and each image underwent preprocessing such as normalization, resizing, and color normalization. The YOLOv5 network architecture was selected as the baseline, with its backbone network configured as a CSPDarknet structure, responsible for extracting multi-scale deep features from the preprocessed images; the neck network was configured as a path aggregation network structure, used to achieve multi-level fusion of feature pyramids; and the detection head was configured with three detection layers of different scales, responsible for detecting large, medium, and small-sized disease targets, respectively.

[0030] Key improvements were made to the initial model. An additional independent detection layer for calculating pixel area was added after the detection head. This layer consists of a convolutional layer, a batch normalization layer, and an activation function layer in sequence. The feature maps output from the original three-scale detection layers were concatenated with the feature map output from this additional detection layer along the channel dimension to form a fused feature map. Based on this fused feature map, the area of ​​each identified lesion in the image can be directly calculated by summing pixel values. A channel attention mechanism module was embedded after each cross-stage local network module of the backbone network CSPDarknet. This module performs global average pooling on the input feature map to generate channel statistical vectors, which are then sequentially transformed through a first fully connected layer, processed by an activation function, and transformed through a second fully connected layer. Channel attention weight vectors are generated through activation function mapping. This weight vector is multiplied channel-wise with the original feature map to obtain a channel attention-weighted feature map, thereby enhancing the model's ability to focus on key lesion features. This completed the construction of the improved YOLO target detection model.

[0031] The improved model was deployed on a ground processing workstation. A georeferenced image dataset was input into the model for inference. The model performed forward propagation, outputting a set of bounding box prediction parameters and a set of class prediction probabilities at three scale detection layers, and outputting the pixel area value corresponding to each predicted bounding box through an additional detection layer. A confidence score was calculated for each bounding box, which is the product of the target presence probability and the highest predicted class probability. A confidence threshold was set, and all predicted boxes above the threshold were selected to form an initial recognition result set. Non-maximum suppression was applied to this set to eliminate overlapping redundant bounding boxes, obtaining the final recognition result containing disease category labels, bounding box coordinates, pixel area, and confidence score.

[0032] After acquiring the recognition results, a high-precision geographic coordinate transformation is performed. Based on the geographic coordinate information recorded in each image, the geodetic coordinates of its corresponding image center point are obtained. The bounding box coordinates of each road hazard target are extracted from the recognition results, and its pixel center point in the image is calculated. Based on the camera's intrinsic parameters and distortion coefficients, a transformation relationship between the pixel coordinate system and the image plane coordinate system is established, converting the pixel center point into distortion-free image plane coordinates. Then, combined with the flight altitude and attitude angle data recorded during UAV flight, a projection model from the image plane coordinate system to a local ground coordinate system centered on the UAV is established, projecting the image coordinates onto the ground to obtain the local planar location of the road hazard. Finally, using the geodetic coordinates of the image center point and the UAV attitude data, through spatial rotation and translation transformations, the local planar coordinates are converted into longitude, latitude, and elevation in the global geodetic coordinate system. The category, bounding box, area, and confidence information of each road hazard target are associated with these three-dimensional geographic coordinates, generating a road hazard target record with precise geographic coordinates.

[0033] To further verify system performance and achieve model self-optimization, all disease records with geographic coordinates generated in this detection were used as incremental training samples. The disease bounding boxes, categories, and area information from these samples were back-annotated onto their corresponding original digital images, forming labeled images. These labeled images underwent random horizontal flipping, rotation, and brightness adjustment for data augmentation, and were then merged with the initial set of original training images to form a larger updated training image set. This dataset was used to train the improved YOLO object detection model anew. During training, the backpropagation algorithm was used to optimize parameters, and the loss function comprehensively considered bounding box regression loss, category prediction loss, and pixel area prediction loss. After a set number of iterations, the model's performance on unseen test data was further improved, achieving closed-loop parameter optimization and updating.

[0034] To quantitatively evaluate the technical effectiveness of this invention, the detection performance of the traditional general-purpose YOLOv5 model (Model A) and the improved YOLO model of this invention (Model B) were compared on the same road section. The test set contained 1000 images covering various types of defects. Evaluation metrics included mean accuracy, mean accuracy for small-target defects, mean pixel area prediction error, geolocation horizontal error, and mean processing time per image. Specific test data comparisons are shown in the table below:

[0035] Table 1. Model Performance Comparison Test Results

[0036] Analysis of the data in the table above shows that the improved YOLO model (Model B) significantly outperforms the traditional general model (Model A) in all core performance indicators. In terms of overall recognition accuracy, Model B's average accuracy remained stable between 88.7% and 92.1% across five samples, a significant improvement compared to Model A's 76.8% to 82.1%. This indicates that the introduction of the channel attention mechanism and pixel area auxiliary features effectively enhances the model's ability to identify defects in complex road scenarios. For the most difficult-to-detect small-target defects, Model B achieved an average accuracy of 83.5% to 88.1%, a particularly significant improvement compared to Model A's 62.1% to 70.5%, demonstrating the improved model's stronger ability to capture and recognize small-sized features such as minute cracks. Regarding area quantization accuracy, Model B's pixel area prediction error was only 35 to 46 pixels, far lower than Model A's 138 to 167 pixels, indicating that the additional detection layer's direct regression prediction of defect area is extremely effective, providing accurate data support for defect severity classification. In terms of geolocation accuracy, Model B controls the horizontal positioning error to within 0.10 to 0.15 meters, achieving centimeter-level accuracy, while Model A's error is between 0.78 and 0.91 meters. This invention achieves sub-meter or even centimeter-level precise positioning through a rigorous coordinate transformation chain, which is crucial for guiding precise maintenance operations. Regarding processing efficiency, Model B's single-image processing time is slightly increased, remaining at 36 to 40 milliseconds, still meeting real-time requirements. This is in exchange for a significant leap in the aforementioned accuracy indicators, resulting in substantial overall benefits.

[0037] This embodiment, through a complete implementation process and detailed comparative data, fully verifies that the system and method provided by this invention have higher recognition accuracy in automated road defect detection, especially in the ability to detect small defects, accurate defect area quantification, centimeter-level geolocation capability, and the feasibility and significant technical advantages of continuous performance optimization through incremental learning.

Claims

1. A UAV road defect detection and analysis system and method based on the YOLO algorithm, characterized in that, Includes the following steps: S1. Collect images of the road surface using drones and simultaneously record the acquisition time and geographic coordinates of each image frame to form a geographically referenced image dataset; S2. Based on a georeferenced image dataset, construct an initial object detection model with the YOLO model as its architecture; S3. Based on the initial object detection model, an additional detection layer for detecting pixel area is introduced, and a channel attention mechanism is embedded in the feature extraction network of the initial object detection model to obtain the optimized and improved YOLO object detection model; S4. Based on the improved YOLO object detection model, input a georeferenced image dataset for inference, and output the recognition results including disease category labels, bounding box coordinates and confidence scores; S5. The pixel coordinates in the recognition results are converted into coordinates in the geographic coordinate system through a coordinate transformation algorithm, and then correlated with the geographic coordinate information of the corresponding frame image to generate a road defect target record with accurate geographic coordinates. S6. Based on road damage target records with accurate geographic coordinates, the parameters of the improved YOLO target detection model are optimized and updated as incremental training samples.

2. The UAV road defect detection and analysis system and method based on the YOLO algorithm according to claim 1, characterized in that, S1 specifically includes: S1.1 is equipped with an unmanned aerial vehicle (UAV) flight platform that uses a high-resolution visible light camera to collect digital images of the road surface at fixed time intervals during flight; S1.2 While acquiring digital images, simultaneously record the longitude, latitude, and elevation coordinates at the moment of digital image acquisition; S1.3 Timestamp matching is performed between the digital images and their corresponding longitude, latitude, and elevation coordinates to form a georeferenced image dataset.

3. The UAV road defect detection and analysis system and method based on the YOLO algorithm according to claim 1, characterized in that, S2 specifically includes: S2.1 Based on the georeferenced image dataset, extract all digital images to form the original training image set; S2.2 Preprocess each digital image in the original training image set to obtain the preprocessed image set; S2.3 Select the YOLOv5 network architecture as the baseline structure for the initial object detection model. The YOLOv5 network architecture includes an input end, a backbone network, a neck network, and a detection head. S2.2 Configure the backbone network of the YOLOv5 network architecture as the CSPDarknet structure, which is used to extract multi-scale features from the preprocessed image set; S2.3 Configure the neck network of the YOLOv5 network architecture as a path aggregation network structure to achieve multi-level feature fusion of the feature pyramid; S2.4 Configures the YOLOv5 network architecture with a detection header consisting of three detection layers at different scales, used for detecting large, medium, and small targets respectively. S2.5 yields an initial object detection model based on the YOLO model architecture.

4. The UAV road defect detection and analysis system and method based on the YOLO algorithm according to claim 1, characterized in that, S3 specifically includes: S3.1 After the three different scale detection layer structures in the detection head of the YOLOv5 network architecture, an additional detection layer for calculating pixel area is added; the additional detection layer includes a convolutional layer, a batch normalization layer, and an activation function layer; S3.2 The output feature maps of the three detection layer structures at different scales are concatenated with the output feature map of the additional detection layer in the channel dimension to form a fused feature map containing pixel area information; S3.3 Based on the fusion feature map of the additional detection layer, the pixel area of ​​each detected target is calculated by summing the pixel values. S3.4 After each cross-stage local network module of the backbone network CSPDarknet structure, a channel attention mechanism module is embedded; the channel attention mechanism module performs global average pooling on the fused feature map to generate channel statistical vectors; S3.5 Perform the first fully connected layer transformation, activation function processing, and second fully connected layer transformation sequentially on the channel statistical vector; S3.6 The output of the second fully connected layer transformation is mapped through an activation function to generate a channel attention weight vector; S3.7 Multiply the channel attention weight vector with the original feature map output by the cross-stage local network module channel by channel to obtain the channel attention weighted feature map, thus obtaining the improved YOLO object detection model.

5. The UAV road defect detection and analysis system and method based on the YOLO algorithm according to claim 1, characterized in that, S4 specifically includes: S4.1 Input digital images from the georeferenced image dataset into the improved YOLO object detection model, and the improved YOLO object detection model performs forward propagation computation; S4.2 The improved YOLO target detection model outputs a set of bounding box prediction parameters at each position of the three different scale detection layer structures. The set of bounding box prediction parameters includes the x-coordinate of the center point, the y-coordinate of the center point, the width of the bounding box, and the height of the bounding box. S4.3 Improve the YOLO object detection model by outputting a set of class prediction probability values ​​at each position of the detection layer structure at three different scales; S4.4 The improved YOLO object detection model outputs the pixel area value corresponding to each predicted bounding box by fusing the feature map of the additional detection layer and performing pixel-wise value summation. S4.5 For each bounding box prediction parameter set, calculate the corresponding confidence score, where the confidence score is the product of the target existence probability value and the highest category prediction probability value; S4.6 Based on a preset confidence threshold, filter the set of bounding box prediction parameters with confidence scores greater than the confidence threshold to form an initial recognition result set; S4.7 Apply the non-maximum suppression algorithm to the initial recognition result set to eliminate overlapping redundant bounding boxes and obtain the final recognition result, which includes disease category label, bounding box coordinates, pixel area and confidence score.

6. The UAV road defect detection and analysis system and method based on the YOLO algorithm according to claim 1, characterized in that, S5 specifically includes: S5.1 Based on the longitude, latitude, and elevation coordinates of the digital image acquisition time recorded in the image dataset with geographic reference, obtain the geographic coordinates of the UAV image center point corresponding to each digital image; S5.2 From the final recognition results, extract the bounding box coordinates of each road defect target. The bounding box coordinates are represented by the pixel coordinate system with the upper left corner of the image as the origin. S5.3 Calculate the pixel center position of each disease target in the digital image based on the bounding box coordinates of each disease target; S5.4 Based on the imaging parameters of the high-resolution visible light camera, establish the transformation relationship between the pixel coordinate system of the digital image and the image plane coordinate system with the image center as the origin, and convert the pixel center point position of the disease target into image plane coordinates. S5.5 Based on the flight altitude data and camera attitude angle data recorded during UAV flight, establish the projection relationship between the image plane coordinate system and the ground local coordinate system centered on the UAV, project the image plane coordinates of the disease target onto the ground local coordinate system, and obtain the planar position of the disease target in the ground local coordinate system. S5.6 Using the geographic coordinates of the center point of the UAV image and the UAV attitude data, establish the spatial transformation relationship between the local ground coordinate system and the geodetic coordinate system, and convert the planar position of the disease target in the local ground coordinate system into the longitude coordinates, latitude coordinates and elevation coordinates in the geodetic coordinate system. S5.7 associates the disease category label, bounding box coordinates, pixel area, confidence score of each disease target with the longitude, latitude and elevation coordinates in the geodetic coordinate system to generate a road disease target record with accurate geographic coordinates.

7. The UAV road defect detection and analysis system and method based on the YOLO algorithm according to claim 1, characterized in that, S6 specifically includes: S6.1 Extract the disease category label, bounding box coordinates, pixel area, confidence score, longitude coordinates, latitude coordinates and elevation coordinates from the road disease target records with precise geographic coordinates to form an incremental training sample dataset; S6.2 Label the bounding box coordinates, disease category label, and pixel area of ​​each disease target in the incremental training sample dataset onto the corresponding original digital image to form the labeled training image; S6.3 Perform data augmentation processing on the labeled training images, the data augmentation processing including random horizontal flipping, random rotation and random brightness adjustment; S6.4 Merge the data-augmented labeled training images with the original training image set to form an updated training image set; S6.5 Using the updated training image set, a new round of training is performed on the improved YOLO object detection model, wherein the training uses the backpropagation algorithm to optimize the model parameters; S6.6 During training, the loss function value between the model prediction result and the annotation result is calculated. The loss function includes bounding box loss, category prediction loss and pixel region area loss. S6.7 When the number of training iterations reaches the preset number, stop training, save the optimized model parameters, and complete the parameter optimization and update of the improved YOLO object detection model.

8. The UAV road defect detection and analysis system and method based on the YOLO algorithm according to claim 1, characterized in that, Includes the following modules: The drone image acquisition device is configured to acquire digital images of the road surface and simultaneously record the longitude, latitude, and elevation coordinates at the moment of acquisition for each frame of the image. The data preprocessing device is configured to receive digital images and geographic coordinate information output by the UAV image acquisition device, perform timestamp matching, and form an image dataset with geographic reference. The model training device is configured to construct an initial object detection model based on the georeferenced image dataset, and to introduce an additional detection layer for detecting pixel area into the initial object detection model, and to embed a channel attention mechanism in the feature extraction network, thereby obtaining an improved YOLO object detection model. The target detection inference device is configured to input the georeferenced image dataset into the improved YOLO target detection model and output a recognition result including disease category labels, bounding box coordinates, and confidence scores. The geographic coordinate transformation device is configured to convert the pixel coordinates in the recognition result into coordinates in the geographic coordinate system, and perform correlation calculation with the geographic coordinate information of the corresponding frame image to generate a road defect target record with accurate geographic coordinates. The model incremental update device is configured to construct an incremental training sample dataset based on the road disease target records with accurate geographic coordinates, and to optimize and update the parameters of the improved YOLO target detection model.

Images