Unmanned aerial vehicle aerial photography vehicle detection method based on improved YOLO11n

Abstract

The invention discloses an unmanned aerial vehicle aerial vehicle detection method based on improved YOLO11n. The method specifically comprises the following steps: (1) obtaining an unmanned aerial vehicle aerial vehicle image data set; (2) adding annotation information to the images in the data set and dividing the data set; (3) constructing an unmanned aerial vehicle aerial photography vehicle detection model based on the improved YOLO11n; (4) training the model by adopting the training set and the verification set, and storing the trained model as an optimal model; and (5) testing the optimal model by using the test set, and obtaining the final unmanned aerial vehicle aerial photography vehicle detection model based on the improved YOLO11n if the test result meets the precision requirement. Compared with the prior art, the unmanned aerial vehicle aerial vehicle detection method based on the improved YOLO11n disclosed by the invention effectively improves the detection precision of the unmanned aerial vehicle aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method for detecting vehicles in aerial photography by an unmanned aerial vehicle based on improved YOLO11n. Background Art

[0002] With the continuous acceleration of the national urbanization process and the rapid growth of the number of motor vehicles, traditional traffic management methods are facing unprecedented challenges. In recent years, drone technology has made great progress and has gradually become one of the key components of intelligent transportation systems. Drone aerial photography has the advantages of wide field of view, large shooting range, and high flexibility. As a supplementary perspective for vehicle detection, it has broad application prospects in the field of transportation. However, vehicles photographed by drones have problems such as large scale changes and a large number of small target vehicles. Therefore, studying drone aerial vehicle detection methods is of great significance to the development of intelligent transportation systems.

[0003] Traditional vehicle detection methods mainly rely on manually designed features and classifiers, and achieve vehicle detection through several steps: feature extraction, sliding window, classification, and post-processing. Due to the limitations of manually designed features, the high computational complexity of the sliding window method, and the lack of end-to-end training, traditional vehicle detection algorithms have problems such as poor real-time performance, poor accuracy, and poor generalization in the vehicle detection process.

[0004] With the rapid development of deep learning, improving the object detection technology of convolutional neural network has become a hot research topic. Convolutional Neural Network (CNN) is a mainstream deep learning algorithm, including R-CNN, Fast-R-CNN, Faster-R-CNN, YOLO and other models. These models have achieved good results in object detection in different application scenarios. Object detection algorithms can be divided into two categories: two-stage detection algorithms using regional candidate boxes and single-stage detection algorithms without regional candidate boxes. The two-stage algorithm has high accuracy but slow detection speed. The improved algorithm has improved the speed to a certain extent, but due to the setting of candidate areas, the two-stage model cannot meet the real-time detection requirements of drone aerial photography vehicles. Therefore, the single-stage algorithm is usually used in drone aerial photography vehicle detection. Compared with the two-stage detection algorithm, the single-stage target detection algorithm has improved real-time performance, but its accuracy is lower. Especially in drone aerial photography vehicle detection, there are problems such as large changes in vehicle scale and a large number of small target vehicles, which can easily lead to missed detection and false detection. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a UAV aerial photography vehicle detection method based on improved YOLO11n. By improving the YOLO11n target detection model, the problem of low vehicle detection accuracy caused by large changes in vehicle scale and a large number of small target vehicles is solved.

[0006] The technical solution provided by the present invention comprises the following steps: Step 1: Obtain drone aerial images of vehicles to form a first data set; the images in the first data set can be taken by a drone camera or collected from the Internet; Step 2: adding annotation information to the images in the first data set to form a second data set, and dividing the second data set into a training set, a validation set, and a test set; Preferably, the second data set is divided into a training set, a validation set and a test set in a ratio of 8:1:1; Step 3: Construct a drone aerial vehicle detection model based on improved YOLO11n, the model includes a Backbone network, a Neck network and a Head network, and the construction of the model further includes steps 3.1 to 3.4: Step 3.1: The Backbone network consists of 5 Conv modules, 4 C3k2 modules, one SPPF module and one C2PSA module; the training set and the validation set in the second data set are used as inputs of the Backbone network; Step 3.2: The Neck network consists of 2 upsample modules, 4 Concat modules, 4 C3k2 modules, 2 Conv modules, 1 SPD-Conv module and 1 CSP-OmniKernel module to form a new feature pyramid named SOEFPN; The SPD-Conv module processes the features output by the P2 feature layer to obtain features containing small target information, and then passes them to the P3 feature layer for feature fusion; the SPD-Conv module consists of an SPD layer and a non-stride convolution layer. The SPD layer is first used to convert the spatial dimension of the feature map into a depth dimension, and then the non-stride convolution layer is used to maintain the spatial dimension and reduce the number of channels; In the CSP-OmniKernel module, OmniKernel is a fully kernel-shaped image restoration network, which consists of three branches: global branch, large branch and local branch. It can reasonably expand the convolution kernel to the feature size and effectively learn the feature representation from global to local, thereby improving the detection performance of small targets. The introduction of the CSP idea divides the input features into two parts: one part passes through the OmniKernel module to generate adaptive features; the other part retains the input features as a lightweight path to reduce computational overhead. Step 3.3: The Head network includes three Dyhead-DCNv4 detection heads, and the outputs of the three C3k2 modules in the neck network are respectively used as inputs of the three Dyhead-DCNv4 detection heads; The Dyhead-DCNv4 detection head replaces the deformable convolution DCNv2 with DCNv4 based on the Dyhead detection head; compared with Dyhead, Dyhead-DCNv4 adds a dynamic weight generation module weight, which uses offset and weight to perform convolution operations in the convolution, thereby improving detection performance and reducing computational overhead; Step 3.4: Replace the loss function in the original model with the Inner-SIoU function. The optimized loss function formula is: ; In formula (1), L SIoU Represents the loss of SIoU function; IoU inner represents the Inner-IoU loss function; IoU represents the intersection-over-union ratio between the predicted box and the true box; L SIoU The calculation formula is: ; In formula (2), Δ represents distance loss and Ω represents shape loss; In formula (1), IoU inner The calculation formula is: ; In formula (3), inter represents the area of ​​the intersection of the predicted box and the true box; union represents the area of ​​the union of the predicted box and the true box; the calculation formulas are: ; ; In formula (4) and formula (5), and b r Represents the right boundary horizontal coordinates of the real box and the predicted box respectively; and b l Represents the left boundary horizontal coordinates of the real box and the predicted box respectively; and b b Represents the lower boundary ordinates of the real box and the predicted box respectively; and b t Represent the upper boundary ordinates of the real box and the predicted box respectively; and Represent the width and height of the real box respectively; w and h represent the width and height of the predicted box respectively; ratio corresponds to the scaling factor; Step 4: Use the training set and the validation set to train the drone aerial vehicle detection model based on the improved YOLO11n, and save the trained model as the optimal model; Furthermore, the step 4 specifically includes steps 4.1 to 4.4: Step 4.1: Setting the training parameters of the drone aerial vehicle detection model based on improved YOLO11n; Specifically, the training parameters include: Epoch is 200, batch size is 16, optimizer is SGD, initial learning rate 1r0 is 0.01, momentum is 0.937, weight decay weight_decay is 0.0005, and number of workers is 4; Step 4.2: Input the training set and validation set and the corresponding labels into the improved YOLO11n drone aerial vehicle detection model, use the back propagation algorithm to calculate the gradient of the loss function to the model parameters, and adjust the model parameters by minimizing the loss function to gradually approach the optimal solution; Step 4.3: Use the optimizer SGD to update the model parameters in the direction of gradient descent until the loss function of the training set and the validation set no longer decreases, and the evaluation indicators mean average precision mAP, recall rate R, and accuracy P no longer increase; Step 4.4: Save the trained model parameters as the optimal model; Step 5: Use the test set to test the optimal model, evaluate the test set test results, and meet the accuracy requirements, so as to obtain the final drone aerial vehicle detection model based on improved YOLO11n; Furthermore, the step 5 specifically includes steps 5.1 to 5.3: Step 5.1: Input the test set into the optimal model; Step 5.2: Calculate the model performance indicators: accuracy P, recall R, mean average precision mAP, number of parameters, and model size. The specific calculation formula is as follows: ; ; ; ; Where P is the precision, R is the recall, mAP is the average precision of all categories, AP is the average precision, m is the total number of vehicle categories, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, and FN is the number of positive samples incorrectly identified as negative samples; Step 5.3: When the performance indicators meet the accuracy requirements, the final UAV aerial vehicle detection model based on improved YOLO11n is obtained.

[0007] Compared with the prior art, the present invention has the following beneficial effects: The present invention discloses a UAV aerial photography vehicle detection method based on improved YOLO11n, constructs a new feature pyramid SOEFPN, improves the accuracy of small target vehicle detection under the perspective of UAV aerial photography; adopts a new detection head Dyhead-DCNv4, improves the detection accuracy of vehicles with large scale changes; adopts a new loss function Inner-SIoU, avoids the large accuracy loss caused by small target vehicles, and accelerates the convergence of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a flow chart of the drone aerial vehicle detection method based on improved YOLO11n of the present invention; Figure 2 This is a schematic diagram of the structure of the drone aerial vehicle detection model based on the improved YOLO11n of the present invention; Figure 3 Schematic diagram of the SOEFPN feature pyramid structure; Figure 4 This is the flow chart of the CSP-OmniKernel module; Figure 5 This is a schematic diagram of the OmniKernel module structure; Figure 6 This is a schematic diagram of the Dyhead-DCNv4 structure; DETAILED DESCRIPTION

[0009] In order to make the technical solution, structural features, achieved purposes and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific implementation methods and accompanying drawings. It should be noted that the specific embodiments described herein are only used to more clearly explain the present invention and are not used to limit the present invention.

[0010] Figure 1This is a flow chart of a drone aerial vehicle detection method based on improved YOLO11n disclosed in the present invention, and its implementation process is as follows: Step 1: Obtain drone aerial vehicle images to form a first data set; the drone aerial vehicle images in the first data set can be taken by a drone camera or collected from the Internet; In this embodiment, in order to better evaluate the detection effect of the drone aerial vehicle detection method based on improved YOLO11n disclosed in the present invention, the public data set VisDrone2019 was used for drone aerial vehicle detection, and five common road vehicles, including cars, vans, trucks, buses and motorcycles, were selected for detection to form the first data set.

[0011] Step 2: Add annotation information to the images in the first dataset. Since the public dataset VisDrone2019 used in this embodiment already has annotation information, this step is skipped.

[0012] The label file in the first data set described in this embodiment is converted into a txt format label file required by YOLO11n to form a second data set, and the second data set is divided into a training set, a validation set and a test set in a ratio of 8:1:1; the divided training set includes 6903 pictures, the test set includes 863 pictures, and the validation set includes 863 pictures. In this embodiment, the image size of the second data set is 640×640.

[0013] Step 3: Build a drone aerial vehicle detection model based on improved YOLO11n. The model includes Backbone network, Neck network and Head network. The structure is as follows: Figure 2 As shown, the model building process specifically includes steps 3.1 to 3.4: Step 3.1: The Backbone network consists of 5 Conv modules, 4 C3k2 modules, one SPPF module and one C2PSA module; the training set and the validation set in the second data set are used as inputs of the Backbone network; Step 3.2: The Neck network consists of 2 upsample modules, 4 Concat modules, 4 C3k2 modules, 2 Conv modules, 1 SPD-Conv module and 1 CSP-OmniKernel module to form a new feature pyramid named SOEFPN with the following structure: Figure 3 As shown; The SPD-Conv module processes the features output by the P2 feature layer to obtain features containing small target information, and then passes them to the P3 feature layer for feature fusion; the SPD-Conv module consists of an SPD layer and a non-stride convolution layer. The SPD layer is first used to convert the spatial dimension of the feature map into a depth dimension, and then the non-stride convolution layer is used to maintain the spatial dimension and reduce the number of channels; The CSP-OmniKernel module structure is as follows Figure 4 As shown in the figure, OmniKernel is a fully kernel-shaped image restoration network consisting of three branches: global branch, large branch and local branch. It can reasonably expand the convolution kernel to the feature size and effectively learn the feature representation from global to local, thereby improving the detection performance of small targets. The structure is shown in Figure 5 As shown in the figure, the introduction of CSP idea divides the input features into two parts: one part passes through the OmniKernel module to generate adaptive features; the other part retains the input features as a lightweight path to reduce computational overhead; Step 3.3: The Head network includes three Dyhead-DCNv4 detection heads, and the outputs of the three C3k2 modules in the neck network are respectively used as inputs of the three Dyhead-DCNv4 detection heads; The Dyhead-DCNv4 detection head replaces the deformable convolution DCNv2 with DCNv4 based on the Dyhead detection head. Compared with Dyhead, Dyhead-DCNv4 adds a dynamic weight generation module weight, which uses offset and weight to perform convolution operations at the same time, thereby improving detection performance and reducing computational overhead. The structure is as follows: Figure 6 As shown; Step 3.4: Replace the loss function in the original model with the Inner-SIoU function. The optimized loss function formula is: ; In formula (1), L SIoU Represents the loss of SIoU function; IoU inner represents the Inner-IoU loss function; IoU represents the intersection-over-union ratio between the predicted box and the true box; L SIoU The calculation formula is: ; In formula (2), Δ represents distance loss and Ω represents shape loss; In formula (1), IoU inner The calculation formula is: ; In formula (3), inter represents the area of ​​the intersection of the predicted box and the true box; union represents the area of ​​the union of the predicted box and the true box; the calculation formulas are: ; ; In formula (4) and formula (5), and b r Represents the right boundary horizontal coordinates of the real box and the predicted box respectively; and b l Represents the left boundary horizontal coordinates of the real box and the predicted box respectively; and b b Represents the lower boundary ordinates of the real box and the predicted box respectively; and b t Represent the upper boundary ordinates of the real box and the predicted box respectively; and Represent the width and height of the real box respectively; w and h represent the width and height of the predicted box respectively; ratio corresponds to the scaling factor; Step 4: Use the training set and the validation set to train the drone aerial vehicle detection model based on the improved YOLO11n, and save the trained model as the optimal model; the drone aerial vehicle detection model training process based on the improved YOLO11n further includes steps 4.1 to 4.4: Step 4.1: Setting the training parameters of the drone aerial vehicle detection model based on improved YOLO11n; In this embodiment, the training parameters include: the iteration round Epoch is 200, the batch size batchsize is 16, the optimizer optimizer is SGD, the initial learning rate 1r0 is 0.01, the momentum momentum is 0.937, the weight decay weight_decay is 0.0005, and the number of workers is 4; Step 4.2: Input the training set and validation set and the corresponding labels into the improved YOLO11n drone aerial vehicle detection model, use the back propagation algorithm to calculate the gradient of the loss function to the model parameters, and adjust the model parameters by minimizing the loss function to gradually approach the optimal solution; Step 4.3: Use the optimizer SGD to update the model parameters in the direction of gradient descent until the loss function of the training set and the validation set no longer decreases, and the evaluation indicators mean average precision mAP, recall rate R, and accuracy P no longer increase; Step 4.4: Save the trained model parameters as the optimal model; Step 5: Use the test set to test the optimal model described in step 4, evaluate the test set test results, and meet the accuracy requirements, that is, obtain the final drone vehicle real-time detection model based on improved YOLO11n. Specifically, step 5 further includes steps 5.1 to 5.3: Step 5.1: Input the test set into the optimal model described in step 4; Step 5.2: Calculate model performance indicators: The performance indicators include accuracy P, recall R, mean average precision mAP, number of parameters, and model size. The specific calculation formula is as follows: ; ; ; ; Where P is the precision, R is the recall, mAP is the average precision of all categories, AP is the average precision, m is the total number of vehicle categories, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, and FN is the number of positive samples incorrectly identified as negative samples; Step 5.3: When the performance index meets the accuracy requirements, the final drone vehicle detection model based on the improved YOLO11n is obtained.

[0014] In this embodiment, in order to verify the effect of the improved model disclosed in the present invention, the present invention uses the YOLOv8n model, the YOLOv10n model, the YOLO11n model, the YOLO11s model and the detection model disclosed in this patent to test on the VisDrone2019 data set. The evaluation index data is shown in Table 1: ; As can be seen from Table 1, the drone aerial vehicle detection model disclosed in the present invention has improved accuracy P, recall R, mAP50 and other indicators compared with the original YOLO11n model, and is better than other YOLO models with the same parameter level, and even better than the YOLO11s model with a higher parameter level. It meets the real-time requirements and can be deployed on the drone platform.

[0015] The above description is only one embodiment of the present invention and does not limit the patent scope of the present invention. For those skilled in the art, the present invention may have various changes and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A drone aerial vehicle detection method based on improved YOLO11n, characterized in that: The specific steps include: Step 1: Obtain drone aerial images of vehicles to form a first data set; the images in the first data set can be taken by a drone camera or collected from the Internet; Step 2: adding annotation information to the images in the first data set to form a second data set, and dividing the second data set into a training set, a validation set, and a test set; Step 3: Construct a drone aerial vehicle detection model based on improved YOLO11n, the model includes a Backbone network, a Neck network and a Head network, and the construction of the model further includes steps 3.1 to 3.4: Step 3.1: The Backbone network consists of 5 Conv modules, 4 C3k2 modules, 1 SPPF module and 1 C2PSA module; the training set and the validation set in the second data set are used as the input of the Backbone network; Step 3.2: The Neck network consists of 2 upsample modules, 4 Concat modules, 4 C3k2 modules, 2 Conv modules, 1 SPD-Conv module and 1 CSP-OmniKernel module to form a new feature pyramid named SOEFPN; The SPD-Conv module processes the features output by the P2 feature layer to obtain features containing small target information, and then passes them to the P3 feature layer for feature fusion; the SPD-Conv module consists of an SPD layer and a non-stride convolution layer. The SPD layer is first used to convert the spatial dimension of the feature map into a depth dimension, and then the non-stride convolution layer is used to maintain the spatial dimension and reduce the number of channels; In the CSP-OmniKernel module, OmniKernel is a fully kernel-shaped image restoration network, which consists of three branches: global branch, large branch and local branch. It can reasonably expand the convolution kernel to the feature size and effectively learn feature representation from global to local. The introduction of CSP idea divides the input features into two parts: one part passes through the OmniKernel module to generate adaptive features; the other part retains the input features as a lightweight path to reduce computational overhead. Step 3.3: The Head network includes three Dyhead-DCNv4 detection heads, and the outputs of the three C3k2 modules in the neck network are respectively used as inputs of the three Dyhead-DCNv4 detection heads; The Dyhead-DCNv4 detection head replaces the deformable convolution DCNv2 with DCNv4 based on the Dyhead detection head; compared with Dyhead, Dyhead-DCNv4 adds a dynamic weight generation module weight, which uses offset and weight to perform convolution operations in the convolution, thereby improving detection performance and reducing computational overhead; Step 3.4: Replace the loss function in the original model with the Inner-SIoU function. The optimized loss function formula is: ； In formula (1), L SIoU Represents the loss of SIoU function; IoU inner represents the Inner-IoU loss function; IoU represents the intersection-over-union ratio between the predicted box and the true box; L SIoU The calculation formula is: ； In formula (2), Δ represents distance loss and Ω represents shape loss; In formula (1), IoU inner The calculation formula is: ； In formula (3), inter represents the area of ​​the intersection of the predicted box and the true box; union represents the area of ​​the union of the predicted box and the true box; The calculation formulas are: ； ； In formula (4) and formula (5), and b r Represents the right boundary horizontal coordinates of the real box and the predicted box respectively; and b l Represents the left boundary horizontal coordinates of the real box and the predicted box respectively; and b b Represents the lower boundary ordinates of the real box and the predicted box respectively; and b t Represent the upper boundary ordinates of the real box and the predicted box respectively; and Represent the width and height of the real box respectively; w and h represent the width and height of the predicted box respectively; ratio corresponds to the scaling factor; Step 4: Use the training set and the validation set to train the drone aerial vehicle detection model based on the improved YOLO11n, and save the trained model as the optimal model; the drone aerial vehicle detection model training process based on the improved YOLO11n further includes steps 4.1 to 4.4: Step 4.1: Setting the training parameters of the drone aerial vehicle detection model based on improved YOLO11n; Model training parameters include: number of iterations, batch size, optimizer, learning rate, momentum, weight decay, and number of threads; Step 4.2: Input the training set and validation set and the corresponding labels into the improved YOLO11n drone aerial vehicle detection model, use the back propagation algorithm to calculate the gradient of the loss function to the model parameters, and adjust the model parameters by minimizing the loss function to gradually approach the optimal solution; Step 4.3: Use the optimizer SGD to update the model parameters in the direction of gradient descent until the loss function of the training set and the validation set no longer decreases, and the evaluation indicators mean average precision mAP, recall rate R, and accuracy P no longer increase; Step 4.4: Save the trained model parameters as the optimal model; Step 5: Use the test set to test the optimal model, evaluate the test set test results, and meet the accuracy requirements, so as to obtain the final drone aerial vehicle detection model based on the improved YOLO11n.

2. The method for detecting vehicles in drone aerial photography based on improved YOLO11n according to claim 1, characterized in that: In step 3.2, a feature pyramid SOEFPN for small target vehicles is designed; first, the P2 feature layer is processed by SPD-Conv to obtain features containing small target information, and then the features are passed to the P3 feature layer for fusion, and then CSP-OmniKernel is used for feature integration to effectively learn feature representation from global to local, thereby improving the detection performance of small target vehicles.

3. The method for detecting vehicles in drone aerial photography based on improved YOLO11n according to claim 1, characterized in that: The step 5 further includes steps 5.1 to 5.3: Step 5.1: Input the test set into the optimal model; Step 5.2: Calculate the model performance indicators: accuracy P, recall R, mean average precision mAP, number of parameters, model size. The specific calculation formula is as follows: ； ； ； ； Where P is the precision, R is the recall, mAP is the average precision of all categories, AP is the average precision, m is the total number of vehicle categories, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, and FN is the number of positive samples incorrectly identified as negative samples; Step 5.3: When the performance indicators meet the accuracy requirements, the final UAV aerial vehicle detection model based on the improved YOLO11n is obtained.

Images