Unmanned aerial vehicle aerial target detection method based on improved YOLOv7
By improving the YOLOv7 model, adding a small target detection layer, using the BiFPN structure and WIoU loss function, adding DyHead, and combining data augmentation methods, the problem of high proportion of small targets and large scale variations in UAV aerial images was solved, achieving high-precision and efficient target detection.
Patent Information
- Application Number
- CN202311052927.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-21
- Publication Date
- 2025-12-30
- Estimated Expiration
- 2043-08-21
AI Technical Summary
The high proportion of small targets, large variations in target scale, and complex backgrounds in drone aerial images result in low target detection accuracy and difficulty in meeting real-time requirements.
The YOLOv7 model was improved by adding a small object detection layer, using a BiFPN structure and WIoU loss function, adding an attention-based detection head DyHead, and combining Mosaic, Mixup methods and adaptive anchor box calculation to optimize the training process and improve detection accuracy.
Without increasing computational load, it significantly improves the accuracy and speed of target detection in UAV aerial photography, adapts to small targets and scale variations, and enhances the model's expressive power.
Smart Images
Figure CN117011728B_ABST
Abstract
Description
Technical fields:
[0001] This invention belongs to the field of deep learning and target detection, specifically a drone aerial target detection method based on improved YOLOv7, which can be used for drones to identify ground targets. Background technology:
[0002] Unmanned aerial vehicles (UAVs) are a wide range of aviation equipment with compact size, flexible maneuverability, and low operating costs. They play an irreplaceable role in many fields, including plant and animal protection, urban traffic management, agricultural monitoring, and military security. Accurate and effective target identification of aerial images captured by UAVs enhances their intelligence level and facilitates various automated operations, such as automatic target tracking and flight path planning. Therefore, researching UAV aerial target detection algorithms has significant practical implications.
[0003] Compared with natural scenes, drone aerial images have the following characteristics: (1) Complex background. Thanks to the altitude of the drone, drone aerial images contain rich object features, which can easily cause interference from similar targets and also cause objects to occlude each other. (2) High proportion of small targets. Due to the phenomenon of "near objects appearing larger and far objects appearing smaller", the same object occupies different pixel sizes in drone images. At the same time, the proportion of small targets is relatively high, which can easily cause missed detections or false detections. (3) Large changes in target scale. During the flight of the drone, the altitude changes constantly, and the target scale changes drastically.
[0004] Deep learning technology is one of the key solutions to the challenge of target detection in drone aerial photography. Research in the field of deep learning target detection can be basically divided into two directions: one is a two-stage detector based on candidate regions, such as Faster-RCNN, and the other is a single-target detector based on regression calculation, such as YOLO. Each direction has its own characteristics; generally, two-stage detectors have higher accuracy, while single-stage detectors are faster. Considering the real-time requirements of target detection in drone aerial images, and the limited endurance of drones, the high-efficiency YOLO algorithm is more suitable. This study selects the YOLOv7 algorithm, optimizes it for the characteristics of drone aerial target detection, improves recognition accuracy, and maintains a lightweight network model and high inference speed. Summary of the Invention:
[0005] The purpose of this invention is to propose a drone aerial target detection method based on an improved YOLOv7, which aims to enhance the expression of semantic features of small targets, improve the target detection accuracy of drone aerial images, and ensure that drones can successfully complete various tasks.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] The drone aerial target detection method based on the improved YOLOv7 includes the following steps:
[0008] (1) Divide the VisDrone drone aerial photography dataset into training set, validation set and test set in a ratio of 8:1:1, and the division is done randomly.
[0009] (2) Improve the YOLOv7 model. The YOLOv7 model is divided into three parts: Backbone, Neck and Head. Optimize the Neck and Head parts.
[0010] (3) Preprocess the training set image data, input the preprocessed images into the feature extraction network, and extract feature maps at different scales;
[0011] (4) Use the feature maps of each scale obtained in (3) to train the improved YOLOv7 model. During the training process, Precision, Recall and Mean Average Precision (mAP) are used as indicators to evaluate the model performance. The validation set data is used to monitor whether the model is overfitting until the loss curve converges.
[0012] (5) Input the test set data into the weight file of the improved YOLOv7 model obtained from training to obtain the actual recognition results of the target in the drone aerial image.
[0013] Further technology of the present invention:
[0014] Preferably, step (2) is specifically performed as follows:
[0015] We selected the YOLOv7-tiny model with SiLU as the activation function as the basis for improvement;
[0016] Add a small target detection layer: After the feature map is extracted by the Backbone of the YOLOv7-tiny network, the resolution is continuously reduced, and a large number of small target features and positional information are gradually lost in this process. The original YOLOv7-tiny network introduces three feature maps of different scales into the Neck part for fusion and forms three prediction heads in the Head part; a feature map with a resolution of 160*160 is also introduced into the network for feature fusion and forms a fourth prediction head to deal with small targets.
[0017] Using BiFPN structure: YOLOv7-tiny uses PANet for feature fusion. The BiFPN structure of the Neck layer upsamples the input features and fuses them with the features extracted by the Backbone. The FPN structure adds edges with context information and multiplies each edge by a corresponding weight to generate four new feature layers. The detection results are output based on these feature layers.
[0018] Replace CIoU with WIoU: YOLOv7-tiny uses the CIoU loss function to perform regression prediction on bounding boxes, while WIoU replaces IoU with outlier evaluation to assess the quality of the anchor boxes through a dynamic non-monotonic focusing mechanism;
[0019] Add an attention-based detection head, DyHead: DyHead is a dynamic detection head framework that combines scale awareness, spatial awareness, and task awareness in object detection into a single framework by incorporating an attention mechanism.
[0020] Preferably, step (3) is specifically performed as follows:
[0021] The image preprocessing process includes the Mosaic method, the Mixup method, adaptive anchor box calculation, and adaptive image scaling. After the above preprocessing steps, the training set image data is sent to the input of the improved YOLOv7 network.
[0022] Preferably, step (4) is specifically performed as follows:
[0023] When training the improved YOLOv7-tiny network model using feature maps of different scales obtained in step (3), the parameters of the neural network are updated through forward propagation and backward propagation. In the forward propagation process, the approximate location of the target is determined based on the fused features, and the predicted box is obtained by filtering with a pre-set threshold. In the backward propagation process, the error between the real box and the predicted box is calculated through the loss function, and the parameters of the neural network are optimized by the gradient descent method. At the same time, the mAP index of the validation set is observed to determine that no overfitting has occurred. When the loss curve tends to converge, the training of the model is completed.
[0024] Preferably, step (5) is specifically performed as follows:
[0025] Input the test set images into the trained model weight file, and the model will outline and label the detected targets on the images to verify the model's actual detection performance and inference speed.
[0026] The beneficial effects of this invention are as follows:
[0027] This invention provides a target detection method for UAV aerial images based on an improved YOLOv7, which improves the YOLOv7-tiny model using SiLU as the activation function. It adds a small target detection layer to effectively address the high proportion of small targets in UAV aerial images; uses a Bi FPN structure to fuse more features; replaces CIoU with WIoU to further improve detection performance; and adds an attention-based detection head, DyHead, to enhance the expressive power of the model's target detection head without increasing computational cost. Compared to the original YOLOv7-tiny network, the improved model does not significantly increase in size, but achieves a significant improvement in accuracy. Attached image description:
[0028] Figure 1 The overall structure diagram of the improved YOLOv7 network designed for embodiments of the present invention;
[0029] Figure 2 A comparison diagram of PANet and BiFPN structures;
[0030] Figure 3 This is a structural diagram of the DyHead attention module used in an embodiment of the present invention;
[0031] Figure 4 This is a comparison chart of the improved model and the original model mAP in an embodiment of the present invention;
[0032] Figure 5 The image shows the actual target detection effect of the improved model of the present invention when photographed by a drone. Detailed implementation method:
[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0034] This example provides a drone aerial target detection method based on an improved YOLOv7, including:
[0035] (1) The VisDrone drone aerial photography dataset proposed by the Machine Vision and Data Mining Laboratory of Tianjin University was divided into training set, validation set and test set in a ratio of 8:1:1, and the division was carried out in a random manner.
[0036] (2) Improve the YOLOv7 model. The YOLOv7 model can be divided into three parts: Backbone, Neck and Head. The main focus is on optimizing the Neck and Head parts.
[0037] (3) Preprocess the training set image data, input the preprocessed images into the feature extraction network, and extract feature maps at different scales;
[0038] (4) Use the feature maps of each scale obtained in (3) to train the improved YOLOv7 model. During the training process, Precision, Recall and Mean Average Precision (mAP) are used as indicators to evaluate the model performance. The validation set data is used to monitor whether the model is overfitting until the loss curve converges.
[0039] (5) Input the test set data into the weight file of the improved YOLOv7 model obtained from training to obtain the actual recognition results of the target in the drone aerial image.
[0040] In this embodiment, step (1) specifically includes:
[0041] Download the VisDrone dataset from the Tianjin University Machine Vision and Data Mining Laboratory website. It contains 10209 images: 6471 for training, 548 for validation, and 3190 for testing. The dataset includes ten categories: pedestrian, people, bicycle, car, van, truck, tricycle, awning-tricycle, bus, and motor. This example uses only the 6471 training images and 548 validation images with publicly available labels. A Python program, split.py, is written to randomly divide the dataset into training, validation, and testing sets in an 8:1:1 ratio. The corresponding labels are placed in their respective folders, and the data is converted to YOLO format.
[0042] In this embodiment, step (2) specifically includes:
[0043] Since the application environment is a drone, and considering the limitations of the equipment, a lightweight YOLOv7-tiny model with SiLU as the activation function was chosen as the basis for improvement. The specific steps are as follows:
[0044] A small object detection layer is added: The YOLOv7-tiny network backbone consists of two Conv layers, a first feature extraction module, a second feature extraction module, a third feature extraction module, and a fourth feature extraction module. The Conv layers consist of convolutional layers, Batch Normalization (BN) layers, and SiLU activation functions. Except for the first feature extraction module, which contains only one extraction layer, each of the other feature extraction modules consists of a downsampling layer and an extraction layer. The feature maps P2 to P5 extracted by the four feature extraction modules have sizes of 160*160, 80*80, 40*40, and 20*20 pixels, respectively. The improved network further optimizes the detection of the other three features from the 160*160 pixel feature image. Figure 1 The feature maps are fused through a Conv layer and a Neck layer after feature extraction and upsampling, thus improving the overall structure of the YOLOv7 network. Figure 1 As shown, a fourth prediction head is generated using the fused features. Since this prediction head has a high resolution, it is advantageous for small target recognition. Therefore, the structure that introduces additional feature maps for fusion and finally generates the prediction head is called the small target detection layer.
[0045] Using a BiFPN structure: The original YOLOv7-tiny network used the PANet structure, but this embodiment replaces it with a BiFPN structure. The node connection methods of the two structures are compared below. Figure 2 As shown, the BiFPN structure in the Neck layer upsamples the input features and fuses them with the features extracted from the Backbone. The FPN structure adds edges with contextual information and multiplies each edge by a corresponding weight, generating four new feature layers. The detection results are output based on these feature layers. Furthermore, the ConCat layer of the previous network is changed to a BiFPN_ConCat layer, meaning its weights are considered during each fusion.
[0046] Replace CIoU with WIoU: The WIoU loss calculation method is shown in formulas (1) and (2).
[0047]
[0048] Among them, b,b gt ρ represents the center of the predicted box and the center of the ground truth box, respectively. ρ represents the Euclidean distance between the two centers. d is the diagonal distance between the predicted box and the ground truth box's smallest enclosed region. β is the outlier. α and δ are hyperparameters.
[0049] The original CIoU is replaced with the WIoU loss function. WIoU uses a dynamic non-monotonic focusing mechanism to replace IoU with outlier evaluation to assess anchor frame quality.
[0050] Add an attention-based detection head, DyHead: DyHead combines attention mechanisms to unify scale awareness, spatial awareness, and task awareness in object detection into a single framework. Figure 3 The specific structural arrangement of each attention module in DyHead is shown. DyHead converts self-attention function into three consecutive attentions, each focusing on only one angle, as shown in formula (3):
[0051]
[0052] in, Represents the eigenvector. Let L be the set of input feature vectors, S = H * W represent the reshaping of the height and width dimensions of the feature map, C represent the number of channels in the feature map, and π represent the input feature vector set. L π S π C Let represent the attention functions for the three dimensions of scale, space, and task, respectively, corresponding to formulas (4), (5), and (6):
[0053]
[0054] Where f(x) is a linear function approximated by a 1×1 convolution. It is a hard-sigmoid function.
[0055]
[0056] Where K is the number of sparse sampling locations, w is the corresponding weight coefficient, and p k +Δp k It is the position shifted by the self-learned spatial offset, Δm k It is a self-learning position p k Scalar.
[0057]
[0058] in, For the feature slice of channel C, [α] 1 ,α 2 ,β 1 ,β 2 ] T =θ(·) is a hyperfunction that learns to control the activation threshold.
[0059] DyHead can be stacked; this embodiment uses two layers. The specific locations within the network are as follows: Figure 1 As shown.
[0060] In this embodiment, step (3) specifically includes:
[0061] The Mosaic method involves randomly selecting four images from the VisDrone training set and performing augmentation operations such as flipping, scaling, and color gamut changes. The images are then randomly stitched together, retaining the original labels to form new sample images.
[0062] Mixup method: Randomly select two images, perform augmentation operations such as flipping, scaling, and color gamut changes, and then merge them according to a certain mixing coefficient to improve the robustness of the network during the testing phase.
[0063] Adaptive Anchor Box Calculation: YOLOv7 has preset anchor boxes with different widths and heights. The network outputs predicted boxes based on the initial anchor boxes, compares them with the ground truth boxes, and then updates them in reverse to adaptively calculate the best anchor box values for the training set.
[0064] Adaptive image scaling: Different images may have different lengths and widths. In order to unify the standard and facilitate calculation, the images are scaled to a specified size. The input size used here is 640*640.
[0065] In this embodiment, step (4) specifically includes:
[0066] Experimental environment setup: The experimental platform uses Ubuntu 20.04 as the operating system, PyTorch 1.11.0 as the framework, Nvidia RTX 3090 (24G VRAM) as the GPU, Python version 3.8, and CUDA version 11.3.
[0067] Set the training parameters as follows: batchsize = 16, initial learning rate lr = 0.01, and training epochs = 200.
[0068] During training, the parameters of the neural network are updated through forward propagation and backpropagation. The forward propagation process determines the approximate location of the target based on the fused features and filters it using a pre-set threshold to obtain predicted boxes. The backpropagation process calculates the error between the true and predicted boxes using a loss function and optimizes the neural network parameters using gradient descent to continuously reduce the error, causing the loss curve to gradually converge. Simultaneously, the precision (P), recall (R), and mean average precision (mAP) of the validation set are used as metrics to evaluate the model's performance. The calculation methods for P and R are shown in formulas (7) and (8):
[0069]
[0070] Where TP indicates correct classification; FP indicates that a negative sample is predicted as a positive sample; FN indicates that a positive sample is predicted as a negative sample; and TN indicates that a negative sample is predicted as a negative sample.
[0071] With P as the ordinate and R as the abscissa, the area enclosed by the PR curve and the coordinate axis is the AP value, while mAP represents the mean AP of all categories in the dataset.
[0072] After training, the changes in the mAP curves before and after model improvement are as follows: Figure 4 As shown, the model's mAP finally reached 0.414, which is a significant improvement compared to the previous mAP of 0.373.
[0073] In this embodiment, step (5) specifically includes:
[0074] Import the weight file obtained from step (4) into the detect.py detection program, add the test set data path prepared in step (1) to the program, run the program, and obtain the actual detection effect image of the improved YOLOv7 model for drone aerial targets. Select one of them as shown in the image. Figure 5 As shown in the figure, the actual recognition effect is good.
[0075] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An unmanned aerial vehicle aerial target detection method based on improved YOLOv7, characterized in that, The method comprises the following steps: (1) dividing an unmanned aerial vehicle aerial photography data set VisDrone into a training set, a verification set and a test set, the division ratio being 8:1:1, and the division being in a random form; (2) improving a YOLOv7 model, the YOLOv7 model being divided into three parts of Backbone, Neck and Head, and the two parts of Neck and Head being optimized; The specific method of step (2) is: a YOLOv7-tiny model taking SiLU as an activation function is selected as a basis for improvement; a small target detection layer is added: a feature map is extracted through the Backbone of the original YOLOv7-tiny network, the resolution is continuously reduced, and a large amount of small target features and position information are gradually lost in the process; the original YOLOv7-tiny network introduces three feature maps of different scales into the Neck part for fusion, and forms three prediction heads in the Head part; a feature map with a resolution of 160*160 is also introduced into the network for feature fusion, and a fourth prediction head for small targets is formed; a BiFPN structure is used: the YOLOv7-tiny uses the PANet method for feature fusion, the BiFPN structure of the Neck layer up-samples the input features and fuses them with the features extracted by the Backbone, the FPN structure adds a context information edge, and each edge is multiplied by a corresponding weight to generate four new feature layers, and the detection results are output according to the feature layers; CIoU is replaced by WIoU: the original YOLOv7-tiny uses a CIoU loss function to regress the bounding box, and the WIoU replaces the IoU with an outlying degree evaluation mechanism to evaluate the quality of the anchor box; a detection head DyHead based on an attention mechanism is added: DyHead is a dynamic detection head framework, which combines the attention mechanism to unify scale perception, spatial perception and task perception in a framework; (3) pre-processing the picture data of the training set, inputting the pre-processed pictures into a feature extraction network, and extracting feature maps of different scales; (4) training the improved YOLOv7 model by using the feature maps of different scales obtained in (3), using the precision, recall and mean average precision mAP as indexes for evaluating the performance of the model during the training process, and using the verification set data to monitor whether the model is over-fitted, until the loss curve converges; (5) inputting the test set data into the improved YOLOv7 model weight file obtained by training, and obtaining the actual recognition result of the unmanned aerial vehicle aerial image target.
2. The improved YOLOv7-based unmanned aerial vehicle aerial target detection method according to claim 1, characterized in that, The specific method of step (3) is: the picture pre-processing process includes a Mosaic method, a Mixup method, adaptive anchor box calculation and adaptive picture scaling, and after the above pre-processing steps, the training set picture data is sent to the input end of the improved YOLOv7 network.
3. The improved YOLOv7-based unmanned aerial vehicle aerial target detection method according to claim 1, characterized in that, The specific method of step (4) is: When training the improved YOLOv7-tiny network model with the feature maps of different scales obtained in step (3), the parameters of the neural network are updated through forward propagation and back propagation. In the forward propagation process, the rough position of the target is determined according to the fused features, and the prediction box is obtained by screening with a pre-set threshold. In the back propagation process, the error between the true box and the prediction box is calculated through the loss function, and the neural network parameters are optimized through the gradient descent method. At the same time, the mAP index of the validation set is observed to determine whether overfitting has occurred. When the loss curve tends to converge, the training of the model is completed.
4. The improved YOLOv7-based unmanned aerial vehicle aerial target detection method according to claim 1, characterized in that, The specific method of step (5) is: Input the test set picture into the trained model weight file, and the model will frame and label the detected target on the picture to verify the actual detection effect and inference speed of the model.