A method for target detection in UAV aerial images

Abstract

The present invention discloses a method for detecting targets in unmanned aerial vehicle aerial images, comprising the following steps: (1) obtaining and preparing a small target data set of unmanned aerial vehicles and performing preprocessing; (2) introducing a C3CBAM attention module, and replacing all C3 modules in a Backbone feature extraction network of yolov5s with C3CBAM modules; (3) adding a FEM module to the fourth layer of the Backbone feature extraction network of yolov5s; (4) introducing a SCAM module into a head layer of yolov5s to add three feature processing layers 23, 24, and 25; (5) introducing an ASFF module into a Detect detection layer of yolov5s; (6) using an improved yolov5s model for training and verification, and evaluating the improved model; the present invention enhances the performance of small target detection.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a target detection method for unmanned aerial vehicle (UAV) aerial images. Background Art

[0002] In recent years, with the continuous development of the drone industry, the cost of drones has also decreased, and their various performances have been significantly improved. Aerial images are obtained by carrying cameras on drones, and various targets are automatically detected using artificial intelligence technology. They can be widely used in surveying, monitoring, rescue, inspection and other tasks. However, the images taken by drones have defects such as high background complexity, small target size, and blurred appearance.

[0003] Existing UAV small target detection algorithms have their own advantages and disadvantages. For example, the Faster R-CNN network has high accuracy and can handle targets of different sizes, but it is slow and difficult to meet the needs of real-time applications; the SSD network is fast and suitable for real-time detection tasks, but has low detection accuracy for small targets. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method for detecting targets in UAV aerial images, so as to solve the problem that the existing technology has low detection accuracy for small targets of UAV in complex backgrounds and is prone to false detection and missed detection.

[0005] Technical solution: The method for detecting targets in unmanned aerial images of an unmanned aerial vehicle according to the present invention comprises the following steps:

[0006] (1) Obtain and preprocess the UAV small target dataset;

[0007] (2) Introduce the C3CBAM attention module and replace all C3 modules in the Backbone feature extraction network of yolov5s with the convolutional block attention module C3CBAM module;

[0008] (3) Add a feature enhancement module FEM module to the fourth layer of the Backbone feature extraction network of yolov5s;

[0009] (4) Introduce the spatial context perception module SCAM module into the head layer of yolov5s; add three feature processing layers 23, 24, and 25;

[0010] (5) The Detect layer of yolov5s is introduced into the adaptive spatial feature fusion module ASFF module;

[0011] (6) Use the improved yolov5s model for training and verification, and evaluate the improved model.

[0012] Furthermore, in step (1), the public dataset VisDrone2019 is used; the preprocessing is as follows: randomly cropping to a uniform size, dividing into training set, validation set and test set; converting the dataset into txt format.

[0013] Furthermore, step (2) is as follows: the C3CBAM module includes a channel attention module and a spatial attention module; wherein the channel attention module performs average pooling and maximum pooling on the input feature map, and outputs an average pooling feature map AvgPool(F) and a maximum pooling feature map MaxPool(F), respectively, to capture the global average information and global maximum information of the feature map, and then outputs two feature vectors with the same dimension through a multi-layer perceptron MLP with shared weights, and normalizes each element in the vector with a sigmoid function to form a channel attention feature map; the spatial attention module performs global average and maximum pooling operations on the feature map after the channel attention processing; and increases the local sensitivity. The convolutional layers capture local features and boundary details to generate spatial attention feature maps.

[0014] Furthermore, step (3) is as follows: the FEM module includes four branches; the first branch first passes the feature map F through Convolution is used to reduce the dimension. Convolution extracts local features to obtain output M1; the second branch is the convolution layer, including: convolution, convolution, convolution, Convolution; the third branch is the convolution layer, including: convolution, convolution, convolution, Convolution; the fourth branch is convolution.

[0015] Furthermore, in step (4), the SCAM module is introduced into the head layer of yolov5s, as follows: the SCAM module processes the input feature map through global average pooling and global maximum pooling, performs linear transformation on the processed feature map, and adds the calculated correlation feature map to the original feature map element by element.

[0016] Furthermore, in step (4), three feature processing layers 23, 24, and 25 are added as follows: the 23rd layer is added to extract high-level features; the 24th layer is added to extract medium-scale features; and the 25th layer is added to extract low-level, fine-grained features.

[0017] Furthermore, step (5) is as follows: the ASFF module receives the feature maps of the 23rd, 24th and 25th layers in the Nect module, and obtains three feature maps X(1), X(2) and X(3) after processing by the ASFF module. X(1), X(2) and X(3) are respectively compared with the weight parameters , , Multiply and sum, and after feature fusion, we get , the formula is as follows:

[0018] ;

[0019] in, represents the new feature map obtained by ASFF, , , Represent the weight parameters of the three feature layers respectively, and the Softmax function is used to satisfy them:

[0020] ;

[0021] in, , , They represent the features of the 23rd, 24th, and 25th layers respectively.

[0022] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: by replacing all C3 modules in the Backbone feature extraction network of yolov5s with C3CBAM modules and adding a FEM module to the fourth layer, the FEM module increases the distinction between background and small targets, and C3CBAM can better pay attention to small targets; the SCAM module is introduced into the head layer to add three feature processing layers 23, 24, and 25 to enhance the features of small targets; the Detect layer introduces the ASFF module; the present invention enhances the performance of small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of the present invention;

[0024] Figure 2 This is a schematic diagram of the improved network structure of yolov5s of the present invention;

[0025] Figure 3 Figure 1 shows the training results of the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0027] like Figure 1As shown, an embodiment of the present invention provides a method for detecting targets in drone aerial images, comprising the following steps:

[0028] (1) Obtain and prepare a drone small target dataset and perform preprocessing. The public dataset VisDrone2019 is used. The dataset contains 10 predefined categories and a total of 10,209 static images, 6,471 of which are used for training, 548 for verification, and 3,190 for testing. The public dataset VisDrone2019 is preprocessed as follows: randomly crop to a uniform size and convert the dataset into txt format.

[0029] (2) Introduce the C3CBAM attention module and replace all C3 modules in the Backbone feature extraction network of yolov5s with C3CBAM modules; Figure 2 As shown in the figure, the details are as follows: The C3CBAM module includes a channel attention module and a spatial attention module; among them, the channel attention module After average pooling and maximum pooling, the average pooling feature map AvgPool(F) and the maximum pooling feature map MaxPool(F) are output respectively, capturing the global average information and global maximum information of the feature map respectively. Then, two feature vectors with the same dimension are output through the multi-layer perceptron MLP with shared weights. Each element in the vector is normalized with the sigmoid function to form a channel attention feature map. The formula is as follows:

[0030] ;

[0031] Among them, C, H, and W are the number of feature map channels, height, and width, respectively. is the sigmoid activation function, , , r is the parameter reduction rate, and are the weights of the multilayer perceptron (MLP), and They use average pooling and maximum pooling in space respectively;

[0032] The spatial attention module performs global averaging and maximum pooling operations on the feature map after channel attention processing in the channel dimension, which will lose the important spatial positions in the image. To solve this problem, a local sensitive The convolution layer can help the network better capture local features and boundary details, and then generate a spatial attention feature map through a convolution layer. The formula is as follows:

[0033] ;

[0034] in, , and Respectively represent the use of average pooling and maximum pooling for F in the channel dimension, for The convolution, for convolution.

[0035] (3) Add the FEM module to the fourth layer of the Backbone feature extraction network of yolov5s; the details are as follows: The FEM module includes four branches; the first branch first passes the feature map F through Convolution is used to reduce the dimension. Convolution extracts local features to obtain output M1; the second branch is the convolution layer, including: convolution, convolution, convolution, Convolution; the third branch is the convolution layer, including: convolution, convolution, convolution, Convolution; the fourth branch is Convolution. The formula is as follows:

[0036] ;

[0037] ;

[0038] ;

[0039] ;

[0040] in, represents the output feature map obtained by the first branch, represents the output feature map obtained by the second branch, Represents the output feature map obtained by the third branch, A represents three branches , , The output feature map of ; obtained by concatenation, and then combined with The final output feature map is obtained by element-wise addition of the original input feature map processed by convolution; Represents Conv convolution, Represents Conv convolution, Represents Conv convolution, Represents Conv Convolution; Cat represents the concatenation operation of the feature map. Represents the feature map after fusion of M1, M2, and M3 and the input feature map Add.

[0041] (4) Introduce the SCAM module into the head layer of yolov5s to add three feature processing layers 23, 24, and 25; Introduce the SCAM module into the head layer of yolov5s, as follows: The SCAM module processes the input feature map through global average pooling and global maximum pooling, performs linear transformation on the processed feature map, and adds the calculated correlation feature map to the original feature map element by element. The formula is as follows:

[0042] ;

[0043] ;

[0044] in, and Represent the input and output of the jth pixel in the i-th layer feature map, respectively. It represents the attention weight of the jth pixel in the feature map of the i-th layer. Represents the total number of pixels, and is the linear transformation matrix used for feature mapping, exp is the exponential function used for weighted calculation, and GAP and GMP are performed separately, where GAP and GMP represent global average pooling and global maximum pooling respectively.

[0045] The details of adding three feature processing layers 23, 24, and 25 are as follows: adding the 23rd layer is used to extract high-level features, which helps the improved model capture more comprehensive global context information, thereby improving the detection ability of small targets; adding the 24th layer is used to extract medium-scale features, which can enhance the model's feature representation ability for medium-scale targets, and also helps to integrate global information from different scales; adding the 25th layer is used to extract low-level, fine-grained features, which can help the model better understand the details in the image and detect small targets in combination with global context information.

[0046] (5) The Detect layer of yolov5s is introduced into the ASFF module. The ASFF module receives the feature maps of the 23rd, 24th and 25th layers in the Nect module. After being processed by the ASFF module, three feature maps X(1), X(2) and X(3) are obtained. X(1), X(2) and X(3) are respectively correlated with the weight parameters , , Multiply and sum, and after feature fusion, we get ; The formula is as follows:

[0047] ;

[0048] in, Represents the new feature map obtained by ASFF, , , Represent the weight parameters of the three feature layers respectively, and the Softmax function is used to satisfy them:

[0049] ;

[0050] in, , , They represent the features of the 23rd, 24th, and 25th layers respectively.

[0051] (6) Use the improved yolov5s model for training and verification, and evaluate the improved model. Figure 3 As shown, training / localization loss: bounding box loss on the training set. This metric measures the difference between the bounding box predicted by the model on the training data and the true bounding box. The smaller the loss, the better the model performs in locating objects.

[0052] Training / Confidence Loss: The target loss on the training set. This metric measures the accuracy of the model's prediction of the presence or absence of objects on the training data. The smaller the loss, the better the model performs in identifying the presence of objects.

[0053] Training / Classification Loss: The classification loss on the training set. This metric measures the model’s prediction accuracy for object categories on the training data. The smaller the loss, the better the model performs in object classification.

[0054] Precision: The precision on the training set. Precision indicates how many of the positive samples predicted by the model are true positive samples. The higher the precision, the fewer negative samples the model falsely reports.

[0055] Recall rate: The recall rate on the training set. The recall rate indicates how many of the real positive samples are correctly identified by the model. The higher the recall rate, the fewer positive samples the model misses.

[0056] Validation / Localization Loss: Bounding Box Loss on the Validation Set. This is the bounding box loss of the model on the validation data and is used to evaluate the generalization ability of the model.

[0057] Validation / Confidence Loss: Target loss on validation set. This is the target existence loss of the model on validation data, which is used to evaluate the model's performance in identifying objects on new data.

[0058] Validation / Classification Loss: Classification loss on validation set. This is the classification loss of the model on validation data and is used to evaluate the model’s performance in classifying objects on new data.

[0059] 0.5 Average Precision: calculated on the training set when the IOU threshold is 0.5, used to evaluate the overall detection performance of the model on the training data.

[0060] 0.5-0.95 Average Precision: The average accuracy calculated at different IOU thresholds (from 0.5 to 0.95) on the training set is used to more comprehensively evaluate the detection performance of the model.

Claims

1. A method for detecting targets in drone aerial images, characterized in that: The following steps are involved: S1 obtains the drone small target dataset and preprocesses it; the public dataset VisDrone2019 is used; the preprocessing is as follows: randomly crop to a uniform size, divide into training set, validation set and test set; convert the dataset into txt format; S2 introduces the C3CBAM attention module, replacing all C3 modules in the Backbone feature extraction network of yolov5s with the convolutional block attention module C3CBAM module; specifically, the C3CBAM module includes a channel attention module and a spatial attention module; among them, the channel attention module After average pooling and maximum pooling, the average pooling feature map AvgPool(F) and the maximum pooling feature map MaxPool(F) are output respectively, capturing the global average information and global maximum information of the feature map respectively. Then, two feature vectors with the same dimension are output through the multi-layer perceptron MLP with shared weights. Each element in the vector is normalized with the sigmoid function to form a channel attention feature map. The formula is as follows: ； Among them, C, H, and W are the number of feature map channels, height, and width, respectively. is the sigmoid activation function, , , r is the parameter reduction rate, and are the weights of the multilayer perceptron MLP, and They use average pooling and maximum pooling in space respectively; The spatial attention module performs global averaging and maximum pooling operations on the feature maps processed by the channel attention module in the channel dimension; adding a local sensitive The convolution layer helps the network capture local features and boundary details, and then generates a spatial attention feature map through a convolution layer. The formula is as follows: ； in, , and Respectively represent the use of average pooling and maximum pooling for F in the channel dimension, for The convolution, for Convolution; S3 adds a feature enhancement module FEM module to the fourth layer of the Backbone feature extraction network of yolov5s; the details are as follows: the FEM module includes four branches; among them, the first branch first passes the feature map F through Convolution is used to reduce the dimension. Convolution extracts local features to obtain output M1; the second branch is the convolution layer, including: convolution, convolution, convolution, Convolution; the third branch is the convolution layer, including: convolution, convolution, convolution, Convolution; the fourth branch is Convolution; the formula is as follows: ； ； ； ； in, represents the output feature map obtained by the first branch, represents the output feature map obtained by the second branch, Represents the output feature map obtained by the third branch, A is the three branches , , After splicing, The final output feature map is obtained by element-wise addition of the original input feature map processed by convolution; Represents Conv convolution, Represents Conv convolution, Represents Conv convolution, Represents Conv Convolution; Cat represents the concatenation operation of the feature map. Represents the feature map after fusion of M1, M2, and M3 and the input feature map Addition; S4 introduces the spatial context perception module SCAM module into the head layer of yolov5s; adds three feature processing layers 23, 24, and 25; among them, the SCAM module is introduced into the head layer of yolov5s, as follows: the SCAM module processes the input feature map through global average pooling and global maximum pooling, performs linear transformation on the processed feature map, and adds the calculated correlation feature map to the original feature map element by element; the formula is as follows: ； ； in, and Represent the input and output of the jth pixel in the i-th layer feature map, respectively. It represents the attention weight of the jth pixel in the feature map of the i-th layer. Represents the total number of pixels, and is the linear transformation matrix used for feature mapping, exp is the exponential function used for weighted calculation, and Execute GAP and GMP respectively, GAP and GMP represent global average pooling and global maximum pooling respectively; The details of adding three feature processing layers 23, 24, and 25 are as follows: adding layer 23 to extract high-level features helps improve the model to capture more comprehensive global context information, thereby improving the ability to detect small targets; adding layer 24 to extract medium-scale features can enhance the model's feature representation ability for medium-scale targets and also help integrate global information from different scales; adding layer 25 to extract low-level, fine-grained features can help the model better understand the details in the image and detect small targets in combination with global context information; S5 introduces the Detect layer of yolov5s into the ASFF module. The ASFF module receives the feature maps of the 23rd, 24th and 25th layers in the Nect module. After being processed by the ASFF module, three feature maps X(1), X(2) and X(3) are obtained. X(1), X(2) and X(3) are respectively compared with the weight parameters , , Multiply and sum, and after feature fusion, we get ; The formula is as follows: ； in, Represents the new feature map obtained by ASFF, , , Represent the weight parameters of the three feature layers respectively, and the Softmax function is used to satisfy them: ； in, , , They represent the features of the 23rd, 24th, and 25th layers respectively; S6 uses the improved yolov5s model for training and verification, and evaluates the improved model.

Images