A small target detection method for UAV based on improved multi-head self-attention

Abstract

The present invention provides a method for detecting small targets of unmanned aerial vehicles based on improved multi-head self-attention, comprising: step 1: collecting images from the deployed resident camera monitoring to obtain a picture data set containing low-altitude unmanned aerial vehicles; step 2: using the picture data set containing low-altitude unmanned aerial vehicles to pre-train the original YOLOv5s model to obtain the pre-training weights of YOLOv5s; step 3: improving the multi-head self-attention module according to the characteristics of the small targets of unmanned aerial vehicles, and adding it to the YOLOv5s model; step 4: using the picture data set containing low-altitude unmanned aerial vehicles and the pre-training weights to re-train the YOLOv5s model to obtain new training weights of YOLOv5s; step 5: loading the new training weights into the YOLOv5s model to detect the resident camera monitoring data. Adding the improved multi-head self-attention module on the basis of the YOLOv5s target detection model is not only suitable for real-time small target detection tasks of unmanned aerial vehicles, but also enhances the ability of the target detection model to extract small target features.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection in computer vision, and in particular to a method for detecting small targets in unmanned aerial vehicles based on improved multi-head self-attention. Background Art

[0002] In the field of computer vision, target detection, as an important technology, is widely used in practical scenarios, such as face detection, pedestrian detection, vehicle detection, drone detection, etc. There are mainly traditional target detection methods and deep learning-based target detection methods for target detection. In the direction of small target detection, the small effective proportion of the target leads to poor performance of traditional target detection algorithms, while the target detection method based on deep learning can maintain a high detection accuracy. Therefore, the target detection method based on deep learning has gradually become the mainstream technology for small target detection.

[0003] The target detection methods based on deep learning are mainly divided into two categories: one-stage target detection methods and two-stage target detection methods. The one-stage target detection method mainly regards the target detection problem as a regression problem, predicting the target category while detecting the target position. Its representative algorithms mainly include YOLO, YOLO9000, YOLOv3, SSD, etc.; the two-stage target detection method is similar to the traditional method, which requires generating a large number of candidate regions first and then classifying the regions. Its representative algorithms include R-CNN, Fast R-CNN, Faster R-CNN, Cascade R-CNN, etc. In recent years, with the continuous development of the target detection field, the one-stage target detection method has greatly improved the detection accuracy while retaining its speed. Therefore, in order to meet the real-time requirements of small target detection of drones, the one-stage target detection method based on deep learning can be adopted.

[0004] For low-altitude drone small targets in urban areas, using surveillance cameras already deployed in the city for detection is undoubtedly the most economical method. However, in an urban environment with complex backgrounds and noisy sounds, the one-stage target detection method based on deep learning cannot detect small drone targets well. To this end, the present invention selects a more effective one-stage target detection model—YOLOv5s, and improves a general attention mechanism to enhance the detection accuracy of the detection model for small drone targets. Summary of the invention

[0005] In order to solve the technical problems raised by the background technology, the present invention provides a UAV small target detection method based on improved multi-head self-attention. An improved multi-head self-attention module is added on the basis of the YOLOv5s target detection model, which is not only suitable for real-time UAV small target detection tasks, but also enhances the ability of the target detection model to extract small target features.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for detecting small targets of unmanned aerial vehicles based on improved multi-head self-attention includes the following steps:

[0008] Step 1: Collect images from the deployed resident camera surveillance, manually annotate the collected images, and obtain a picture dataset containing low-altitude drones;

[0009] Step 2: Use the image dataset containing low-altitude drones to pre-train the original YOLOv5s model and obtain the pre-training weights of YOLOv5s;

[0010] Step 3: Improve the multi-head self-attention module according to the characteristics of small drone targets, and add the improved multi-head self-attention module to the YOLOv5s model;

[0011] Step 4: Retrain the YOLOv5s model using the image dataset containing low-altitude drones and pre-trained weights to obtain new training weights for YOLOv5s;

[0012] Step 5: Load the newly trained weights into the YOLOv5s model to detect the resident camera surveillance data.

[0013] Furthermore, in step 3, improving the multi-head self-attention module includes:

[0014] The multi-head self-attention module has three input ports and one output port. The input port inputs the original features, and the output port outputs the new features processed by the multi-head self-attention module.

[0015] The first input port is connected to the improved residual channel attention submodule, which is used to improve the significance of the effective features of small objects in the input features;

[0016] The second input port is connected to the improved residual space feature attention submodule, which is used to improve the significance of the effective features of small objects in the input features;

[0017] The third input port and two improved attention submodules are connected to the convolutional layer submodule. The convolutional layer replaces the original fully connected layer to improve the plug-and-play performance of multi-head self-attention.

[0018] The multi-head scaled dot product attention submodule is connected to the convolutional layer submodule and outputs new features through concatenation and convolution processing.

[0019] Furthermore, the improved residual channel attention submodule uses a residual mechanism to add the original input features to the adjusted features so that the output features can retain some shallow features and then retain more small target features.

[0020] Furthermore, the improved residual space feature attention submodule includes:

[0021] Use maximum pooling to reduce the number of input feature channels;

[0022] Use three layers of convolution to form a spatial pyramid to expand the feature search space and improve the capture rate of effective features of small target scales;

[0023] Concatenate the maximum pooling features and the three features of the three-layer convolutional spatial pyramid;

[0024] Calculate the mean feature after splicing;

[0025] The dot product of the input feature and the mean feature is used to obtain the weighted feature;

[0026] The input features are added to the weighted features using a residual mechanism so that the output features can retain some shallow features and then retain more small target features.

[0027] Furthermore, in the step 3, the improved multi-head self-attention module is added to the YOLOv5s model, that is, the improved multi-head self-attention module is placed after the splicing layer module of YOLOv5s to adjust the significance of the effective features of the small target after splicing, thereby reducing the difficulty of the detection layer to capture the effective features of the small target.

[0028] Furthermore, in step 5, the monitoring data of the residents' cameras is detected, and the process is as follows:

[0029] Extract single-frame images from the deployed resident camera surveillance;

[0030] Call the YOLOv5s model loaded with the newly trained weights to detect a single frame image;

[0031] If a small target drone is detected, the small target drone area is automatically selected using a rectangular frame.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1) A method for detecting small targets of unmanned aerial vehicles based on improved multi-head self-attention in the present invention provides an improved multi-head self-attention module, uses a convolutional layer to replace the original fully connected layer, uses improved residual channel attention and improved residual spatial feature attention to adjust input features, improves the plug-and-play nature of multi-head self-attention and the significance of small target effective features, and thus improves the detection accuracy of small targets of unmanned aerial vehicles;

[0034] 2) The present invention provides a method for detecting small targets of unmanned aerial vehicles based on improved multi-head self-attention, provides an improved residual spatial feature attention submodule, adds spatial pyramid features and residual mechanism to the spatial attention, so that the submodule can search for small target features from multiple spaces, and retains shallow small target features, greatly improving the proportion of effective features of small targets;

[0035] 3) The present invention provides a method for detecting small targets in unmanned aerial vehicles based on improved multi-head self-attention, provides an improved residual channel attention submodule, and introduces a residual mechanism so that the output features can retain some shallow features, thereby increasing the proportion of small target features. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of a method for detecting small targets in unmanned aerial vehicles based on improved multi-head self-attention provided by an embodiment of the present invention;

[0037] Figure 2 A structural diagram of an improved residual space feature attention submodule provided in an embodiment of the present invention;

[0038] Figure 3 A structural diagram of the improved multi-head self-attention module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The specific implementation modes provided by the present invention are described in detail below with reference to the accompanying drawings.

[0040] The present invention provides a method for detecting small targets of unmanned aerial vehicles based on improved multi-head self-attention. The main process is as follows: Figure 1 As shown, the following steps are included:

[0041] S1: Real-time image acquisition from residents’ camera monitoring1;

[0042] S2: The collected image 1 is fed into the YOLOv5s model with an improved multi-head self-attention module for forward reasoning;

[0043] S3: Determine whether image 1 contains a small target drone based on the forward reasoning result. If so, proceed to S4; otherwise, proceed to S6.

[0044] S4: Extract the location information of the small target drone from the result of forward reasoning and select it in image 1;

[0045] S5: Issue an abnormal warning to the administrator;

[0046] S6: Get the next frame of image from the resident camera monitoring and enter S2.

[0047] The S2 described above has a YOLOv5s model with an improved multi-head self-attention module, and the acquisition process is as follows:

[0048] Collect images from the deployed resident cameras, manually annotate the collected images, and obtain a picture dataset containing low-altitude drones;

[0049] Use a dataset of images containing low-altitude drones to pre-train the original YOLOv5s model and obtain the pre-training weights of YOLOv5s.

[0050] The multi-head self-attention module is improved according to the characteristics of small drone targets, and the improved multi-head self-attention module is added to the YOLOv5s model;

[0051] Retrain the YOLOv5s model using a low-altitude drone image dataset and pre-trained weights to obtain new training weights for YOLOv5s.

[0052] The S2 improved multi-head self-attention module, such as Figure 3 As shown, including:

[0053] It has three input ports Q, K and V, and one output port A. The input port inputs the original features, and the output port outputs the new features processed by the multi-head attention module.

[0054] The input port Q is connected to an improved residual channel attention submodule to improve the significance of effective features of small objects in the input features;

[0055] The input port K is connected to the improved residual space feature attention submodule to improve the significance of the effective features of small objects in the input features;

[0056] The input port V and the two improved attention submodules are connected to the convolutional layer submodule. The convolutional layer replaces the original fully connected layer to improve the plug-and-play performance of multi-head self-attention.

[0057] The scaled dot product attention submodule has H heads, and all submodules in the front end also use H heads to correspond to it. The multi-head scaled dot product attention submodule is connected to the convolutional layer submodule, and outputs new features through splicing and convolution processing.

[0058] The output formula of the multi-head self-attention module is as follows:

[0059]

[0060] R_CA represents the residual channel attention calculation, RSP_SA represents the residual spatial feature attention calculation, and F 1x1 represents 1×1 convolution, d k Represents the dot product variance of R_CA and RSP_SA.

[0061] The improvements of the multi-head self-attention module are as follows:

[0062] 1) Use convolutional layers to replace the original fully connected layers, which improves the plug-and-play nature of multi-head self-attention;

[0063] 2) Using improved residual channel attention, we adjust the input features of the input port Q of the multi-head self-attention, which improves the significance of the effective features of small targets;

[0064] 3) Using improved residual space feature attention, the input features of the input port K of the multi-head self-attention are adjusted to improve the significance of effective features of small targets.

[0065] The improved residual channel attention submodule comprises:

[0066] The channel attention module will lose shallow features, which are particularly important in small object detection.

[0067] The residual mechanism is used to add the original input features to the adjusted features so that the output features can retain some shallow features to retain more small target features.

[0068] The calculation formula of the residual channel attention submodule is as follows:

[0069] H(x)=F(x)+x

[0070] F(x) represents the channel attention calculation, H(x) represents the output feature, and x represents the input feature.

[0071] The improved residual space feature attention submodule, such as Figure 2 As shown, including:

[0072] Use maximum pooling to reduce the number of input feature channels;

[0073] Use three layers of convolution to form a spatial pyramid to expand the feature search space and improve the capture rate of effective features of small target scales;

[0074] Concatenate the maximum pooling features and the three features of the three-layer convolutional spatial pyramid;

[0075] Calculate the mean feature after splicing;

[0076] The dot product of the input feature and the mean feature is used to obtain the weighted feature;

[0077] The input features are added to the weighted features using a residual mechanism.

[0078] The purpose of improving the residual space feature attention submodule is to extract the features of small targets from multiple spaces and retain the features of small targets in some shallow layers, thereby improving the capture rate of effective features of small target scales.

[0079] The calculation formula of the improved residual space feature attention submodule is as follows:

[0080]

[0081] F represents the input feature, f 5x5 represents a 5x5 convolution, It means taking the maximum value of the same position on each channel.

[0082] The improved multi-head self-attention module is added to the YOLOv5s model, which is mainly to place the improved multi-head self-attention module after the splicing layer module of YOLOv5s to adjust the significance of the effective features of the small target after splicing, and reduce the difficulty of the detection layer to capture the effective features of the small target.

[0083] The above embodiments are implemented based on the technical solution of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the above embodiments. The methods used in the above embodiments are conventional methods unless otherwise specified.

Claims

1. A method for detecting small targets of unmanned aerial vehicles based on improved multi-head self-attention, characterized in that: The steps include: Step 1: Collect images from the deployed resident camera surveillance, manually annotate the collected images, and obtain a picture dataset containing low-altitude drones; Step 2: Use the image dataset containing low-altitude drones to pre-train the original YOLOv5s model and obtain the pre-training weights of YOLOv5s; Step 3: Improve the multi-head self-attention module according to the characteristics of small drone targets, and add the improved multi-head self-attention module to the YOLOv5s model; Step 4: Retrain the YOLOv5s model using the image dataset containing low-altitude drones and pre-trained weights to obtain new training weights for YOLOv5s; Step 5: Load the newly trained weights into the YOLOv5s model to detect the resident camera surveillance data; In step 3, improving the multi-head self-attention module includes: The multi-head self-attention module has three input ports and one output port. The input port inputs the original features, and the output port outputs the new features processed by the multi-head self-attention module. The first input port is connected to the improved residual channel attention submodule, which is used to improve the significance of the effective features of small objects in the input features; The second input port is connected to the improved residual space feature attention submodule, which is used to improve the significance of the effective features of small objects in the input features; The third input port and two improved attention submodules are connected to the convolutional layer submodule. The convolutional layer replaces the original fully connected layer to improve the plug-and-play performance of multi-head self-attention. The multi-head scaled dot product attention submodule is connected to the convolutional layer submodule and outputs new features through concatenation and convolution processing; The improved residual channel attention submodule uses a residual mechanism to add the original input features to the adjusted features, so that the output features can retain some shallow features and then retain more small target features; The improved residual space feature attention submodule comprises: Use maximum pooling to reduce the number of input feature channels; Use three layers of convolution to form a spatial pyramid to expand the feature search space and improve the capture rate of effective features of small target scales; Concatenate the maximum pooling features and the three features of the three-layer convolutional spatial pyramid; Calculate the mean feature after splicing; The dot product of the input feature and the mean feature is used to obtain the weighted feature; The input features are added to the weighted features using a residual mechanism so that the output features can retain some shallow features and then retain more small target features.

2. The method for detecting small targets of unmanned aerial vehicles based on improved multi-head self-attention according to claim 1 is characterized in that: In the step 3, the improved multi-head self-attention module is added to the YOLOv5s model, that is, the improved multi-head self-attention module is placed after the splicing layer module of YOLOv5s to adjust the significance of the effective features of the small target after splicing, and reduce the difficulty of the detection layer to capture the effective features of the small target.

3. The method for detecting small targets of unmanned aerial vehicles based on improved multi-head self-attention according to claim 1 is characterized in that: In step 5, the monitoring data of the residents' cameras is detected, and the process is as follows: Extract single-frame images from the deployed resident camera surveillance; Call the YOLOv5s model loaded with the newly trained weights to detect a single frame image; If a small target drone is detected, the small target drone area is automatically selected using a rectangular frame.

Images