Small target detection method and device for unmanned aerial vehicle image, medium and equipment

Abstract

The invention discloses a small target detection method and device for an unmanned aerial vehicle image, a medium and equipment, and relates to the field of image target detection, and the method comprises the steps: on the basis of an original single multi-frame detection SSD algorithm, adding a self-attention mechanism after feature extraction through a backbone network, adding an attention mechanism in front of a detection head, and constructing an improved SSD algorithm; training the improved SSD algorithm to obtain a detection model for detecting small targets in the unmanned aerial vehicle image; and inputting a to-be-detected unmanned aerial vehicle image into the detection model to obtain a target image, and detecting the target image through the detection head. According to the method, multi-scale feature fusion and an attention mechanism are applied to unmanned aerial vehicle image target detection, deep features and shallow features are fused, channels and spatial features are serialized, and the detection precision of small targets in unmanned aerial vehicle images can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image target detection, and in particular to a method, device, medium and equipment for detecting small targets in unmanned aerial vehicle images. Background Art

[0002] Object detection using drone images is widely used in emergency relief, power inspection, and traffic monitoring, thanks to the small size, flexibility, low cost, and ease of operation of drones. Through images captured by drones, researchers can perform intelligent analysis and quickly and efficiently extract information about objects on the ground, saving a lot of manpower and material resources.

[0003] However, the targets in drone aerial images are small and numerous, and the background is complex, which seriously affects the detection accuracy. With the development of deep learning technology, the target detection algorithm based on deep learning has made great improvements in detection accuracy and speed, which has promoted the development of target detection tasks based on drone images.

[0004] The deep learning-based object detection algorithm automatically learns image features on large-scale data sets, significantly improving the accuracy of object detection and showing good generalization ability for complex scenes. Although the deep learning-based algorithm has achieved remarkable results in detecting medium to large-sized objects, it still has the problem of insufficient accuracy in detecting small objects that frequently appear in drone images. Summary of the invention

[0005] The present invention provides a method, device, medium and equipment for detecting small targets in drone images, which are used to solve the above-mentioned problem existing in the prior art, that is, how to improve the detection accuracy of small targets in drone images in the prior art. The present invention provides a method for detecting small targets in drone images, which includes:

[0006] Construct an improved single-shot multi-border detection SSD algorithm; based on the original single-shot multi-border detection SSD algorithm, add a self-attention mechanism after global feature extraction through the backbone network VGG-16, and add an attention mechanism before the detection head;

[0007] Train the improved single-shot multi-border detection (SSD) algorithm to obtain a detection model for detecting small targets in drone images;

[0008] The drone image to be detected is input into the detection model, and the relevant data in the global features is obtained through the self-attention mechanism. The feature correlation of the drone image is established, and the features of different scales are fused to obtain the fused feature map. Then, based on the attention mechanism, the fused feature map is serialized in the channel and space dimensions. The serialized feature map is multiplied with the feature map of the original input to obtain the target image, and the target image is detected by the detection head.

[0009] Optionally, the self-attention mechanism is used to obtain relevant data in the global features, and feature correlation of the drone image is established to fuse features of different scales, specifically including:

[0010] Based on the features of different scales in the global features, the feature matrix is ​​determined by a lightweight convolution kernel;

[0011] According to the feature matrix, the global weighted feature mapping matrix is ​​determined by matrix multiplication and softmax function, and the global weighted feature mapping matrix is ​​multiplied by the feature matrix to obtain the weight value matrix of all positions in the same feature map;

[0012] Features of different scales are fused according to the original features and the weighted value matrix.

[0013] Optionally, the global features specifically include:

[0014] Deep features and shallow features; the deep features include semantic features; the shallow features include edge features and texture features.

[0015] Optionally, the fused feature map is serialized in two dimensions, channel and space, and then the serialized feature map is multiplied by the original input feature map to obtain the target image, specifically including:

[0016] The channel attention module is used to perform maximum pooling and average pooling operations on the drone image in the channel dimension, and then the two results are concatenated and normalized to obtain the weight matrix on the channel;

[0017] The spatial attention module is used to perform maximum pooling and average pooling operations on the drone image in the spatial dimension, and the two results are spliced. According to the weight matrix on the channel, a 7×7 convolution layer is passed to compress the number of channels to 1 to obtain the target image.

[0018] The present invention provides a small target detection device for unmanned aerial vehicle images, comprising:

[0019] A construction module is used to construct an improved single-shot multi-border detection SSD algorithm. Based on the original single-shot multi-border detection SSD algorithm, a self-attention mechanism is added after global feature extraction through the backbone network VGG-16, and an attention mechanism is added before the detection head.

[0020] The training module is used to train the improved single-shot multi-border detection SSD algorithm to obtain a detection model for detecting small targets in drone images;

[0021] The detection module is used to input the drone image to be detected into the detection model, obtain relevant data in the global features through the self-attention mechanism, establish the feature correlation of the drone image, fuse the features of different scales, and obtain the fused feature map. Then, based on the attention mechanism, the fused feature map is serialized in the channel and space dimensions, and the serialized feature map is multiplied with the feature map of the original input to obtain the target image, and the target image is detected by the detection head.

[0022] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned small target detection method for drone images is implemented.

[0023] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned small target detection method for drone images when executing the program.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides a small target detection method for drone images. When extracting features from drone images, the method introduces a self-attention mechanism so that the detection model efficiently and selectively fuses the semantic features with the shallow features in the backbone network, and applies this information to the small target detection process, thereby improving the accuracy of small target detection; through the attention mechanism module, the attention feature information in the two dimensions of space and channel is multiplied with the original feature for adaptive feature correction, which can focus the detection process on the target object, suppress the interference of background and other useless noise information, and more accurately determine the target information and position, thereby improving the accuracy of small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0026] Figure 1 A flowchart of a small target detection method for a drone image provided by an embodiment of the present invention;

[0027] Figure 2 A structural diagram of the SSD algorithm provided by an embodiment of the present invention;

[0028] Figure 3 A structural diagram of a multi-scale feature fusion module provided in an embodiment of the present invention;

[0029] Figure 4 A schematic diagram of a computer device for a small target detection method for drone images provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Existing drone image target detection mainly relies on deep learning algorithms. First, the backbone network is used to construct a feature pyramid, and then predictions are made at different feature layers. This process does not take into account the lack of semantic information in shallow features for small target detection, and does not use a suitable attention mechanism to focus on analyzing target features in drone images. As a result, missed detections and false detections are prone to occur during target detection, thus restricting the accuracy of drone image target detection.

[0032] In view of the above problems, the present invention provides a small target detection method for drone images. The method applies spatial and channel attention mechanisms to the feature pyramid construction process through an drone image target detection algorithm based on attention mechanism and feature fusion, and introduces deep semantic features into shallow features, so that the shallow features contain both details, texture and other information as well as semantic information, thereby improving the precision and accuracy of target detection.

[0033] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0034] Figure 1 is a flow chart of a small target detection method for a drone image provided by an embodiment of the present invention. Figure 1 As shown, this embodiment shows a small target detection method for drone images, including:

[0035] S1: Construct an improved single-shot multi-border detection SSD algorithm; based on the original single-shot multi-border detection SSD algorithm, add a self-attention mechanism after global feature extraction through the backbone network VGG-16, and add an attention mechanism before the detection head.

[0036] Optionally, the global features include deep features and shallow features;

[0037] Exemplarily, deep features may include semantic features, for example, and shallow features may include edge features and texture features.

[0038] S2: Train the improved single-shot multi-border detection (SSD) algorithm to obtain a detection model for detecting small targets in drone images.

[0039] S3: Input the drone image to be detected into the detection model, obtain relevant data in the global features through the self-attention mechanism, establish the feature correlation of the drone image, fuse the features of different scales, and obtain the fused feature map. Then, based on the attention mechanism, serialize the fused feature map in the channel and space dimensions, and then multiply the serialized feature map with the original input feature map to obtain the target image, and detect the target image through the detection head.

[0040] like Figure 2 As shown in the figure, the SSD algorithm uses the VGG-16 network as the basic architecture to extract image features and generate feature maps of multiple scales, which are used to detect objects of different sizes. A series of prior boxes with various aspect ratios and sizes are arranged on each feature map, which cover the possible locations of objects in the image. Finally, the SSD algorithm uses the non-maximum suppression (NMS) technology to filter out high-confidence prediction boxes to ensure the final output of the object detection result.

[0041] like Figure 3 As shown in the figure, the deep features are upsampled by deconvolution, and then a 3×3 convolution kernel is used to unify the channel dimension and perform channel compression, so that the deep features and shallow features in the backbone network are efficiently and selectively fused together; in addition, in order to alleviate the problem of increasing the order of magnitude of fusion of feature layers at different levels during the fusion process, a non-parametric averaging operation is used to obtain the fused feature layer, thereby obtaining the final fusion layer.

[0042] Exemplarily, the backbone network extracts the global features of the drone, reorganizes and concatenates them, and then obtains the global features according to the following formula. The specific calculation formula is as follows:

[0043]

[0044] Among them, s i The feature information extracted by the backbone network is C represents the channel dimension of the result tensor X, and H×W represents the resolution of the result tensor X; Indicates the use of a 3×3 convolution kernel to unify the channel dimension; [] indicates connection calculation; n indicates the number of fused feature layers in the feature pyramid.

[0045] By adopting the self-attention algorithm, features of different scales in the global features are fused through a lightweight fusion method to obtain a multi-scale feature fusion module.

[0046] Optionally, based on features of different scales in the global features, a feature matrix is ​​determined by a lightweight convolution kernel;

[0047] According to the feature matrix, the global weighted feature mapping matrix is ​​determined by matrix multiplication and softmax function, and the global weighted feature mapping matrix is ​​multiplied by the feature matrix to obtain the weight value matrix of all positions in the same feature map;

[0048] According to the original features and the weighted value matrix, features of different scales are fused.

[0049] Exemplarily, the global weighted feature mapping matrix is ​​obtained by using matrix multiplication and softmax function. The specific calculation formula is as follows:

[0050] D=Q T K;

[0051]

[0052] in, A ij represents the element in row i and column j, all A ij Constitute the global weighted feature map matrix A.

[0053] Optionally, the global feature feature is added to the matrix multiplication result to obtain the output of the self-attention unit AU. The specific formula is as follows:

[0054] X'=X+(AV T ) T ;

[0055] The feature X' obtained from the self-attention unit AU i In order to alleviate the problem of increasing order of magnitude of fusion of feature layers at different levels during feature fusion, non-parametric averaging operation is used to obtain the fused feature layer, thereby obtaining the final fusion layer.

[0056] The feature X' obtained from the self-attention unit AU i Perform feature fusion, the specific calculation formula is as follows:

[0057]

[0058] Among them, i∈n-1, the i-th layer is shallower than the i+1th layer and the i+2th layer; f 1×1 Used to reduce the number of channels.

[0059] Optionally, perform maximum pooling and average pooling operations on the drone image in the channel dimension through the channel attention module, and then concatenate and normalize the two results to obtain the weight matrix on the channel;

[0060] The spatial attention module is used to perform maximum pooling and average pooling operations on the drone image in the spatial dimension, and the two results are concatenated. According to the weight matrix on the channel, a 7×7 convolution layer is used to compress the number of channels to 1 to obtain the feature map.

[0061] For example, for the feature map F∈R generated by the network backbone C×H×W , obtain the 1-dimensional channel attention feature map M through the attention algorithm C ∈R C×1×1 , 2D spatial attention feature map M S ∈R 1×H×W .

[0062] The above is a method for detecting small targets in drone images provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding device for detecting small targets in drone images, including:

[0063] A construction module is used to construct an improved single-shot multi-border detection SSD algorithm. Based on the original single-shot multi-border detection SSD algorithm, a self-attention mechanism is added after global feature extraction through the backbone network VGG-16, and an attention mechanism is added before the detection head.

[0064] The training module is used to train the improved single-shot multi-border detection SSD algorithm to obtain a detection model for detecting small targets in drone images;

[0065] The detection module is used to input the drone image to be detected into the detection model, obtain relevant data in the global features through the self-attention mechanism, establish the feature correlation of the drone image, fuse the features of different scales, and obtain the fused feature map. Then, based on the attention mechanism, the fused feature map is serialized in the channel and space dimensions, and the serialized feature map is multiplied with the feature map of the original input to obtain the target image, and the target image is detected by the detection head.

[0066] For the specific definition of the small target detection device for drone images, please refer to the definition of the small target detection method for drone images above, which will not be repeated here. Each module in the above-mentioned small target detection device for drone images can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0067] The present invention also provides a computer-readable storage medium, which stores a computer program, and the computer program can be used to execute the small target detection method for drone images provided above.

[0068] The present invention also provides Figure 4 The structural diagram of the computer device shown in FIG. Figure 4 As shown, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the small target detection method for drone images provided in the above embodiment.

[0069] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A small target detection method for drone images, characterized in that: include: Construct an improved single-shot multi-border detection SSD algorithm; based on the original single-shot multi-border detection SSD algorithm, add a self-attention mechanism after global feature extraction through the backbone network VGG-16, and add an attention mechanism before the detection head; Train the improved single-shot multi-border detection (SSD) algorithm to obtain a detection model for detecting small targets in drone images; The drone image to be detected is input into the detection model, and the relevant data in the global features is obtained through the self-attention mechanism. The feature correlation of the drone image is established, and the features of different scales are fused to obtain the fused feature map. Then, based on the attention mechanism, the fused feature map is serialized in the channel and space dimensions. The serialized feature map is multiplied with the feature map of the original input to obtain the target image, and the target image is detected by the detection head.

2. The small target detection method for drone images as claimed in claim 1, characterized in that: The self-attention mechanism is used to obtain relevant data in the global features, establish feature correlation of drone images, and fuse features of different scales, specifically including: Based on the features of different scales in the global features, the feature matrix is ​​determined by a lightweight convolution kernel; According to the feature matrix, the global weighted feature mapping matrix is ​​determined by matrix multiplication and softmax function, and the global weighted feature mapping matrix is ​​multiplied by the feature matrix to obtain the weight value matrix of all positions in the same feature map; Features of different scales are fused according to the original features and the weighted value matrix.

3. The small target detection method for drone images as claimed in claim 1, characterized in that: The global features specifically include: Deep features and shallow features; the deep features include semantic features; the shallow features include edge features and texture features.

4. The small target detection method for drone images as claimed in claim 1, characterized in that: The fused feature map is serialized in two dimensions, channel and space, and then the serialized feature map is multiplied with the original input feature map to obtain the target image, specifically including: The channel attention module is used to perform maximum pooling and average pooling operations on the drone image in the channel dimension, and then the two results are concatenated and normalized to obtain the weight matrix on the channel; The spatial attention module is used to perform maximum pooling and average pooling operations on the drone image in the spatial dimension, and the two results are spliced. According to the weight matrix on the channel, a 7×7 convolution layer is passed to compress the number of channels to 1 to obtain the target image.

5. A small target detection device for drone images, characterized in that: include: A construction module is used to construct an improved single-shot multi-border detection SSD algorithm. Based on the original single-shot multi-border detection SSD algorithm, a self-attention mechanism is added after global feature extraction through the backbone network VGG-16, and an attention mechanism is added before the detection head. The training module is used to train the improved single-shot multi-border detection SSD algorithm to obtain a detection model for detecting small targets in drone images; The detection module is used to input the drone image to be detected into the detection model, obtain relevant data in the global features through the self-attention mechanism, establish the feature correlation of the drone image, fuse the features of different scales, and obtain the fused feature map. Then, based on the attention mechanism, the fused feature map is serialized in the channel and space dimensions, and the serialized feature map is multiplied with the feature map of the original input to obtain the target image, and the target image is detected by the detection head.

6. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting small targets in drone images as described in any one of claims 1 to 4 above is implemented.

7. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting small targets in the drone image as claimed in any one of claims 1 to 4 is implemented.

Images