Unmanned aerial vehicle small target detection method based on deep learning

Abstract

The invention relates to the technical field of deep learning image recognition, in particular to an unmanned aerial vehicle small target detection method based on deep learning, and the method comprises the steps: embedding a parallel block perception attention module in a C2f module of YOLOv8; a large separable nuclear attention mechanism is fused in the SPPF module of the YOLOv8; the original nearest neighbor interpolation is replaced by CARAFE dynamic up-sampling; the Soft-NMS is adopted to replace the NMS; reconstructing a Neck part of the backbone network; a dynamic target detection head is combined with a scale sensing module, a space sensing module and a task sensing module; a WFS-IoU loss function fusing Wise-IoU, Focaler-IoU and Shape-IoU is designed, aiming at the core bottlenecks of feature loss, poor scale adaptability, dense target processing defects and the like in small target detection of the unmanned aerial vehicle, the balance between precision and efficiency is realized through multi-module collaborative optimization and lightweight design, and the detection accuracy is improved. And an efficient and reliable detection solution is provided for aerial photography application of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning image recognition, and in particular to a method for detecting small targets in unmanned aerial vehicles (UAVs) based on deep learning. Background Art

[0002] Drone aerial photography technology has been widely used in fields such as disaster relief and urban management. However, the objects captured in these high-altitude images are extremely small and densely distributed. Traditional object detection algorithms face significant challenges in such scenarios: small objects suffer from severe information loss during multiple downsampling processes, resulting in inadequate feature extraction and a significant decrease in detection accuracy.

[0003] Specifically, small objects from a drone's perspective account for a very small percentage of pixels in an image. After multiple layers of downsampling in the network, key details are blurred or even completely lost. Existing methods attempt to mitigate this problem by adding detection layers or introducing attention mechanisms, but they lack systematic design:

[0004] Traditional attention modules do not jointly model the local details and long-distance dependencies of small targets, resulting in insufficient feature fusion; fixed pooling operations (such as SPPF modules) are difficult to dynamically capture multi-scale contextual information, further exacerbating the feature degradation of small targets; feature maps generated by upsampling methods such as nearest neighbor interpolation have jagged problems and insufficient detail recovery capabilities.

[0005] Furthermore, drones have limited onboard computing resources, and existing improvement solutions often lead to a surge in model parameters, making it difficult to balance accuracy and real-time performance. Therefore, an end-to-end optimization method specifically for small target detection scenarios using drones is needed. This method systematically addresses information loss through multi-module collaborative design, improving detection accuracy while meeting lightweight deployment requirements. Summary of the Invention

[0006] This invention addresses core bottlenecks in UAV small target detection, such as feature loss, poor scale adaptability, and dense target processing defects. Through multi-module collaborative optimization and lightweight design, it achieves a balance between accuracy and efficiency, providing an efficient and reliable detection solution for UAV aerial photography applications.

[0007] The present invention is achieved through the following technical solutions:

[0008] The deep learning-based UAV small target detection method includes the following steps:

[0009] S1. Embed a parallelized block-aware attention module in the C2f module of YOLOv8 to enhance the local-global feature fusion capability of small targets through multi-branch feature extraction and adaptive attention processing;

[0010] S2. Integrate a large separable kernel attention mechanism into the SPPF module of YOLOv8, and use spatial dilated convolution and separable kernel decomposition to expand the contextual perception range of multi-scale features;

[0011] S3, replace the original nearest neighbor interpolation with CARAFE dynamic upsampling, and reduce the loss of detail information caused by high-rate downsampling through content-aware kernel prediction and feature reorganization;

[0012] S4. Soft-NMS is used instead of NMS. Based on the Gaussian confidence decay strategy, it suppresses redundant detection boxes of dense small targets and improves the recall rate in occlusion scenarios.

[0013] S5, reconstruct the Neck part of the backbone network, build a multi-scale feature pyramid through the scale sequence feature fusion module, three-feature encoder and channel position attention mechanism, and add a P2 detection layer to capture the low-resolution features of the bottom layer;

[0014] S6: Combine the dynamic object detection head with the scale, space, and task perception modules to enhance the bounding box regression of small objects through global pooling, deformable convolution, and channel modeling.

[0015] S7. Design the WFS-IoU loss function that integrates Wise-IoU, Focaler-IoU and Shape-IoU, and combine the dynamic focusing mechanism, sample difficulty classification and shape sensitivity factor to optimize the positioning of small object bounding boxes.

[0016] Furthermore, in step S1, the parallelized block-aware attention module includes a local branch, a global branch, and a serial convolution branch, wherein the local branch extracts local features through non-overlapping patch aggregation, the global branch captures long-distance dependencies through displacement operations, and the convolution branch refines features through serial convolution; after the output features of the three branches are added and fused, they are adaptively enhanced through a one-dimensional channel attention mechanism and a two-dimensional spatial attention mechanism in turn.

[0017] Furthermore, in step S2, the large separable kernel attention mechanism decomposes the standard convolution kernel into separable kernels in the horizontal and vertical directions, extracts the horizontal and vertical spatial context information of the feature map respectively, and combines spatial dilation convolution with different dilation rates to expand the receptive field, and adaptively fuses features of different scales through a dynamic weight allocation mechanism.

[0018] Furthermore, in step S3, the CARAFE dynamic upsampling performs weighted reorganization on the local feature area by predicting the upsampling kernel related to the input feature content; the generation of the upsampling kernel includes feature map channel compression, kernel prediction and Softmax normalization steps. The upsampling process retains high-frequency detail information and performs dot product operation on the local area features through the feature reorganization module.

[0019] Furthermore, in step S4, the Soft-NMS method performs confidence attenuation on the candidate frames that overlap with the high-confidence detection frames. The attenuation amplitude is dynamically adjusted by a Gaussian function according to the degree of overlap, and the detection frames with confidence higher than a preset threshold are retained.

[0020] Furthermore, in step S5, the scale sequence feature fusion module normalizes and splices the feature maps of different downsampling ratios, and fuses multi-scale information through three-dimensional convolution; the three-feature encoder separates the large, medium and small size features and re-enlarges and fuses them to supplement the underlying detail information; the channel position attention mechanism jointly modulates the feature response through channel weights and spatial weights; the resolution of the P2 detection layer is 160×160 pixels.

[0021] Furthermore, in step S6, the dynamic target detection head includes a scale perception module, a spatial perception module and a task perception module, wherein the scale perception module dynamically adjusts the channel weight through global pooling and a fully connected layer, the spatial perception module captures the target deformation features through deformation convolution, and the task perception module optimizes the weight distribution of classification and regression tasks through a fully connected layer.

[0022] Furthermore, in step S7, the WFS-IoU loss function reduces the gradient influence of low-quality samples through a dynamic focusing mechanism, distinguishes easy-to-detect and difficult-to-detect targets through sample difficulty grading, and constrains the width-to-height ratio deviation between the predicted box and the true box through a shape-sensitive factor, thereby comprehensively improving the positioning accuracy of small targets.

[0023] A preferred embodiment includes a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, any one of the above-mentioned methods for detecting small targets of unmanned aerial vehicles based on deep learning is implemented.

[0024] Beneficial effects of the present invention:

[0025] (1) The deep learning-based small target detection method for UAVs proposed in this paper effectively integrates the local details of small targets with global context information through the collaborative design of parallelized block perception attention modules and large separable kernel attention mechanisms, significantly improving the feature expression ability of small targets.

[0026] (2) The deep learning-based small target detection method for drones proposed in this paper combines spatial dilation convolution with a dynamic weight allocation mechanism to expand the model's perception range for targets of different scales and enhance the detection capability of targets with varying scales from the drone's perspective.

[0027] (3) The deep learning-based small target detection method for drones proposed in this paper adopts a dynamic content-aware upsampling method to reduce the loss of high-frequency detail information, effectively improve the blur and jagged edges of small targets, and enhance contour clarity.

[0028] (4) The deep learning-based small target detection method for drones proposed in this paper dynamically adjusts the weights of overlapping detection frames through a confidence decay strategy, reducing missed detections and false detections in densely distributed or occluded scenes, and significantly improving the target recall rate in complex environments;

[0029] (5) The deep learning-based small target detection method for UAVs proposed in this paper integrates the loss function design of the dynamic focusing mechanism and the shape-sensitive factor to enhance the model's sensitivity to small target position deviations and achieve more accurate bounding box regression.

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0031] Figure 1 This is the overall flow chart of the deep learning-based UAV small target detection method proposed in this invention;

[0032] Figure 2 This is the PPA diagram of the parallelized block perception attention module of the deep learning-based UAV small target detection method proposed in this invention;

[0033] Figure 3 Bottleneck and P-Bottleneck structure diagrams of the deep learning-based UAV small target detection method proposed in this invention;

[0034] Figure 4 This is the C2f-PPA structure diagram of the deep learning-based UAV small target detection method proposed in this invention;

[0035] Figure 5 This is the LSKA structure diagram of the deep learning-based UAV small target detection method proposed in this invention;

[0036] Figure 6 This is the SPPF-LSKA structure diagram of the deep learning-based UAV small target detection method proposed in this invention;

[0037] Figure 7 This is the workflow diagram of the CARAFE upsampling method for the deep learning-based UAV small target detection method proposed in the present invention;

[0038] Figure 8 This is the scale sequence feature fusion SSFF graph of the deep learning-based UAV small target detection method proposed in this invention;

[0039] Figure 9 This is the three-feature encoder TFE diagram of the deep learning-based UAV small target detection method proposed in this invention;

[0040] Figure 10 This is the channel and position attention mechanism CPAM diagram of the deep learning-based UAV small target detection method proposed in this invention;

[0041] Figure 11 This is a schematic diagram of the Neck structure reconstruction of the UAV small target detection method based on deep learning proposed in the present invention and the Neck network structure after adding the small target detection layer;

[0042] Figure 12 This is the Dyhead structure diagram of the deep learning-based UAV small target detection method proposed in this invention;

[0043] Figure 13 This is a visualization diagram of the bounding box function of the deep learning-based UAV small target detection method proposed in this invention;

[0044] Figure 14 This is a WFS-IoU performance comparison chart of the deep learning-based UAV small target detection method proposed in this paper;

[0045] Figure 15 This is a schematic diagram of the terminal device of the deep learning-based UAV small target detection method proposed in the present invention;

[0046] Figure 16 A schematic diagram of a readable storage medium for the deep learning-based small target detection method for drones proposed in the present invention;

[0047] In the figure, 200 - terminal device, 210 - memory, 211 - RAM, 212 - cache memory, 213 - ROM, 214 - program / utility, 215 - program module, 220 - processor, 230 - bus, 240 - external device, 250 - I / O interface, 260 - network adapter, 300 - program product. DETAILED DESCRIPTION

[0048] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with examples and drawings. The exemplary embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0049] Example 1

[0050] The deep learning-based small target detection method for drones proposed in this embodiment includes the following steps:

[0051] 1. A parallelized block-aware attention module is embedded in the C2f module of YOLOv8. Through multi-branch feature extraction and adaptive attention processing, the local-global feature fusion capability of small objects is enhanced.

[0052] In small object detection, due to their small size, often accompanied by blurred outlines and fewer pixels, there is a significant risk of information loss during multiple downsampling processes, significantly reducing feature extraction for small objects. Therefore, the PPA (Parallelized Block Perception Attention) module, introduced in the HCF-Net infrared small target network architecture, replaces traditional convolution operations in the basic components of the encoder and decoder. This effectively addresses the issue of key information loss during downsampling operations, helping to solve this problem for small objects.

[0053] The functional principle of PPA is proposed in this embodiment, specifically including:

[0054] (1) Multi-branch feature extraction, the main advantage of PPA lies in its multi-branch feature extraction strategy. Figure 2 As shown in Figure 1, PPA adopts a parallel multi-branch approach, where each branch is responsible for extracting features of different scales and levels. This multi-branch strategy is conducive to capturing the multi-scale features of objects, thereby improving the accuracy of small object detection. The multi-branch strategy includes three parallel branches: local branch, global branch, and serial convolution branch. Given an input feature tensor After that, we first adjust the convolution point by point to get Then through these three branches, we can calculate and Finally, add these three results together to get The distinction between local and global branches is achieved by controlling the patch size parameter p, which is achieved by aggregating and shifting non-overlapping patches in the spatial dimension. Furthermore, this embodiment calculates an attention matrix between non-overlapping patches to enable local and global feature extraction and interaction.

[0055] (2) Feature fusion and attention, after feature extraction through multi-branch feature extraction, the attention mechanism is used to perform adaptive feature enhancement. The attention module consists of a series of efficient channel attention and spatial attention. In this case Sequentially from the one-dimensional channel attention map and a two-dimensional spatial attention map This process can be summarized as follows:

[0056]

[0057] in, represents element-wise multiplication, and represents the characteristics after channel and space selection, δ(·) and Represent the rectified linear unit ReLU and batch normalization BN respectively, is the final output of PPA, the P-Bottleneck structure is as follows Figure 3 shown.

[0058] The PPA module is integrated with the C2f module in YOLOv8 to obtain the C2f-PPA model, which improves the feature extraction and feature fusion capabilities of the YOLOv8 algorithm for small targets. The specific structure is as follows Figure 4 shown.

[0059] 2. Integrate a large separable kernel attention mechanism into the SPPF module of YOLOv8, using spatially dilated convolution and separable kernel decomposition to expand the contextual perception range of multi-scale features;

[0060] The original SPPF in YOLOv8 consists of three consecutive pooling layers, combining the outputs of each layer to ensure multi-scale fusion. While this reduces computational complexity and significantly improves speed compared to SPP, achieving adaptively sized outputs, it struggles to effectively integrate small, fine-grained features, often associated with small objects. Therefore, the LSKA large separable kernel attention mechanism was introduced and combined with SPPF, significantly enhancing the model's multi-scale feature extraction capabilities.

[0061] The structure and function of the LSKA in this embodiment are divided into the following steps:

[0062] (1) Initialize the convolutional layers. The two convolutional layers are responsible for extracting the horizontal and vertical features of the input feature map respectively. This step is to generate a preliminary attention map to help the model focus on the important parts of the image.

[0063] (2) Spatially dilated convolutional layers: After obtaining the initial attention map, LSKA uses spatially dilated convolutions with different dilation rates to further extract features. These convolutional layers can cover a larger receptive field without increasing computational cost, capturing a wider range of contextual information. By operating in the horizontal and vertical directions respectively, they can process image features more finely and enhance the model's understanding of the spatial relationship of the image.

[0064] (3) Fusion and application of attention. After the convolution operation, LSKA fuses the features obtained through the last convolution layer to generate the final attention map. The attention map is multiplied with the original input feature map at the element level, that is, the attention mechanism is applied so that each element in the original feature map is weighted according to the value of the attention map, highlighting important features and suppressing unimportant features.

[0065] The LSKA large-kernel separable attention mechanism is a design optimized for the attention module in convolutional neural networks. Its core concept is to decompose the 2D convolution kernels of the deep convolutional layer into cascaded horizontal 1D and vertical 1D kernels, allowing the direct use of the large kernels of the deep convolutional layer in the attention module. The advantage of LSKA is that it can enhance the model's ability to capture long-range dependencies through larger convolution kernel sizes. At the same time, due to the use of separable convolutions, it does not significantly increase the number of model parameters and computational complexity like full-size convolutions. By utilizing large and separable convolution kernels and spatially dilated convolutions to capture extensive contextual information of the image, an attention map is generated, and this attention map is used to weight the original features, thereby enhancing the network's focus on important features and improving model performance.

[0066] In this embodiment, the LSKA attention mechanism is placed between Concat and Conv of the SPPF module to form a new SPPF-LSKA module. The LSKA attention mechanism is applied to SPPF to perform weighted processing on multi-scale feature maps. Specifically, LSKA first calculates the spatial weight and channel weight on each feature map, and then adjusts the feature map according to these weights. The model can pay more attention to the features related to the target, thereby improving the detection accuracy. The LSKA structure is as follows Figure 5 As shown in Figure 6 shown.

[0067] The specific output formula of LSKA is as follows:

[0068]

[0069] A C =W 1×1 *Z C

[0070]

[0071] in represents the output of depthwise convolution, Z C It is the output of the depthwise convolution obtained by convolving a kernel of size k×k with the input feature map. 2d-1 and k / d represent the size of the convolution kernel, and * represents convolution. Represents Hadamard.

[0072] It should be clear that each channel C in F is convolved with the corresponding channel in kernel W. The output T of LSKA C is the attention map A C And the input feature map F C The Hadamard product of .

[0073] 3. CARAFE dynamic upsampling replaces the original nearest neighbor interpolation, and reduces the loss of detail information caused by high-rate downsampling through content-aware kernel prediction and feature reorganization;

[0074] YOLOv8 uses an upsampling operator to perform upsampling operations, and uses the nearest neighbor interpolation method for upsampling. The nearest neighbor interpolation uses the nearest distance between pixels to represent upsampling, that is, the grayscale value of the transformed pixel is equal to the grayscale value of the original feature map pixel closest to it. However, this method only considers the relationship between adjacent pixels and ignores the contextual semantic information of the feature map. This results in different grayscale values ​​of image pixels generated at the same point in the original image when calculating the nearest neighbor value, and the image has jagged edges, resulting in the loss of image detail information and the lack of fine-grained feature expression capabilities. This will cause the algorithm to lose some of the feature extraction details, which is not conducive to small target detection. In this embodiment, the CARAFE upsampling method is introduced to dynamically adjust the upsampling kernel to adapt to different upsampling requirements, so as to better retain and extract feature information. While maintaining computational efficiency, it can also reduce information loss. In addition, by fusing feature maps of different scales, richer contextual information can be obtained to enhance the feature expression capabilities of the network, thereby improving the perception range and positioning accuracy of the algorithm. The specific implementation process is as follows: Figure 7 shown.

[0075] In the upsampling kernel prediction part, the input feature map of size H×W×C is first compressed by a 1×1 Conv module to reduce the number of channels of the input feature map to C. m , and then the number of output channels is changed to Where σ represents the upsampling rate and the upsampling kernel size is k up ×k up , if a different upsampling kernel is used for each position of the output feature map, the predicted upsampling kernel size is Finally, the obtained up-sampling kernel is normalized by Softmax so that the weight sum of the convolution kernel is 1. For the feature reorganization module, each position in the output feature map is mapped back to the input feature map, and the k positions centered on it are taken out. up ×k upThe region is subjected to a dot product operation with the predicted upsampling kernel of the point. Different channels at the same position share the same upsampling kernel. For each recombinant kernel, the content-aware recombinant module recombines the features of the local region. For the target position l′ in the output feature map and the square area N(χ l , k up ), the reorganization formula is as follows:

[0076]

[0077] where r = [k up / 2],N(χ l , k up ) is dot-producted with the upsampling kernel corresponding to this point. Each channel at the same position uses the same upsampling kernel, that is, the values ​​of the upsampling pixels are rearranged, and N(χ l , k up ) contributes differently to the upsampling kernel, making different contributions based on the content of the feature map. After the upsampling operation is completed, the output feature map result is of shape σH×σW×C.

[0078] 4. Soft-NMS is used instead of NMS. Based on the Gaussian confidence decay strategy, it suppresses redundant detection boxes of dense small objects and improves the recall rate in occlusion scenarios.

[0079] YOLOv8 is an anchor-free model that directly predicts the center of an object rather than locating it by predicting the offset of an anchor box. This means that YOLOv8 directly predicts the center of an object rather than the offset of a known anchor box. By reducing the number of box predictions, this new approach accelerates a very complex inference step, called non-maximum suppression (NMS). However, accuracy cannot be fully guaranteed for small objects with dense occlusions.

[0080] This example introduces Soft-NMS, an improvement on the traditional NMS algorithm. It suppresses the detection of overlapping objects by reducing the confidence score of overlapping objects, thereby improving the accuracy of object detection. A different hyperparameter is introduced to determine the degree of discount of the confidence score of overlapping detections through Gaussian distribution.

[0081] Soft-NMS differs from NMS in how it handles redundant boxes. NMS directly deletes redundant boxes, while Soft-NMS reduces the confidence of redundant boxes and performs another thresholding on the boxes on the candidate list. Since only the confidence reduction is performed on redundant boxes, boxes with non-zero confidence levels will fall into the candidate list. At this point, when the boxes in the candidate list are processed once using the threshold, boxes below the threshold are deleted. For the algorithm, only overlapping boxes with sufficiently strong confidence output by the model are considered to have overlapping objects nearby.

[0082] Specifically, Soft-NMS receives the IOU of two anchors and imposes a penalty, modifying the anchor confidence according to the degree of penalty: when the IOU is 0, no penalty is imposed; when the IOU is large, the penalty is increased; when the IOU is small, the penalty is gradually increased in ascending order of the IOU size. The calculation logic of NMS and Soft-NMS is as follows:

[0083]

[0084] The specific steps of Soft-NMS are as follows:

[0085] (1) Sort the anchors according to their confidence scores;

[0086] (2) Add the anchor with the highest score to the output list and delete it from the bounding box list;

[0087] (3) Calculate the IoU between the anchor with the highest score in step (2) and other anchors;

[0088] (4) If the IoU exceeds the set threshold, reduce the confidence score of the anchor (not the anchor with the highest score in (2)) and remove the anchors with confidence scores lower than the threshold;

[0089] (5) Repeat the above steps until the bounding box list is cleared.

[0090] 5. Reconstruct the Neck part of the backbone network, build a multi-scale feature pyramid through the scale sequence feature fusion module, three-feature encoder and channel position attention mechanism, and add a P2 detection layer to capture the low-resolution features of the bottom layer;

[0091] ASF-YOLO, a YOLO framework based on attention-based scale sequence fusion, combines spatial and scale features. Based on the YOLO segmentation framework, it extracts a scale sequence feature fusion (SSFF) module to enhance the network's multi-scale information extraction capabilities. A three-feature encoder (TPE) module is used to fuse feature maps of different scales to increase detailed information. Finally, a channel and position attention mechanism (CPAM) is introduced, integrating the SSFF and TPE modules. This module focuses on small objects related to information channels and spatial positions to improve detection performance for small objects.

[0092] The scale sequence feature fusion (SSFF) module in this embodiment provides complementary information for small object segmentation, combining global or high-level semantic information from multiple scale images. The TFE module captures fine local details of small objects, separating large, medium, and small features, adding a larger feature map, and then performing feature amplification to enhance detailed feature information. Combining local and global feature information produces a more accurate segmentation map. The channel and position attention mechanism (CPAM) consists of two parts: the first is a channel attention network, which receives input from TFE input 1 and extracts representative feature information contained in different channels. The second is a position attention network, which receives and superimposes the outputs from the channel attention network and SSFF input 2 to introduce position information. In this way, CPAM is able to fuse different attention mechanisms and comprehensively utilize channel and position information to improve object recognition performance.

[0093] like Figure 8 As shown in the figure, the scale sequence feature fusion SSFF corresponds to the ScalSeq structure. The feature maps of P3, P4 and P5 are normalized to the same size, upsampled and then stacked together as the input of 3D convolution to fuse multi-scale features.

[0094] The three-feature encoder TFE corresponds to the Zoom_cat structure. For the Zoom_cat module, in order to obtain rich information at each feature layer, the module splits the large, medium, and small features, processes the large-scale feature map, and amplifies the features to improve the detailed feature information. Figure 9 As shown in Figure 2, the structure of the module is shown, where C represents the number of channels, S represents the feature map size, and each Zoom_cat uses three feature maps of different sizes as input.

[0095] like Figure 10 As shown in the figure, the position attention mechanism CPAM corresponds to the asf_attention structure, and extracts relevant representative information by fusing the channel and position attention mechanisms. The network receives input from the superposition of the channel attention network output and the SSFF input 2.

[0096] Small objects typically occupy a limited number of pixels in an image, making them easily overlooked or misclassified. To improve YOLOv8's efficiency in small object detection, the P2 small object detection layer was introduced. The P2 layer's detection head has a resolution of 160×160 pixels, equivalent to only two downsampling operations in the backbone network. This layer contains richer underlying feature information about the target, facilitating the detection of small objects.

[0097] The scale sequence feature fusion SSFF, three-feature encoder TFE and channel and position attention mechanism CPAM are integrated with the YOLOv8 algorithm structure, and the P2 layer with fewer convolutions is added to make the feature map larger in size, which is more conducive to small target recognition, resulting in the ASF-P2 structure. The output feature map is continuously fused with the feature maps of P3, P4 and P5 through downsampling operations in the structure after Neck reconstruction optimization design, providing more position information for these feature maps and improving the overall prediction accuracy of the network. The improved network structure is as follows Figure 11 shown.

[0098] Since the asf_attention module can be added to two locations in the algorithm, namely the 25th and 30th layers, the performance of these two layers with or without the asf_attention module is tested. There are four cases in total, namely ASF-P2, ASF-P2(25), ASF-P2(30), and ASF-P2(25, 30), as shown in Table 1. As can be seen from Table 1, the best effect is achieved when the asf_attention module is placed in the 25th layer, that is, ASF-P2(25), so ASF-P2(25) is used as the final algorithm structure arrangement.

[0099]

[0100] Table 1 ASF-P2 structural attention mechanism position performance change table

[0101] 6. Combining the dynamic object detection head with the scale, space, and task perception modules, we enhance the bounding box regression of small objects through global pooling, deformable convolution, and channel modeling.

[0102] Small targets may be partially or completely obscured by other objects or the background, making observation of the target challenging. This highlights the importance of the detection algorithm's perception capabilities. To address this issue, this embodiment introduces a dynamic head (Dyhead), which combines the target detection head with an attention mechanism.

[0103] This method effectively integrates multiple self-attention mechanisms into the scale-aware feature layer, spatial-aware spatial position module, and task-aware output channel module, thereby improving the representational capabilities of the object detection head. This enhancement significantly improves the model's ability to describe objects, making the detection of small objects more flexible and accurate. By combining multiple self-attention mechanisms with the feature layers of the scale-aware module, spatial-aware spatial position module, and task-aware output channel module, this method significantly improves the feature description capabilities of the object detection head.

[0104] The core principle of the DyHead algorithm lies in the unified attention mechanism for feature maps, through the scale perception module π L , spatial perception module π S and task perception module π C The serial processing of π can achieve full attention to the detection head. L Perform global pooling, 1×1 convolution, ReLU activation, and hard sigmoid activation. AdaptiveAvgPool2d is responsible for performing global pooling on H×W to obtain the maximum value of the feature map. Convolution is responsible for integrating all channels together to form a number. S Perform deformation convolution V2 (offset offset + feature amplitude modulation) operation. π C A two-layer fully connected neural network is used for channel modeling, as shown in the following diagram: Figure 12 shown.

[0105] 7. Design the WFS-IoU loss function that integrates Wise-IoU, Focaler-IoU, and Shape-IoU, and combines the dynamic focusing mechanism, sample difficulty classification, and shape sensitivity factor to optimize the positioning of small object bounding boxes;

[0106] In traditional object detection tasks, the commonly used loss function is IoU (Intersection over Union), which is used to measure the degree of overlap between the predicted box and the ground-truth box. However, IoU has some problems, such as poor detection of small objects and insensitivity to highly compressed bounding boxes. To address these problems, this embodiment combines the characteristics of Wise-IoU, Focaler-IoU, and Shape-IoU to design a new WFS-IoU loss function for small objects.

[0107] (1) Wise-IoU. Compared to the original YOLOv8’s Complete-IoU (CIoU), which adds aspect ratio similarity to evaluate the quality of anchor boxes, WiseIoU adds a new gradient gain strategy. WiseIoU is divided into WiseIoU-v1, WiseIoU-v2, and WiseIoU-v3 according to different gradient gain strategies. WiseIoU-v1 does not interfere too much with training when the anchor box and the target box overlap well, and constructs a distance attention mechanism based on the distance metric, combining aspect ratio similarity to form a bounding box loss function with a two-layer attention mechanism. The core idea of ​​Wise-IoU is to use the outlier degree of the anchor box, that is, the relative position of the sample and other samples in the feature space, to evaluate the quality of the sample, and dynamically adjust the weights of each part of the loss function based on the quality.

[0108] The calculation formula of WIoU-v1 is as follows:

[0109]

[0110] in, is the original bounding box loss function, w, h are the width and height of the prediction box; w gt , h gt is the actual frame width and height; W i H i is the width and height of the overlapping part of the predicted box and the real box; x, y are the coordinates of the predicted center point; x gt ,y gt Target center coordinates; W g H g is the width and height of the minimum bounding box; is the distance attention mechanism function, Figure 13 Shown is a bounding box function visualization.

[0111] WIoU-v2 and WIoU-v3 construct a gradient gain-based calculation method based on WIoU-v1 and add a focusing mechanism, among which WIoU-v2 constructs a monotonic focusing coefficient Used to adjust the anchor box weight.

[0112] The calculation formula of WIoU-v2 is as follows:

[0113]

[0114] WIoU-v3 uses outlier to describe the quality of the anchor frame. A small outlier indicates a high-quality anchor frame, and a small gradient gain is assigned to it. Anchor frames with a large outlier are assigned a large gradient gain, thereby constructing a dynamic non-monotonic focusing coefficient.

[0115] The calculation formula of WIoU-v3 is as follows:

[0116]

[0117] (2) Focaler-IoU. Bounding box regression plays a vital role in the field of object detection, and the positioning accuracy of object detection depends largely on the loss function of bounding box regression. Existing research improves regression performance by utilizing the geometric relationship between bounding boxes, but ignores the impact of sample difficulty distribution on bounding box regression. Focaler-IoU can improve the performance of detectors in different detection tasks by focusing on regression samples of different difficulty levels.

[0118] In order to focus on different regression samples and be able to focus on different detection tasks, a linear interval mapping method is used to reconstruct the IoU loss, which allows for improved edge regression. The formula is as follows:

[0119]

[0120] Among them, IoU Focaler is the reconstructed Focaler-IoU, IoU is the original IoU value, [d,u]∈[0,1]. By adjusting the values ​​of d and u, IoU can be Focaler The focus is on different regression samples. Its loss is defined as follows:

[0121] L Focaler-IoU =1-IoU Focaler

[0122] (3) Shape-IoU: Based on the current academic research and analysis on the characteristics of bounding box regression, it is found that the shape and scale factors of the bounding box itself will affect the regression results. Since small target detection and recognition have high requirements for the size of IoU and the bounding box, the Shape-IoU method is introduced. It can calculate the loss by focusing on the shape and scale of the bounding box itself, thereby making the bounding box regression more accurate.

[0123] When calculating the loss, the Shape-IoU loss function first calculates the overlapping area between the ground-truth bounding box and the predicted box (Area of ​​Overlap). It then calculates the total area of ​​the combined ground-truth bounding box and the predicted box (Area of ​​Union). It then calculates the ratio of the overlapping area to the total area (i.e., the Intersection of Union (IoU) value), yielding a value between 0 and 1 representing the degree of overlap between the two bounding boxes. Finally, the IoU value is converted into a loss value through a specific method (such as 1-IoU value) to guide model training and optimization.

[0124] The definition formula of ShapeIoU is as follows:

[0125]

[0126]

[0127] L shopeIoU =1-IoU+distance shape +0.5×Ω shape

[0128] Among them, scale is the scaling factor, which is related to the size of the target in the dataset, ww and hh represent the weight coefficients in the horizontal and vertical directions of the prediction box respectively, and θ is the weight coefficient of the shape loss (Ω shape ) is a factor that normalizes the degree of .

[0129] (4) WFS-IoU, which integrates the dynamic focusing mechanism of WiseIoU, the ability of FocalerIoU to focus on regression samples of different difficulties, and the ability of ShapeIoU to focus on the shape and scale of the bounding box itself, is the WFS-IoU proposed in this embodiment.

[0130] The core formula of WFS-IoU is as follows:

[0131] L Wise-Focaler-ShopeIoU =r×L ShapeIoU +IoU-IoU FocalerIoU

[0132] Among them, r is the dynamic focusing coefficient in Wise-IoU, L ShapeIoU is the Loss of ShapeIoU, IoU is the intersection-over-union ratio between the predicted bounding box and the true bounding box, IoU Focaler is the reconstructed Focaler-IoU.

[0133] The performance of WFS-IoU versions WIoU-v1, WIoU-v2, and WIoU-v3 was tested on the YOLOv8s model and verified based on the VisDrone2019 dataset. The performance comparison is shown in Table 2.

[0134]

[0135]

[0136] Table 2 Performance changes of various versions of WFS-IoU

[0137] Specific WFS-IoU performance comparison Figure 14 As shown in the figure, it can be seen that the WFS-IoU-v2 strategy has the best effect, and is the best in many performance indicators such as accuracy, map@0.5, and map@0.5:0.95. Therefore, WFS-IoU-v2 is used as the loss function version.

[0138] Example 2

[0139] like Figure 15 Based on Example 1, this embodiment proposes a terminal device for a drone small target detection method based on deep learning. The terminal device 200 includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0140] The memory 210 may include a readable medium in the form of a volatile memory, such as a RAM 211 and / or a cache memory 212 , and may further include a ROM 213 .

[0141] Among them, the memory 210 also stores a computer program, which can be executed by the processor 220, so that the processor 220 executes any of the above-mentioned methods for detecting small targets in drones based on deep learning in the embodiments of the present application. Its specific implementation method is consistent with the implementation method and the technical effect achieved in the embodiments of the above-mentioned applications, and some of the contents are not repeated here. The memory 210 may also include a program / utility 214 having a set (at least one) of program modules 215. Such program modules include but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each of these examples or some combination may include the implementation of a network environment.

[0142] Accordingly, the processor 220 may execute the aforementioned computer programs, as well as the program / utility 214 .

[0143] The bus 230 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0144] The terminal device 200 can also communicate with one or more external devices 240, such as keyboards, pointing devices, Bluetooth devices, etc., and can also communicate with one or more devices that can interact with the terminal device 200, and / or communicate with any device that enables the terminal device 200 to communicate with one or more other computing devices (such as routers, modems, etc.). Such communication can be carried out through the I / O interface 250. In addition, the terminal device 200 can also communicate with one or more networks (such as local area networks (LANs), wide area networks (WANs) and / or public networks, such as the Internet) through the network adapter 260. The network adapter 260 can communicate with other modules of the terminal device 200 through the bus 230. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in conjunction with the terminal device 200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0145] Example 3

[0146] This embodiment proposes a computer-readable storage medium for a deep learning-based drone small target detection method. Instructions are stored on the computer-readable storage medium. When the instructions are executed by the processor, the specific implementation method of implementing any of the above-mentioned deep learning-based drone small target detection methods is consistent with the implementation method and the technical effect achieved in the above-mentioned application embodiments, and some contents will not be repeated here.

[0147] Figure 16 The program product 300 provided in this embodiment for implementing the above-mentioned application is shown. It can adopt a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product 300 of the present invention is not limited to this. In this embodiment, the readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in conjunction with an instruction execution system, device or device. The program product 300 can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0148] A computer-readable storage medium may include a data signal transmitted in baseband or as part of a carrier wave, carrying readable program code. This transmitted data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which can transmit, transmit, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof. The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a standalone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. Where a remote computing device is involved, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0149] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting small targets of drones based on deep learning, characterized by: The following steps are involved: S1. Embed a parallelized block-aware attention module in the C2f module of YOLOv8 to enhance the local-global feature fusion capability of small targets through multi-branch feature extraction and adaptive attention processing; S2. Integrate a large separable kernel attention mechanism into the SPPF module of YOLOv8, and use spatial dilated convolution and separable kernel decomposition to expand the contextual perception range of multi-scale features; S3, replace the original nearest neighbor interpolation with CARAFE dynamic upsampling, and reduce the loss of detail information caused by high-rate downsampling through content-aware kernel prediction and feature reorganization; S4. Soft-NMS is used instead of NMS. Based on the Gaussian confidence decay strategy, it suppresses redundant detection boxes of dense small targets and improves the recall rate in occlusion scenarios. S5, reconstruct the Neck part of the backbone network, build a multi-scale feature pyramid through the scale sequence feature fusion module, three-feature encoder and channel position attention mechanism, and add a P2 detection layer to capture the low-resolution features of the bottom layer; S6: Combine the dynamic object detection head with the scale, space, and task perception modules to enhance the bounding box regression of small objects through global pooling, deformable convolution, and channel modeling. S7. Design the WFS-IoU loss function that integrates Wise-IoU, Focaler-IoU and Shape-IoU, and combine the dynamic focusing mechanism, sample difficulty classification and shape sensitivity factor to optimize the positioning of small object bounding boxes.

2. The method for detecting small targets of drones based on deep learning according to claim 1, characterized in that: In step S1, the parallelized block-aware attention module includes a local branch, a global branch, and a serial convolution branch, wherein the local branch extracts local features through non-overlapping patch aggregation, the global branch captures long-distance dependencies through displacement operations, and the convolution branch refines features through serial convolution; after the output features of the three branches are added and fused, they are adaptively enhanced through a one-dimensional channel attention mechanism and a two-dimensional spatial attention mechanism in turn.

3. The method for detecting small targets of drones based on deep learning according to claim 1, characterized in that: In step S2, the large separable kernel attention mechanism decomposes the standard convolution kernel into separable kernels in the horizontal and vertical directions, extracts the horizontal and vertical spatial context information of the feature map respectively, and combines spatial dilated convolution with different dilation rates to expand the receptive field, and adaptively fuses features of different scales through a dynamic weight allocation mechanism.

4. The method for detecting small targets of drones based on deep learning according to claim 1, characterized in that: In step S3, the CARAFE dynamic upsampling performs weighted reorganization on the local feature area by predicting the upsampling kernel related to the input feature content; the generation of the upsampling kernel includes feature map channel compression, kernel prediction and Softmax normalization steps. The upsampling process retains high-frequency detail information and performs dot product operation on the local area features through the feature reorganization module.

5. The method for detecting small targets of drones based on deep learning according to claim 1, characterized in that: In step S4, the Soft-NMS method performs confidence attenuation on the candidate boxes that overlap with the high-confidence detection box. The attenuation amplitude is dynamically adjusted through a Gaussian function according to the degree of overlap, and the detection boxes with confidence higher than a preset threshold are retained.

6. The method for detecting small targets of drones based on deep learning according to claim 1, characterized in that: In step S5, the scale sequence feature fusion module normalizes and splices the feature maps of different downsampling ratios, and fuses multi-scale information through three-dimensional convolution; the three-feature encoder separates the large, medium and small size features and re-enlarges and fuses them to supplement the underlying detail information; the channel position attention mechanism jointly modulates the feature response through channel weights and spatial weights; the resolution of the P2 detection layer is 160×160 pixels.

7. The method for detecting small targets of drones based on deep learning according to claim 1, characterized in that: In step S6, the dynamic target detection head includes a scale perception module, a spatial perception module and a task perception module, wherein the scale perception module dynamically adjusts the channel weight through global pooling and a fully connected layer, the spatial perception module captures the target deformation features through deformation convolution, and the task perception module optimizes the weight distribution of classification and regression tasks through a fully connected layer.

8. The method for detecting small targets of drones based on deep learning according to claim 1, characterized in that: In step S7, the WFS-IoU loss function reduces the gradient influence of low-quality samples through a dynamic focusing mechanism, distinguishes easy-to-detect and difficult-to-detect targets through sample difficulty grading, and constrains the width-to-height ratio deviation between the predicted box and the true box through a shape-sensitive factor, thereby comprehensively improving the positioning accuracy of small targets.

9. UAV small target detection system based on deep learning, characterized by: The invention comprises a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, the method for detecting small targets of a drone based on deep learning according to any one of claims 1 to 8 is implemented.

Images