Unmanned aerial vehicle aerial target detection method and device, and storage medium

Abstract

The application discloses a kind of unmanned aerial vehicle aerial target detection method, equipment and storage medium, it is related to computer vision technical field.The method includes constructing detection model based on YOLO framework, replace the convolution module of feature pyramid in its backbone network with block local-global attention module, design triple feature fusion module in neck network, use dynamic detection head of fusion triple attention mechanism in detection head;Block local-global attention module decomposes the input feature map into position-weighted sub-blocks and fuses them, uses unified perception context attention to enhance each sub-block in parallel in compact feature space, while implicitly modeling global context, and finally restores the original structure through inverse mapping;Unmanned aerial vehicle aerial image dataset is used to train and verify the detection model.The application significantly improves the detection accuracy of small and dense targets while maintaining high-speed reasoning.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a UAV aerial target detection method, device and storage medium. BACKGROUND

[0002] The UAV high-altitude aerial technology has become an important means in the fields of large-scale geographic information collection, environmental monitoring, emergency rescue, etc. due to its flexibility and efficiency. However, the image data obtained by the UAV at high-altitude perspective has its own characteristics, mainly manifested in that the targets in the image are mostly small targets or even tiny targets, which brings severe challenges to the target detection task. Specifically, the small targets have low pixel ratio and weak apparent characteristics, and often present a state of dense distribution or even mutual overlap, further increasing the difficulty of accurate detection and identification.

[0003] Although the target detection algorithm based on deep learning has made significant progress in the general field in recent years and has been widely used in UAV aerial image analysis, the existing methods still have obvious deficiencies in dealing with the above-mentioned specific challenges of high-altitude perspective, mainly in two aspects:

[0004] Firstly, the detection algorithm specially designed for multi-scale tiny or extremely tiny targets at high-altitude perspective of UAV mostly comes from the field of remote sensing images, and the network structure is usually complex and has a large number of parameters. Although it can achieve excellent performance in precision, it is difficult to meet the strict requirements of real-time performance of the resource-limited UAV platform due to large amount of calculation and slow reasoning speed.

[0005] Secondly, although the lightweight detection model designed for real-time performance can realize high frame rate reasoning on the performance-limited embedded platform, the model capacity and feature expression ability are limited, and the recognition effect of the dense and tiny scale targets in the high-altitude aerial image is not good, and the detection accuracy and recall rate are difficult to meet the task requirements.

[0006] In addition, with the continuous expansion of the application scenarios of UAV, its working environment is becoming more and more complex and severe. The UAV system needs to operate stably under extreme weather conditions (such as rain, snow, fog), low light, and challenging scenarios such as target occlusion, complex noise background, and multi-scale target overlapping distribution. These factors significantly increase the difficulty of the detection task and put higher requirements on the robustness of the algorithm.

[0007] Currently, the optimization strategies for UAV aerial small target detection mainly focus on the following aspects:

[0008] Super-resolution reconstruction technology: through upsampling and detail reconstruction of low-resolution input images or feature maps, the appearance of the target is enlarged, and the distinguishability of small targets is improved.

[0009] Attention mechanism and context aggregation: Introduce attention modules such as channels, spaces, or Transformers in the network, or build context information extraction modules, to enhance the model's attention to key area features and its ability to model global and local semantic relationships, thereby improving the positioning and recognition of small targets.

[0010] Adaptive convolution operation: Use mechanisms such as deformable convolution to allow the convolution kernel to adaptively adjust according to the shape and spatial position of the target, to more accurately extract features of irregular or specific orientation small targets.

[0011] Multi-scale feature fusion: In the feature pyramid network or detection head, deeply fuse feature maps from different levels to fully utilize the detailed information of shallow networks and the semantic information of deep networks, achieving collaborative detection and precise positioning of multi-scale targets, especially small targets.

[0012] Loss function optimization: To address the issues of inaccurate positioning and confidence assessment bias when dealing with small targets using the original loss function of YOLO and other series models, introduce special loss terms for small targets (such as Focal Loss variants that focus on difficult samples, IoU variant losses that optimize bounding box regression, etc.) or adjust loss weights to strengthen the constraints and learning of small targets during model training.

[0013] In summary, the field of unmanned aerial vehicle high-altitude aerial target detection faces the core contradiction of balancing detection accuracy, real-time processing capability, and environmental robustness. Under the constraints of the inherent computing resources and energy consumption of the unmanned aerial vehicle platform, developing a specialized target detection algorithm that can simultaneously meet the requirements of high precision, high real-time performance, and strong robustness has become a key to promoting the large-scale application of this technology in practical scenarios and a technical bottleneck that needs to be broken through. SUMMARY

[0014] To address the above technical problems in the prior art, the purpose of the present application is to provide a method, device, and storage medium for unmanned aerial vehicle aerial target detection based on the YOLO real-time detection framework, by introducing a lightweight multi-scale feature fusion mechanism, a detection head optimized for small targets, and an enhanced context perception module, while maintaining high-speed inference, significantly improving the detection accuracy and robustness of small and dense targets in aerial images.

[0015] The present application solves the above technical problems by the following technical solutions: a method for unmanned aerial vehicle aerial target detection, comprising:

[0016] Obtain the unmanned aerial vehicle aerial image dataset and preprocess it;

[0017] A detection model is constructed, which is based on a YOLO framework, and a convolution module of a feature pyramid in a backbone network thereof is replaced by a block local-global attention module fused with uniform perception context attention, a triple feature fusion module is designed in a neck network, and a dynamic detection head fused with triple attention mechanisms is adopted in a detection head;

[0018] The block local-global attention module is used for decomposing an input feature map into position-weighted subgraph blocks and fusing them, uniformly enhancing each subgraph block in a compact feature space by using uniform perception context attention, implicitly modeling a global context, and finally restoring an original structure through reverse mapping.

[0019] The detection model is trained and verified using the unmanned aerial vehicle aerial image dataset, and a final detection model is obtained.

[0020] The final detection model is used to input unmanned aerial vehicle aerial images for target detection.

[0021] The application introduces the concepts of context learning and deformable convolution, constructs a block local-global attention module, and designs uniform perception context attention in the block local-global attention module, which significantly enhances the feature expression ability of small targets and significantly reduces the computational resource consumption caused by direct convolution of high-resolution images; a triple feature fusion module is designed in the neck network, which improves the feature extraction and learning ability of small-scale targets while ensuring that the forward reasoning speed is basically unaffected, thereby avoiding excessive resource consumption caused by multiple calculations of feature maps in the neck network.

[0022] Further, the block local-global attention module comprises a block fusion module, a uniform perception context attention module and an embedding fusion module.

[0023] The block fusion module is used for dividing an input feature map into a plurality of subgraph blocks, performing pixel-level weighting according to the spatial positions thereof, splicing all the weighted subgraph blocks along a channel dimension, and generating a compact fusion feature map.

[0024] The uniform perception context attention module is used for capturing anchor point weights of the compact fusion feature map in different directions through a context anchor point perception unit; extracting multi-scale and multi-directional key features by using heterogeneous convolution kernels through a multi-directional feature extraction unit; dynamically calculating mixed attention weights on channels and scales through a mixed attention unit; and adaptively fusing the anchor point weights and the key features according to the mixed attention weights by a feature weighting fusion unit, and outputting an enhanced feature map.

[0025] The embedded fusion module is configured to inversely disassemble the enhanced feature map in the channel dimension according to the order in the block division, and restore each subgraph block to its spatial position and size in the feature pyramid.

[0026] In this embodiment, the adaptive division strategy of the block local-global attention module balances between accelerating feature extraction and maintaining accuracy when processing high-resolution feature maps at the lower layers of the image pyramid, while effectively avoiding the excessive computational overhead caused by directly applying uniform perception context attention to large-size feature maps.

[0027] The uniform perception context attention module replaces the traditional dot product attention calculation with a fully convolutional network, and introduces a context anchor point perception mechanism on this basis, enhancing the joint modeling capability of local and global semantics, and significantly improving the inference speed due to its complete reliance on efficient convolution operations. The uniform perception context attention module integrates deformable convolution and multiple attention mechanisms in channels, space, and scale, enabling adaptive feature extraction for targets of different scales and shapes, and significantly enhancing the feature expression capability of small targets.

[0028] Further, the neck network is composed of three cascaded triple feature fusion modules, each of which adopts a sandwiched three-layer fusion structure integrating a dynamic deformable convolution unit, a depth separable convolution unit, and a dynamic upsampling unit to realize efficient fusion of cross-scale features.

[0029] Further, the feature extraction process of the neck network is as follows: first, the dynamic deformable convolution unit in the bottom triple feature fusion module adaptively captures the geometric morphology of small targets in the high-resolution feature maps at the bottom of the feature pyramid, extracts high-precision detail features, and forms a detail enhancement mapping;

[0030] Subsequently, the high-precision detail features are subjected to channel-by-channel separable convolution by the depth separable convolution unit, which retains key structural information while efficiently reducing the sampling, generating a transition feature map of intermediate scale;

[0031] Meanwhile, the dynamic upsampling unit is used to restore the spatial scale of the low-resolution feature map from the previous layer, so that its resolution is aligned with that of the bottom layer, thereby retaining more semantic details; the features output by the dynamic upsampling unit are then further input into the dynamic deformable convolution unit to enhance its geometric perception ability and context representation quality;

[0032] Finally, the three layers of scale-aligned feature maps are concatenated along the channel dimension to form an enhanced output that integrates multi-scale information, which is used as the input of the next triple feature fusion module or directly passed to the detection head.

[0033] In the embodiment, the TFF module benefits from the synergistic design of efficient components such as dynamic deformable convolution, dynamic upsampling, etc., while strictly controlling the computational complexity. The mechanism avoids the heavy repeated calculation in the traditional neck network, and the core components have been verified through experiments to improve the accuracy while maintaining excellent inference speed, providing a reliable guarantee for realizing high-performance real-time detection.

[0034] Further, the dynamic detection head is configured to uniformly upsample the feature maps of at least two adjacent scales output by the neck network to the scale of the highest resolution branch in the dynamic detection head by means of nearest neighbor interpolation; splice each feature map after up-sampling along the channel dimension to form a unified fusion feature map; the unified fusion feature map is sequentially input into a scale attention unit, a spatial attention unit and a channel attention unit for adaptive weighting processing; finally, the feature map after adaptive weighting is used for target classification and bounding box regression to realize accurate detection of various targets.

[0035] In the embodiment, the dynamic detection head maps the multi-scale feature maps to a unified scale by fusing scale, spatial and channel attention mechanisms, significantly reduces the number of candidate bounding boxes generated with pixel as anchor point, and reduces the computational burden of overlapping boxes in subsequent non-maximum suppression processing; at the same time, the dynamic detection head inherits the independent channel processing concept of modern YOLO series, and realizes fine weighting in scale, spatial and channel dimensions; based on the dynamic adaptive structure, the dynamic detection head can efficiently consider the regression and classification tasks, has good flexibility and universality, and is easy to migrate to other multi-task and multi-class detection scenarios, thereby improving the adaptability of the model in complex environments.

[0036] Further, the scale attention unit is configured to perform global average pooling on the unified fusion feature map to extract global channel features; perform nonlinear mapping on the global channel features; normalize the features after nonlinear mapping to generate scale attention weights; multiply the scale attention weights and the unified fusion feature map element by element to obtain the weighted output feature map.

[0037] Further, the spatial attention unit is configured to calculate the offset of each pixel in the weighted output feature map and generate corresponding mask information, correct the weighted output feature map according to the offset and the corresponding mask information to obtain a corrected feature map.

[0038] Further, the channel attention unit adopts two completely identical but independent parameter structure branches to map the corrected feature map to a semantic decoupled channel space, realizing target bounding box regression and classification.

[0039] Based on the same concept, the present application also provides an electronic device, comprising a memory, a processor and a computer program or instructions stored on the memory, the processor executes the computer program or instructions to realize the unmanned aerial vehicle aerial target detection method as described above.

[0040] Based on the same concept, the present application also provides a computer readable storage medium, having a computer program or instructions stored thereon, the computer program or instructions are executed by a processor to realize the unmanned aerial vehicle aerial target detection method as described above.

[0041] Advantages

[0042] Compared with the prior art, the present application has the advantages that:

[0043] The present application introduces the context learning and deformable convolution design concept, constructs the block local-global attention module, and designs the unified perception context attention in the block local-global attention module, which significantly enhances the feature expression ability of small targets, and significantly reduces the computational resource consumption caused by direct convolution of high-resolution images; The triple feature fusion module is designed in the neck network, which can improve the small scale target feature extraction and learning ability while ensuring that the forward inference speed is basically not affected, thereby avoiding the excessive consumption of resources caused by multiple calculations of feature maps in the neck network. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only one embodiment of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0045] Figure 1 It is the flow chart of the unmanned aerial vehicle aerial target detection method in the embodiment of the present application;

[0046] Figure 2 It is the detection model architecture diagram based on YOLO framework in the embodiment of the present application;

[0047] Figure 3 It is the block local-global attention module architecture diagram in the embodiment of the present application;

[0048] Figure 4 It is the unified perception context attention module architecture diagram in the embodiment of the present application;

[0049] Figure 5 It is the triple feature fusion module architecture diagram in the embodiment of the present application;

[0050] Figure 6is a dynamic detection head architecture diagram fusing triple attention mechanisms in the embodiment of the application. DETAILED DESCRIPTION

[0051] The technical solutions in the application will be described clearly and completely below in conjunction with the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0052] Embodiment one

[0053] In view of the problems of small target size, dense distribution and imbalance in unmanned aerial vehicle aerial image, the embodiment of the application provides a real-time unmanned aerial vehicle aerial target detection method based on a YOLO framework but overall reconstruction of kernels and modules.

[0054] As shown in Figure 1 The unmanned aerial vehicle aerial target detection method provided by the embodiment of the application comprises the following steps:

[0055] Step 1: Obtain unmanned aerial vehicle aerial image data set and pre-process.

[0056] In this embodiment, the public data sets VisDrone2019-Det, DroneVehicle and HazyDet are used to obtain unmanned aerial vehicle aerial images, and then the unmanned aerial vehicle aerial image data set is constructed. The VisDrone2019-Det data set is considered as the most challenging data set under the unmanned aerial vehicle perspective because it covers comprehensive target categories and the proportion of small targets and micro targets in the scale distribution is high. The DroneVehicle data set is designed for unmanned aerial vehicle aerial vehicle detection, and the proportion of low-illumination night images is large, which is suitable for evaluating the night detection performance of the algorithm. The HazyDet data set covers unmanned aerial vehicle aerial vehicle images in foggy weather, which is convenient for testing the robustness of the algorithm under extreme weather conditions.

[0057] The images in the unmanned aerial vehicle aerial image data set are subjected to scale normalization processing.

[0058] Step 2: Construct a detection model.

[0059] In this embodiment, the detection model is based on the YOLO framework. As shown in Figure 2As shown, the detection model only retains the architecture of YOLO, and the rest is replaced with a reconstructed module, where ADS is an adaptive sampling module, SPPF is a fast spatial pyramid pooling, and C2PSCAA is a unified perception context attention mechanism connection module. Specifically, the convolution module of the feature pyramid in the backbone network Backbone is replaced with a blockwise local-global attention module (BLGA) that integrates unified context-aware attention (UCAA); a triple feature fusion module (TFF) is designed in the neck network Neck, and a dynamic detection head (TADHead) that integrates triple attention mechanisms is used in the detection head Head.

[0060] In the detailed description of the present application, as Figure 3 shown, the blockwise local-global attention module aims to replace the pure convolution extraction unit of each layer feature pyramid in the backbone network with an attention mechanism. The design idea is as follows: first, the input feature map is divided into multiple sub-blocks with consistent aspect ratios, and is spliced along the channel dimension to generate a compact fusion feature map; then, the unified perception context attention module is used to extract features from the compact fusion feature map, so as to significantly reduce the computational resource consumption caused by direct convolution on high-resolution images while ensuring the expression ability; finally, the sub-blocks are restored to their spatial positions and sizes in the feature pyramid by inverse disassembly along the channel dimension in the order of the division. The blockwise local-global attention module includes a block fusion module, a unified perception context attention module, and an embedded fusion module, and its detailed processing process is as follows:

[0061] Step A1: The input feature map is divided into multiple sub-blocks by the block fusion module, and is pixel-level weighted according to its spatial position, and all weighted sub-blocks are spliced along the channel dimension to generate a compact fusion feature map.

[0062] Let the feature map input to the BLGA be , H, W and C represent the height, width and channel number of the feature map respectively. The feature map is divided into n×n sub-blocks to form a sub-block set , . The spatial position of the sub-block is pixel-level weighted to embed the spatial structure information, and then spliced along the channel dimension in a predetermined order to form a compact fusion feature map, which realizes the preliminary compression and reorganization of the features while retaining the local structure.

[0063] Step A2: input the compact fusion feature map into the unified perception context attention module, which adopts a multi-scale attention-convolution fusion technology with shared parameters to perform parallel feature extraction on each sub-block. While ensuring resolution reduction, the global context correlation between sub-blocks is implicitly modeled through fixed position embedding, significantly accelerating the calculation while enhancing the representation of local details and overall semantics.

[0064] Step A3: use the embedding fusion module to split the enhanced feature map output by the unified perception context attention module into n x n sub-blocks along the channel dimension according to the order of the original grid layout, and then accurately restore each sub-block to the corresponding spatial position, ensuring that the output feature map is completely consistent with the input in spatial scale. This step realizes the lossless restoration of features from the "split-fusion" state to the original structure, ensuring seamless connection with subsequent network modules.

[0065] Generally, the higher the spatial resolution of the feature map, the more sub-blocks need to be divided; as the number of channels increases, the number of sub-blocks decreases. The adaptive division strategy of the present application enables the BLGA to balance between accelerating feature extraction and maintaining accuracy when processing high-resolution feature maps at the lower layers of the image pyramid, while effectively avoiding the excessive computational overhead caused by directly applying UCAA to large-scale feature maps.

[0066] The most core part of BLGA is UCAA, which replaces the traditional dot product attention calculation with a fully convolutional network and introduces a context anchor point perception mechanism to enhance the joint modeling capability of local and global semantics. As shown in Figure 4 the UCAA includes a context anchor point perception unit, a multi-direction feature extraction unit, a hybrid attention unit, and a feature weighting fusion unit. The context anchor point perception unit uses adaptive pooling operations with different calculation methods for different shaped convolution kernels to calculate anchor point weights, with the specific formula being:

[0067] (1)

[0068] (2)

[0069] (3)

[0070] (4)

[0071] (5)

[0072] (6)​

[0073] wherein, denote the pixel points at positions , , in the compact fusion feature map X, respectively; denote the square anchor points obtained by global average pooling on the whole compact fusion feature map X; denote the horizontal anchor points obtained by average pooling on each row of the compact fusion feature map X in the width direction; denote the vertical anchor points obtained by average pooling on each column of the compact fusion feature map X in the height direction; denote the square anchor point weights; denote the horizontal anchor point weights; denote the vertical anchor point weights; denote functions.

[0074] The pooling kernel size of adaptive average pooling varies according to the scale size of the current feature map to adapt to different levels of the image pyramid. After the anchor points in different directions are calculated, they need to be normalized by the Softmax function to obtain anchor point weights with unified dimensions. Anchor point weights are similar to query vectors in dot product attention mechanism, representing local or global target core feature information.

[0075] The multi-direction feature extraction unit extracts multi-scale and multi-direction key features using heterogeneous convolution kernels, and the specific formula is:

[0076] (7)

[0077] (8)

[0078] (9)

[0079] wherein, denote the square convolution output feature map; denote the horizontal convolution output feature map; denote the vertical convolution output feature map; , and denote square deep convolution, horizontal strip deep convolution kernel and vertical strip deep convolution, respectively; denote the expansion operation; SiLU denotes the activation function.

[0080] To accelerate the calculation speed, the feature extraction adopts the form of deep convolution, which improves the efficiency of convolution calculation and uses the SiLU activation function to retain negative values to enhance feature expression. The size of the square deep convolution kernel and the strip deep convolution kernel is dynamically adjusted according to the scale of the compact fusion feature map X to adapt to the feature extraction requirements of different target sizes. Since the strip deep convolution outputs a long strip feature map, it needs to be restored to the original feature map size through the Expand operation.

[0081] To enhance the feature extraction capability of UCAA, UCAA mixes channel attention mechanism and scale attention mechanism, namely mixed attention unit. The mixed attention unit finally generates attention weights that can fuse the feature maps extracted by the above three different convolution kernels together. The specific formula is:

[0082] (10)

[0083] (11)

[0084] (12)

[0085] (13)

[0086] (14)

[0087] wherein, represents the channel attention statistics; represents the pixel value located at the cth channel, the ith row and the jth column in the compact fusion feature map X; represents the channel attention weight; represents one-dimensional dynamic convolution, a lightweight network layer, which is used for nonlinear transformation to capture complex inter-channel dependencies; represents the sigmoid activation function; k represents the convolution kernel size of one-dimensional dynamic convolution; represents the scale attention statistics; represents the linear layer; represents the scale attention weight; represents the mixed attention weight; represents element-wise multiplication.

[0088] The feature weighted fusion unit adaptively fuses the anchor point weight and the key features (square convolution output feature map, horizontal convolution output feature map and vertical convolution output feature map) according to the mixed attention weight, and outputs the enhanced feature map. The specific formula is:

[0089] (15)

[0090] (16)

[0091] (17)

[0092] (18)

[0093] wherein, , and represent the feature maps in square, horizontal and vertical directions, respectively; represents the enhanced feature map.

[0094] The fusion operation adopts element-wise multiplication and addition, so as to effectively retain multi-scale feature information while keeping the channel dimension unchanged, and avoid excessive weakening of key features due to fusion.

[0095] The UCAA introduces context anchor point perception, channel and scale adaptive attention, and deformable convolution in the full convolution architecture, dynamically selects semantic anchor points, and flexibly weights in the channel and spatial dimensions, while adaptively adjusting the sampling position of the convolution kernel, which not only makes up for the cross-regional semantic fragmentation caused by the BLGA block processing, but also enhances the feature extraction capability of multi-direction and multi-scale targets, and significantly improves the inference speed due to the complete dependence on efficient convolution operation.

[0096] In the specific embodiments of the present application, as shown in Figure 2 The neck network is composed of three cascaded triple feature fusion modules, each of which adopts a sandwiched three-layer fusion structure, and integrates a dynamic deformable convolution unit (DCN), a depth separable convolution unit (DWConv) and a dynamic upsampling unit (DySample) in sequence, to realize efficient fusion of cross-scale features. Figure 5 The architecture diagram of the triple feature fusion module is shown, wherein, Figure (a) is a dynamic deformable convolution unit, Figure (b) is a depth separable convolution unit, and Figure (c) is a dynamic upsampling unit, g represents the offset channel number, and s represents the upsampling multiple. The design purpose of the triple feature fusion module is to fuse feature maps of different scales in adjacent three-layer stepped scales, so as to ensure that the model can realize multi-scale feature information fusion of targets of different scales, and improve the positioning accuracy of the model. The feature extraction process of the neck network based on the triple feature fusion module is:

[0097] Firstly, the dynamic deformable convolution unit DCN in the bottom triple feature fusion module adaptively captures the geometric morphology of small targets in the high-resolution feature map at the bottom of the feature pyramid, extracts high-precision detail features, and forms a detail enhancement mapping;

[0098] Subsequently, high-precision detailed features are separated by a deep separable convolution unit DWConv for channel-by-channel separable convolution, while retaining key structural information, to generate intermediate-scale transition feature maps.

[0099] Meanwhile, a dynamic upsampling unit DySample is used to recover the spatial scale of the low-resolution feature maps from the previous level by bilinear interpolation or transposed convolution, so that the resolution is aligned with the bottom layer features, thereby retaining more semantic details; the features output by the dynamic upsampling unit are then further input into a dynamic deformable convolution unit to enhance its geometric perception ability and context representation quality.

[0100] Finally, the three layers of scale-aligned feature maps are concatenated along the channel dimension to form an enhanced output that fuses multi-scale information, which is used as the input of the next-level triple feature fusion module or directly passed to the detection head.

[0101] The dynamic deformable convolution unit, deep separable convolution unit and dynamic upsampling unit used in the triple feature fusion module can improve the ability of small-scale target feature extraction and learning while ensuring that the forward inference speed is not affected, thereby avoiding excessive resource consumption caused by multiple calculations of feature maps in the neck network. In addition, the dynamic deformable convolution unit and the dynamic upsampling unit have been proven to be feasible in multiple experiments, achieving a good balance between detection accuracy and computational efficiency.

[0102] The traditional detection head usually contains three branches corresponding to the three scales of the feature pyramid: the highest layer is used for large target detection, the middle layer is used for medium and normal target detection, and the bottom layer is used for small target detection. Given that the proportion of small targets in high-altitude aerial photography by unmanned aerial vehicles is high, the present invention adds a fourth branch, a micro target detection head, to the original three-branch detection head, which is dedicated to the identification of extremely small scale targets, to enhance the response ability to small targets. In addition, the present invention also fuses a triple attention mechanism in the detection head.

[0103] Figure 6 The architecture of the dynamic detection head that fuses the triple attention mechanism is shown, which uses nearest neighbor interpolation to uniformly upsample the feature maps from adjacent different scales, so that their spatial resolution is consistent with the micro target detection branch, and there is no need for each scale branch to output candidate boxes. Unlike the multi-layer detection head in the original YOLO series, which predicts redundantly across scales, the present invention only retains the single-layer output of the micro target detection head, significantly reducing the number of candidate boxes, thereby speeding up the processing speed of the subsequent non-maximum suppression algorithm (NMS).

[0104] The feature maps of at least two adjacent scales output by the neck network are uniformly upsampled to the scale corresponding to the small target detection branch through nearest neighbor interpolation. The upsampled feature maps are then concatenated along the channel dimension to form a unified fused feature map. The unified fused feature map is then sequentially input into the scale attention unit, spatial attention unit, and channel attention unit for adaptive weighting. Finally, the adaptively weighted feature map is used for target classification and bounding box regression to achieve accurate detection of various targets.

[0105] Specifically, the scale attention unit performs global average pooling on the unified fused feature map to extract global channel features; it then performs a non-linear mapping on the global channel features; normalizes the non-linearly mapped features to generate scale attention weights; and finally, it multiplies the scale attention weights element-wise with the unified fused feature map to obtain a weighted output feature map. The calculation formula for the scale attention unit is as follows:

[0106] (19)

[0107] (20)

[0108] (twenty one)

[0109] in, Represents a unified and integrated feature map; Indicates a splicing operation; These represent feature maps from different levels of the feature pyramid; Indicates an upsampling operation; This represents depthwise separable convolution; Indicates global average pooling; express Activation function; This represents the weighted output feature map.

[0110] The spatial attention unit calculates the offset of each pixel in the weighted output feature map and generates the corresponding mask information. Based on the offset and the corresponding mask information, it corrects the weighted output feature map to obtain the corrected feature map. The spatial attention unit ensures accurate spatial alignment of feature maps at different scales by correcting the spatial positional errors of pixels after nearest-neighbor interpolation upsampling. The specific calculation formula is as follows:

[0111] (twenty two)

[0112] (twenty three)

[0113] (twenty four)

[0114] (25)

[0115] wherein, denotes the original offset; denotes deformable convolution; denotes the guidance information for generating the initial offset; denotes the offset corresponding mask information, i.e. the importance weight of pixels, only takes effect within the single-layer feature map of the current spatial scale; denotes the offset of each pixel; denotes the convolution layer; denotes the corrected feature map.

[0116] The channel attention unit adopts two completely identical but parameter independent structure branches to map the corrected feature map to the semantic decoupled channel space, realizing the target bounding box regression and classification. The specific calculation formula of the channel attention unit is:

[0117] (26)

[0118] (27)

[0119] (28)

[0120] (29)

[0121] wherein, denotes the initial channel weight vector of the regression task; denotes the normalization processing; denotes the fully connected layer; denotes the final channel attention weight of the regression task; both denote the learnable scaling parameter of the regression branch; both denote the learnable bias parameter of the regression branch; denotes the initial channel weight vector of the classification task; denotes the final channel attention weight of the classification task; both denote the learnable scaling parameter of the classification branch; both denote the learnable bias parameter of the classification branch.

[0122] The dynamic detection head TADHead maps multi-scale feature maps to a unified scale by fusing scale attention, spatial attention and channel attention, significantly reduces the number of candidate boxes generated with pixel as anchor point, and reduces the calculation burden of overlapping boxes in subsequent NMS processing. Meanwhile, the invention inherits the independent channel processing concept of modern YOLO series, and realizes fine weighting in scale, space and channel dimensions; based on the dynamic adaptive structure, the dynamic detection head TADHead can efficiently consider the regression and classification tasks, has good flexibility and universality, is easy to migrate to other multi-task and multi-class detection scenes, thereby improving the adaptability of the model in complex environments.

[0123] Step 3: training and verifying the detection model using the UAV aerial image dataset to obtain the final detection model.

[0124] The UAV aerial image dataset is divided into a training set and a verification set, the training set is used to train the detection model, and the verification set is used to verify the trained detection model to obtain the final detection model. In the specific embodiment of the present application, the specific process of training the detection model using the training set is as follows:

[0125] Step 3.1: a batch of images is obtained from the training set and input into the detection model, and a backbone network based on BLGA is used to extract features from the input images to obtain a series of multi-scale feature maps (such as C3, C4 and C5 three different scales) with rich semantic information;

[0126] Step 3.2: the multi-scale feature maps output by the backbone network are input into the neck network, and feature fusion is performed through three cascaded triple feature fusion modules to obtain a feature pyramid (such as P3, P4 and P5 three different scales) that fuses high-level semantics and low-level details, providing features with strong semantics and precise positioning ability for the dynamic detection head;

[0127] Step 3.3: the feature pyramid (such as P3, P4 and P5 three different scales) output by the neck network is input into the dynamic detection head to obtain four scale prediction tensors, each of which contains bounding box coordinates, confidence and class probability;

[0128] Step 3.4: calculate the loss value (including classification loss, boundary loss, etc.) according to the prediction tensor and the true value, and update all parameters of the model according to the loss value;

[0129] Step 3.5: repeat the above steps until the model converges or reaches the predetermined training period.

[0130] Step 4: using the final detection model, inputting the UAV aerial image to perform target detection.

[0131] The UAV aerial image to be detected is input into the final detection model, and the model outputs a prediction result, specifically including the bounding box coordinates, confidence and class probability of the target.

[0132] In view of the fact that the target under the high-altitude view of the UAV is mostly small in size and densely distributed, and is easily disturbed by the background and noise, the application introduces the design concept of context learning and deformable convolution, constructs a block local-global attention module fused with uniform perception context attention, and replaces the convolution module of the feature pyramid in the backbone network with the block local-global attention module fused with uniform perception context attention, so as to enhance the feature expression capability of the micro target.

[0133] In view of the fact that the target size is uneven in the image in different scenes due to the overhead view, and the existing method is difficult to balance the multi-scale detection effect, the application designs a triple feature fusion module in the neck network, and through the interaction and compensation of multi-layer pyramid feature maps, the effective extraction of targets of different scales is ensured.

[0134] Considering the computing power and real-time performance limitation of the UAV platform, the application replaces the calculation bottleneck point product attention in the attention mechanism with an efficient convolution operation, and designs a variety of lightweight detection heads for attention fusion, thereby reducing the number of candidate boxes and accelerating the post-processing process, so as to realize high-speed reasoning on the embedded GPU.

[0135] In order to verify the effectiveness of the method of the application, the detection model in the application is trained and tested on the public data sets VisDrone2019-Det, DroneVehicle and HazyDet.

[0136] In the experiment, mAP50 and mAP50:95 are selected as the detection precision evaluation indexes, and FPS (frame rate or frame rate per second) is selected as the core index of reasoning speed. Among them, mAP50 refers to the average value of the average precision of each class under the IoU (intersection over union) threshold of 0.50; mAP50:95 refers to the average value of the average precision of each class calculated under multiple thresholds with a step of 0.05 from 0.50 to 0.95. FPS reflects the running efficiency of the algorithm on the hardware device. The method of the application is evaluated on the Nvidia Geforce RTX 3090 GPU.

[0137] In the experiment, the latest real-time target detection model YOLOv11, the classic YOLO series improved algorithm widely used under the UAV view, and the lightweight R18 version of the leading UAVDETR detection precision are selected as the comparison methods, and the experimental results are shown in Table 1.

[0138] Table 1 Summary of experimental results

[0139]

[0140] As can be seen from Table 1, the application is slightly inferior to the latest UAVDETR(R18) lightweight model in terms of mAP50 and mAP50:95 precision indicators, but the inference speed of UAVDETR(R18) is much lower than the real-time application requirement; compared with other methods, the detection speed of the application is at a medium level, and in terms of precision, it is significantly better than the remaining comparative algorithms except UAVDETR(R18), which verifies the good balance between precision and speed of the method in the application.

[0141] Embodiment two

[0142] The embodiment of the application further provides an electronic device, which comprises a memory, a processor and a computer program or instructions stored in the memory, and the processor executes the computer program or instructions to implement the unmanned aerial vehicle aerial target detection method in the embodiment of the application.

[0143] Although not shown, the electronic device comprises a processor, which can perform various appropriate operations and processes according to programs and / or data stored in a read-only memory (ROM) or programs and / or data loaded from a storage part into a random access memory (RAM). The processor can be a multi-core processor or can comprise a plurality of processors. In some embodiments, the processor can comprise a general-purpose main processor and one or more special-purpose coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), etc. In the RAM, various programs and data required for device operation are also stored. The processor, the ROM and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0144] The above processor and memory are used together to execute programs / instructions stored in the memory, and the programs / instructions are executed by a computer to implement the methods, steps or functions described in the above embodiments.

[0145] Although not shown, the embodiment of the application further provides a computer readable storage medium having a computer program or instructions stored thereon, and the computer program or instructions are executed by a processor to implement the unmanned aerial vehicle aerial target detection method in the embodiment of the application.

[0146] Read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technologies, compact disc read only memory (CD-ROM), digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store information which can be accessed by a computing device. According to the definition used herein, computer readable medium does not include transitory media, such as modulated data signals and carrier waves.

[0147] The above disclosure is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or modifications within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for detecting targets in drone aerial photography, characterized in that, The detection method includes: Acquire and preprocess a dataset of drone aerial images; A detection model is constructed based on the YOLO framework, and the convolutional modules of the feature pyramid in its backbone network are replaced with block local-global attention modules that integrate unified perceptual contextual attention. A triple feature fusion module is designed in the neck network, and a dynamic detection head that integrates the triple attention mechanism is adopted in the detection head. The block-based local-global attention module is used to decompose the input feature map into position-weighted sub-maps and fuse them. It uses unified perception contextual attention to enhance each sub-map in parallel in a compact feature space, while implicitly modeling the global context. Finally, it restores the original structure through reverse mapping. The detection model is trained and validated using the drone aerial image dataset to obtain the final detection model; Using the final detection model, target detection is performed with drone aerial images as input; The neck network consists of three cascaded triple feature fusion modules. Each triple feature fusion module adopts a sandwich three-layer fusion structure, which sequentially integrates a dynamically deformable convolutional unit, a depthwise separable convolutional unit, and a dynamic upsampling unit to achieve efficient fusion of cross-scale features. The dynamic detection head is used to upsample the feature maps of at least two adjacent scales output by the neck network to the scale corresponding to the small target detection branch through nearest neighbor interpolation; the upsampled feature maps are then concatenated along the channel dimension to form a unified fused feature map; the unified fused feature map is then sequentially input into the scale attention unit, spatial attention unit, and channel attention unit for adaptive weighting processing; finally, the adaptively weighted feature map is used for target classification and bounding box regression to achieve accurate detection of various targets; The block-based local-global attention module includes a unified-aware contextual attention module. This module is used to capture the anchor weights of the input compact fused feature map in different directions through a context anchor perception unit; extract key features of multiple scales and directions using heterogeneous convolutional kernels through a multi-directional feature extraction unit; dynamically calculate the hybrid attention weights in channels and scales through a hybrid attention unit; and adaptively fuse the anchor weights and key features according to the hybrid attention weights by a feature weighted fusion unit to output an enhanced feature map.

2. The UAV aerial target detection method according to claim 1, characterized in that, The segmented local-global attention module also includes a segmented fusion module and an embedded fusion module; The block fusion module is used to divide the input feature map into multiple sub-map blocks, perform pixel-level weighting based on their spatial location, and stitch all weighted sub-map blocks along the channel dimension to generate a compact fused feature map. The embedding fusion module is used to reverse decompose the enhanced feature map in the channel dimension according to the order of block division, and restore each sub-block to its spatial position and size in the feature pyramid.

3. The UAV aerial target detection method according to claim 1, characterized in that, The feature extraction process of the neck network is as follows: First, the dynamic deformable convolutional unit in the bottom triple feature fusion module adaptively captures the geometric shape of small targets in the high-resolution feature map at the bottom of the feature pyramid, extracts high-precision detail features, and forms a detail enhancement mapping. Subsequently, the high-precision detail features are separated and convolved channel by channel by depthwise separable convolutional units, which retains key structural information while efficiently downsampling, and generates intermediate-scale transition feature maps. Meanwhile, the spatial scale of the low-resolution feature map from the previous level is restored by the dynamic upsampling unit, so that its resolution is aligned with the low-level features, thereby preserving more semantic details; then the features output by the dynamic upsampling unit are further input into the dynamic deformable convolutional unit to enhance its geometric perception ability and context representation quality. Finally, the three scale-aligned feature maps are stitched together along the channel dimension to form an enhanced output that integrates multi-scale information. This output serves as the input to the next-level triple feature fusion module or is directly passed to the detection head.

4. The UAV aerial target detection method according to claim 1, characterized in that, The scale attention unit is used to perform global average pooling on the unified fusion feature map to extract global channel features; Perform nonlinear mapping on the global channel features; The features after nonlinear mapping are normalized to generate scale attention weights; the scale attention weights are then multiplied element-wise with the unified fusion feature map to obtain a weighted output feature map.

5. The UAV aerial target detection method according to claim 1, characterized in that, The spatial attention unit is used to calculate the offset of each pixel in the weighted output feature map and generate the corresponding mask information. Based on the offset and the corresponding mask information, the weighted output feature map is corrected to obtain the corrected feature map.

6. The UAV aerial target detection method according to claim 1, characterized in that, The channel attention unit employs two identical but parameter-independent structural branches to map the corrected feature map to a semantically decoupled channel space, thereby achieving target bounding box regression and classification.

7. An electronic device, comprising: A memory, a processor, and a computer program or instructions stored in the memory, characterized in that the processor executes the computer program or instructions to implement the UAV aerial target detection method as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by the processor, they implement the UAV aerial target detection method as described in any one of claims 1 to 6.

Images