High-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method and device

Abstract

The invention discloses a high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method and device, and belongs to the technical field of target detection, and the method comprises the steps: introducing a target activation network into a backbone network to generate an object activation graph, cutting out a region block containing a target from a feature level through uniform cutting and mapping, and reducing the calculation; a space-channel collaborative attention module and an enhanced multi-level feature reconstruction module are introduced into the neck network to process each layer of feature map and the activation enhancement feature map output by the backbone network to obtain and output high-scale fusion features of each level and low-scale down-sampling features of each level; a dynamic mask sparse calculation module is introduced into the detection network to generate a hard mask of a key area in neck network output features, sparse convolution of the neck network output features is guided by the hard mask to obtain sparse features, classification and bounding box regression are executed after multi-scale sparse feature fusion is performed on all levels of sparse features, and a target detection result is output. The detection speed and the detection accuracy can be improved.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image target detection technology, specifically relating to a high-resolution UAV multi-scale remote sensing image target detection method and device. Background Technology

[0002] In the field of computer vision, object detection technology has made significant progress in recent years, especially on natural image datasets such as MS COCO and Pascal VOC, where deep learning-based detectors (such as Faster R-CNN, YOLO, and SSD) have performed exceptionally well. However, the application of these general object detection methods to aerial images faces serious challenges, mainly due to the unique characteristics of aerial images: image sizes are typically extremely large (up to hundreds of millions of pixels), small objects constitute a high proportion (such as objects with an area smaller than 32×32 pixels), and objects are unevenly distributed (for example, vehicles and pedestrians tend to cluster in specific areas while being sparse in other areas). These factors result in unsatisfactory accuracy and efficiency when directly applying existing detectors, especially with poor performance in small object detection and significant waste of computational resources.

[0003] Existing object detection models, such as the YOLO series and RetinaNet, improve performance through multi-scale feature fusion and detection network design. These models typically consist of three parts: a backbone network, a neck network, and a head network. The neck network is responsible for feature fusion to enhance the extraction of multi-scale information, while the detection network is responsible for predicting the target's location and category. However, in aerial imagery, existing technologies still have significant shortcomings in the neck and head aspects, limiting improvements in lightweight design and inference speed. Furthermore, for aerial images, traditional methods require a high-precision set of calculations for each region of the entire image, including feature extraction, feature integration, and classification, resulting in low inference efficiency and wasted computational resources and time.

[0004] To address these challenges, researchers have proposed several improvement schemes. Among them, image cropping-based strategies are widely adopted, such as unified cropping methods (e.g., R...). 2CNNs and SAHI (Self-Identifying and Detection Networks) segment high-resolution images into overlapping patches for detection. However, this method ignores the non-uniformity of target distribution, resulting in the invalidation of a large number of background regions and low computational efficiency. Subsequently, more advanced schemes focused on locating clustered regions (i.e., densely populated areas of targets) to prioritize processing these regions, thereby improving detection accuracy and speed. Representative works include ClusDet and DMNet: ClusDet introduces a Clustering Region Proposal Network (CPNet) and a Scale Estimation Network (ScaleNet) to generate clustered regions and adjust their scale; DMNet uses density maps to learn the target distribution to generate clustering proposals. While these methods improve accuracy, they rely on additional sub-networks (such as CPNet), increasing model complexity and the number of parameters, and each cropped region needs to be processed independently during inference, slowing down the detection speed.

[0005] Against this backdrop, YOLC was proposed as an innovative solution built on CenterNet, aiming to optimize detection performance for aerial images. YOLC's core improvements include: using a high-resolution heatmap (density map) to enhance small target detection; introducing Local Scale Modules (LSM) for adaptive clustering region search; and employing the Gaussian Wasserstein distance (GWD) loss function to improve regression accuracy. Despite YOLC's breakthroughs in accuracy, its technical approach has significant shortcomings.

[0006] First, the LSM module is essentially a post-processing step, physically cropping clustered regions from the original image and rescaling them before independent detection. This is similar to the cropping strategy of ClusDet, adding extra processing steps. Specifically, YOLC's detection process is divided into two stages: first, generating a heatmap for the entire image and identifying clustered regions, then performing detection on each cropped region individually. This design results in the model not being a single end-to-end network, but rather introducing additional cropping and re-detection steps, increasing computational complexity and memory overhead. Second, the cropping operation of the LSM module is based on image-level processing rather than feature-level integration, which limits its deep integration with the detection network. Since the cropped regions need to be resized to fit the detector, scale distortion or information loss may be introduced, especially sensitive to small targets. Furthermore, LSM relies on hyperparameters (such as grid size and Top-K values), which require empirical tuning, increasing deployment difficulty. The root cause is that YOLC fails to seamlessly embed clustered region detection into a single network architecture, instead using the traditional detection-cropping-re-detection paradigm, resulting in a verbose model.

[0007] The challenge in addressing this issue lies in how to directly locate and enhance clustered regions in the feature space without introducing additional sub-networks, while preserving high-resolution details to support small object detection. Existing attempts (such as feature pyramid networks) can handle multiple scales, but they cannot dynamically optimize for non-uniform distributions. Therefore, although YOLC outperforms previous methods in accuracy, its multi-step process and additional cropping mechanisms limit its efficiency and practicality, especially in large-scale aerial image applications. A simpler, end-to-end solution is urgently needed to integrate clustered region processing between the backbone network and the neck network, avoiding redundant computation. Summary of the Invention

[0008] In view of the above, the purpose of this invention is to provide a high-resolution UAV multi-scale remote sensing image target detection method and apparatus to improve target detection accuracy and detection speed.

[0009] To achieve the above-mentioned objectives, an embodiment provides a high-resolution UAV multi-scale remote sensing image target detection method, comprising the following steps: A target detection network is constructed, comprising a backbone network, a neck network, and a detection network. The backbone network incorporates a target activation network to generate object activation maps. Based on these activation maps, regions containing the target are extracted from the image features output by the backbone network through uniform slicing and mapping. The neck network incorporates a spatial-channel collaborative attention module and an enhanced multi-level feature reconstruction module to process these regions, obtaining high-scale fusion features and low-scale downsampling features during the downsampling process, which are then output. The detection network incorporates a dynamic mask sparse computation module to generate hard masks for key regions in the neck network's output features. These hard masks guide the sparse convolution computation of the neck network's output features, resulting in sparse features. These sparse features are then fused through multi-scale sparse feature fusion and passed through a detection head for classification and bounding box regression, outputting the target detection results. The constructed target detection network is used to perform target detection on the input UAV remote sensing images.

[0010] Preferably, the target activation network is composed of depthwise separable convolutional blocks and convolutions, and performs forward propagation based on the feature map of the last layer output by the backbone network to predict the object activation map; During training, an axially independent Gaussian kernel is used to generate a Gaussian mask as a pseudo-label based on the bounding box annotations. The predicted object activation map and the Gaussian mask are used to calculate the binary cross-entropy loss, and the network weights are updated through backpropagation.

[0011] Preferably, the method of cutting out the region block containing the target from the image features of each layer output by the backbone network by uniformly slicing and mapping according to the object activation map includes: dividing the object activation map into a grid and then filtering out the target region according to the grid based on a threshold, and then mapping the coordinates of the target region to the image features of each layer output by the backbone network, thereby cutting out the region block containing the target.

[0012] The obtained object activation map is divided into a 4×4 grid, the target region is filtered out based on a threshold, and the corresponding coordinates are mapped to layers C3, C4, and C5, stacked into three batch tensors, which serve as the input to the subsequent neck network.

[0013] Preferably, the spatial-channel collaborative attention module and the enhanced multi-level feature reconstruction module process the two input features in parallel. During the upsampling process, the two input features come from the last two layers of the backbone network output, or from the intermediate layer of the backbone network output and the fusion features of the previous scale during the upsampling process. During the downsampling process, the two input features come from the fusion features of the same scale during the upsampling process and the fusion features of the previous scale during the downsampling process. The spatial-channel collaborative attention module simultaneously models the interdependence between the spatial and channel dimensions of two input features, thereby achieving collaborative attention enhancement and obtaining attention-enhanced features. The enhanced multi-level feature reconstruction module adaptively separates, fuses, and reconstructs the two input features based on their own strength attributes to obtain the reconstructed features. During the downsampling process, the attention-enhanced features and reconstructed features at each level are fused to obtain high-scale fused features at each level and then output. The last level of fused features is then further downsampled to generate low-scale downsampled features at each level and then output.

[0014] Preferably, the spatial-channel collaborative attention module simultaneously models the interdependence between the spatial and channel dimensions of two input features to achieve collaborative attention enhancement and obtain attention-enhanced features, including: The preliminary fusion result of the two input features is used as the input feature map of the spatial-channel collaborative attention module; Channel attention branch: Channel attention weights are calculated by using global average pooling, multilayer perceptron, and activation functions on the input feature map. Spatial attention branch: After calculating the mean and maximum values ​​of the input feature maps, the channels are concatenated, and then convolution and activation are performed to generate spatial attention weights; Collaborative weighting and feature interaction: The input feature map is multiplied element-wise with channel attention weights and spatial attention weights to obtain features modulated by dual attention. A grouped convolution is introduced to group the modulated features along the channel dimension, and then an independent convolution is performed within each group. Finally, the features are recombined and then batch normalized. Through residual connections controlled by learnable scaling parameters, the input feature map is added to the grouped interactive features to output the final attention-enhanced features.

[0015] Preferably, the enhanced multi-level feature reconstruction module adaptively separates, fuses, and reconstructs the two input features based on their own strength attributes to obtain reconstructed features, including: Feature preprocessing and weight generation: The two input features are respectively channel aligned and pre-transformed through convolutional layers, then batch normalized and activated to generate two independent normalized weight maps; Feature separation: The preliminary fusion result of the two input features generates a dynamic adaptive threshold map through a threshold generator. The normalized weight maps of the two input features are compared with the adaptive threshold map to generate four binary masks to distinguish each input feature into strong features and weak features. The strong and weak features are obtained by weighted summation of the binary masks. Feature transformation: The strong features and weak features are transformed by convolutional layers to obtain the transformed strong features and the transformed weak features respectively; Gated weighted fusion: A gate generator is applied to the separated weak features to generate soft attention weights in the channel dimension, which are used to weight the transformed weak features to obtain the final weak features; the transformed strong features are added to the final weak features to output the reconstructed features.

[0016] Preferably, the threshold generator employs global average pooling, convolutional layers, and the Sigmoid function; The gated generator employs global average pooling, convolutional layers, and the Softmax function.

[0017] Preferably, the hard mask for key regions in the generated neck network output features includes: The features at each scale of the neck network output are first processed by convolution to extract basic features and then enter the adaptive masking module. The basic features extracted in the adaptive masking module are combined with random noise to generate a continuous soft mask representing the probability of each position being of interest. The soft mask is processed by thresholding and discretized into hard masks that explicitly specify important and unimportant regions under the direct supervision of the real mask labels.

[0018] Preferably, the sparse features are obtained by sparse convolution calculation using hard masking to guide the output features of the neck network, including: The features at each scale output by the neck network are lightly processed by convolution and then simultaneously input into the sparse convolution along with the hard mask, which acts as a dynamic selector. This allows the sparse convolution computation to focus on the key regions indicated by the hard mask, resulting in sparse features.

[0019] Preferably, the sparse features at each level are fused through multi-scale sparse feature fusion and classification and bounding box regression are performed to output target detection results, including: In a bottom-up approach, lower-level sparse features are downsampled and concatenated with sparse features of adjacent higher-level features, thereby transferring and fusing details and semantic information between multiple levels. Finally, the multi-level sparse features that have been fused step by step are fed into the detection head to complete the classification and bounding box regression tasks in a unified manner, and output multi-level object detection results.

[0020] To achieve the above-mentioned objectives, an embodiment also provides a high-resolution UAV multi-scale remote sensing image target detection method, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they implement the above-mentioned high-resolution UAV multi-scale remote sensing image target detection method.

[0021] Compared with the prior art, the beneficial effects of the present invention include at least the following: A Target Activation Network (OSK) for generating binary masks is introduced into the backbone network. This OSK acts as a filtering layer, segmenting regions containing targets at the feature level. The top-level feature map is used to generate a binary map to quickly identify regions with clustered targets and those without. Subsequent processing focuses only on regions with targets, filtering out those without targets to reduce subsequent computation.

[0022] In the neck network, a spatial-channel collaborative attention module (SCJA module) and an enhanced multi-level feature reconstruction module (EMFR module) are introduced to process the fusion features of each level of activation enhancement feature map fusion to obtain multi-scale features at each level. The role of the SCJA module is to enhance the multi-scale fusion information by coordinating spatial and channel multi-angles, while the role of the EMFR module is to separate strong and weak features, enhance the edge detail information of small targets, reduce the extraction of irrelevant information, and reduce the increased computational cost by using lightweight convolution.

[0023] A Dynamic Mask Sparse Computation Module (DMS module) is introduced into the detection network to generate hard masks of key regions in learnable, task-oriented multi-scale features. The hard masks are used to guide the sparse convolution calculation of multi-scale features at all levels to obtain sparse features, thereby reducing the computation of the detection head. The sparse features at all levels are fused through multi-scale sparse features and classification and bounding box regression are performed to output the target detection results. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the structure and process of the target detection network provided in the embodiment; Figure 2 This is a schematic diagram of the structure and training of the target activation network provided in the embodiment; Figure 3 This is a schematic diagram of the structure and inference process of the target activation network and uniform block mapping provided in the embodiment; Figure 4 This is a schematic diagram of the structure and process of the sparse detection module provided in the embodiment. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0027] The inventive concept of this invention is as follows: Existing technologies for target detection in UAV remote sensing images suffer from the following technical problems: Existing improved methods based on clustering regions (such as YOLC) rely on additional sub-networks and physical cropping operations, forming a non-end-to-end two-stage detection process (such as generating a density map first and then cropping and retesting), leading to repeated feature extraction, increased computational complexity, and low inference efficiency; In the feature fusion of the neck network, such as Feature Pyramid Network (FPN), the correlation between inter-layer features cannot be fully utilized, resulting in poor feature matching, especially when processing small targets, where the fusion of shallow details and deep semantic information is insufficient, causing semantic bias and reducing detection accuracy; In the detection network structure, such as RetinaNet, the head uses dense convolutions, with dense parameters and a lack of dynamic adjustment mechanisms, making it difficult to adapt to the large differences in target scale in aerial images, and it performs full detection on the feature map, failing to utilize important information and becoming a bottleneck for inference speed.

[0028] To address the aforementioned technical problems, embodiments of the present invention provide a high-resolution UAV multi-scale remote sensing image target detection method, such as... Figure 1As shown, the target detection network constructed by the company improves the backbone network, neck network, and detection network. Specifically, it introduces a target activation network (OSK) to generate binary masks into the backbone network, a spatial-channel collaborative attention module (SCJA module) and an enhanced multi-level feature reconstruction module (EMFR module) into the neck network, and a dynamic mask sparse computation module (DMS module) into the detection network.

[0029] In this embodiment, the core function of OSK is to generate a class-independent object activation map in the backbone network. This object activation map serves as a mask for efficiently locating foreground regions that may contain objects. Unlike direct regression bounding boxes or density maps, OSK focuses on distinguishing regions containing objects from the background, avoiding the complexity of highly semantic prediction. This design is inspired by the divide-and-conquer approach, first coarsely locating objects and then refining the classification, making it particularly suitable for small object detection scenarios. However, unlike implementations such as yolc, it is an end-to-end process, eliminating the need for multiple image cropping and feeding into the detector.

[0030] In terms of technical implementation, OSK utilizes axially independent Gaussian kernels to generate Gaussian masks used as pseudo-labels for training from bounding box annotations. The original Gaussian kernel formula is radially symmetric, while the improved axially independent Gaussian kernel uses independent Gaussian function multiplication in the X and Y directions to form a Gaussian rectangular mask that better matches the rectangular bounding box. Specifically, given the true bounding box center (x... c , y c The mask value at the feature map location (x,y) with width w and height h is calculated by two independent Gaussian functions: G x = exp(-(xx c ) 2 / (2σ w 2 )), G y = exp(-(yy c ) 2 / (2σ h 2 The final Gaussian mask is M = G. x × G y , where σ w = w / (2k), σ h = h / (2k), where k controls the decay rate.

[0031] During the training phase, such as Figure 2As shown, this Gaussian mask serves as a supervisory signal, guiding OSK to learn and generate high-quality object activation maps for use as masks. OSK consists of depthwise separable convolutional blocks and 1×1 convolutions. The depthwise convolutions use an 11×11 large kernel to expand the receptive field, while the pointwise convolutions output a single-channel mask. During training, forward propagation is performed based on the feature map of the last layer output by the backbone network to obtain the predicted object activation map. This predicted map is then compared with the Gaussian mask used as the label to calculate a binary cross-entropy loss. Backpropagation updates the network weights, enabling OSK to learn to identify object regions from the features.

[0032] During the inference phase, a well-trained OSK can automatically generate object activation maps without the need for real bounding boxes.

[0033] In this way, the Gaussian mask serves as a supervisory signal during the training phase, guiding OSK to learn and generate high-quality object activation maps. During the inference phase, the learned object activation maps are directly used to enhance features without the need for additional Gaussian mask calculations.

[0034] After obtaining the object activation map, it needs to be converted into a specific feature block selection decision. After the object activation map is divided into a grid, the target region is selected according to the grid based on a threshold. Then, the coordinates of the target region are mapped to the image features of each layer output by the backbone network (such as layers C3, C4, and C5), and stacked into three batch tensors, thereby segmenting the region blocks containing the target, which are then used as the input of the subsequent neck network.

[0035] Specifically, since the spatial resolution of the object activation map is 32×32, it is uniformly divided into a 4×4 grid, with each grid being 8×8, corresponding to a 256×256 region in the original image. For each 8×8 grid, the average activation value of the target mask within its region is calculated. For the 16 grids, if the activation value of a grid is greater than or equal to a certain value, the grid contains a potential target and is retained; otherwise, the grid is considered to mainly contain background information and is discarded. Figure 3 As shown, this step filters out N feature blocks containing the target from the original 16 feature blocks. Let the coordinates of the selected k-th feature block in layer C5 be... ,in , Because layers C3, C4, and C5 have different spatial resolutions, the coordinates of layer C5 need to be mapped to the other layers: layer C4 has a downsampling rate of 16, corresponding to coordinates of... The downsampling rate of layer C3 is 8, and the corresponding coordinates are... For the C3, C4, and C5 feature blocks extracted from each target region, we stack them along the batch dimension: all C3 feature blocks from all regions are stacked into one batch tensor, all C4 feature blocks are stacked into another batch tensor, and so on for C5 feature blocks. These three batch tensors—each corresponding to a feature level but containing all regions—perfectly meet the input requirements of FPN: a list containing three elements, each element being a batch of feature maps. This filters out the target region blocks from the original feature maps, saving subsequent computation and allowing them to be used as input to the subsequent neck network.

[0036] In this embodiment, the neck network is based on the traditional feature pyramid network and uses the SCJA module for refined feature recalibration; the EMFR module is designed to achieve adaptive feature separation and fusion, and combined with an efficient multi-scale feature fusion strategy, the feature representation capability and target detection performance are improved.

[0037] In this embodiment, the SCJA module aims to simultaneously model the interdependencies between the spatial and channel dimensions of the feature maps, achieving collaborative attention enhancement. Specifically, the channel attention branch first uses global average pooling to compress the spatial dimension of the input feature maps at each layer, generating channel descriptors. Then, a small multilayer perceptron (MLP) containing two 1×1 convolutional layers with a ReLU activation function sandwiched in between processes the channel descriptors: the first convolution reduces the number of channels by a certain factor (e.g., 16 times) to reduce computational cost, while the second convolution restores the original number of channels. Finally, a channel attention weight ca between 0 and 1 is generated using the Sigmoid function. weights This is used to perform soft weighting on each channel of the input features; Spatial attention branch: First, calculate the average value (avg) of the input feature maps for each layer along the channel dimension. out With the maximum value max out The two 1-channel feature maps are then concatenated along the channel dimension to form a 2-channel description map. This description map is then processed by a convolutional layer (nn.Conv2d) with a large 7×7 kernel and a group size of 1 to capture spatial contextual information within a larger receptive field. Finally, a sigmoid function is used to generate spatial attention weights sa. weights ; Collaborative weighting and feature interaction: The input feature map X of each layer is combined with the channel attention weights ca. weights Spatial attention weights sa weights Element-wise multiplication yields the features after dual attention modulation. x To further facilitate feature interaction, a grouped convolution is introduced to group the features. xThe input feature map is grouped along the channel dimension, then independently convolved within each group, and finally recombinated. This is followed by batch normalization for stable training. Finally, a residual connection controlled by a learnable scaling parameter (gamma) adds the original input feature map X to the grouped features, outputting the final attention-enhanced features. This design ensures that the attention-modulated features enhance important information while preserving the basic information flow of the original features.

[0038] It should be noted that the input feature map of the spatial-channel collaborative attention module comes from the preliminary fusion result of the two input features. Preliminary fusion refers to the low-resolution feature map being upsampled and spatially aligned with its higher-level feature map before being added together. During upsampling, the two input features come from the last two layers of the backbone network output, or from the intermediate layers of the backbone network output and the fused features of the previous scale jointly output by the previous spatial-channel collaborative attention module and the enhanced multi-level feature reconstruction module during upsampling. During downsampling, the two input features come from the fused features of the same scale in upsampling and the fused features of the previous scale jointly output by the previous spatial-channel collaborative attention module and the enhanced multi-level feature reconstruction module during downsampling.

[0039] In this embodiment, the EMFR module is designed to adaptively fuse and reconstruct two input features. The core idea is to separate the features based on their own attributes (strength) before fusing them. The aim is to enhance the edge information of small targets. Specifically, this includes: Feature preprocessing and weight generation: First, the two input feature maps x1 and x2 are respectively channel aligned and initially transformed through independent 1×1 convolutional layers to obtain feature feat1 and feature feat2; then, feature feat1 and feature feat2 are batch normalized and generated through the Sigmoid function to generate two independent normalized weight maps weight1 and weight2.

[0040] Feature separation: Adding features feat1 and feat2 yields a preliminary fusion result. feat A threshold generator consisting of global average pooling, two 1×1 convolutional layers, and a sigmoid function is applied to the initial fused result. featA dynamic adaptive threshold map, `threshold`, is generated. `weight1` and `weight2` are compared with this `threshold` map, generating four binary masks: `mask_strong1`, `mask_weak1`, `mask_strong2`, and `mask_weak2`. This distinguishes the two feature paths into strong features (weight ≥ threshold) and weak features (weight < threshold). The two feature paths are then weighted and summed using the binary masks to obtain the fused strong feature. feat With weak features feat .

[0041] Feature transformation: (1) Strong feature transformation: Strong features are considered to have significant information. A simple 1×1 convolution is used to transform the strong features. feat Perform a transformation to obtain the transformed strong feature. transformed (2) Weak feature transformation: Weak features may contain details or noise. Lighter depthwise separable convolutions are used to transform weak features. feat The process involves first performing a 3×3 depthwise convolution with the number of groups equal to the number of channels to capture spatial information, and then performing a 1×1 pointwise convolution to fuse the channel information, resulting in the transformed weak features. transformed .

[0042] Gated weighted fusion: The separated original weak features are then processed. feat Apply a gate generator (similar in structure to the threshold generator, but using Softmax activation) to generate soft attention weights in the channel dimension. weights Used to weaken the transformed weak features transformed Weighing the data yields the final weak feature. final Ultimately, the transformed strong features will be... transformed The final weak feature after gating weighting final The features are summed to output the reconstructed features at multiple scales and levels. This process enables adaptive selection and enhancement of features.

[0043] The attention-enhanced features output by the SCJA module and the corresponding reconstructed features output by the EMFR module are added together to obtain high-scale fused features at each level, which are then output. The final fused features are then downsampled to generate low-scale downsampled features at each level, which are then output.

[0044] In remote sensing images, detectable targets are typically distributed only in extremely sparse local regions, with most of the image area consisting of background unrelated to the targets. Performing dense convolution calculations uniformly across the entire image, especially in computationally demanding detection networks, would consume a significant amount of computational resources in sparse background regions, contributing little to detection performance and severely impacting inference efficiency. Therefore, a DMS module capable of calculating only the foreground region was designed into the detection network. During sparse convolution calculations, the core mechanism is guided by a binary mask feature, MaskH. This feature explicitly defines the effective computation area: only when MaskH is 1 at a given spatial location will the data at that location participate in the convolution operation; locations with a mask value of 0 are completely skipped and do not undergo any calculation. This mask-driven, selective computation approach allows the convolution operation to automatically focus on predefined effective regions, significantly reducing computational overhead on zero-value or invalid background regions. Especially in remote sensing image analysis, image content often contains large areas of invalid background and sparsely distributed targets. Sparse convolution significantly reduces the amount of computation by avoiding redundant operations on broad background areas, effectively improving the real-time performance of detection models.

[0045] In this embodiment, the detection network incorporates a DMS module. The core objective of the DMS module is to enhance input features through a learnable, task-oriented hard mask and reduce the computational load of the detection network through sparse convolution. Figure 4 As shown, the entire process begins with the input from the fused features and subsampled features Fin, derived from the downsampling process of the neck network. The input flows along two paths: the main path first extracts basic features through a 3×3 convolution, then enters the Adaptive Masking Module (AMM). In the AMM, the extracted features are combined with random noise (such as Gumbel noise) to generate a continuous soft mask, representing the probability of attention at each location. This soft mask is then thresholded (e.g., 0.5) and discretized into a binary hard mask (Hard Mask) that explicitly specifies important and unimportant regions, under the direct supervision of the ground truth mask label Mask GT. On the other parallel path, the input features Fin are lightly processed through a 1×1 convolution. The resulting hard mask acts as a dynamic spatial selector, inputting along with the lightly processed output from this path into a sparse convolution. This allows the sparse convolution computation to focus on the key regions indicated by the hard mask, thus achieving selective feature enhancement. Finally, the masked and enhanced features are fused with the multi-scale features of the original input through a concatenation operation, ultimately producing a more discriminative sparse feature F. outIn summary, the brilliance of the DMS module lies in its differentiable learning mechanism, which progresses from soft to hard. It utilizes Gumbel noise, enabling the model to learn to generate a discrete decision mask during training, guided by the true labels. This hard mask not only makes the attention mechanism more explicit and interpretable but also achieves efficient allocation of computational resources by controlling sparse convolutions, ultimately outputting an enhanced feature that preserves key information while suppressing irrelevant background.

[0046] Building upon the DMS module, a multi-scale sparse feature fusion architecture was constructed to further improve detection accuracy. This architecture first processes the multi-scale features generated by the neck network at each level into independent DMS modules, extracting sparse features of key focused regions at each scale through adaptive spatial masks. Then, in a bottom-up approach, lower-level (high-resolution) sparse features are downsampled and concatenated with adjacent upper-level (lower-resolution but semantically stronger) sparse features, thereby transferring and fusing detail and semantic information across multiple layers. Finally, the information-rich multi-layer sparse features, fused step-by-step, are fed into the detection head to uniformly complete classification and bounding box regression tasks. This design enables the detection head to simultaneously utilize salient region information at multiple scales, enhancing its ability to identify and locate targets of different sizes.

[0047] The target detection network constructed above is used to perform target detection on the input UAV remote sensing images. Specifically, it includes: Input the UAV remote sensing images into the target detection network; The image is first processed by a backbone network to extract features at multiple scales, resulting in feature maps C3, C4, and C5. Starting from feature map C5, it is input into the target activation network OSK to generate an object activation map corresponding to the input image space. The object activation map is divided into a 4×4 grid, and the target region is selected based on the average activation value ≥0.3. The corresponding coordinates are mapped to layers C3, C4, and C5 and stacked into three batch tensors, which are used as the input of the subsequent neck network. Subsequently, feature maps C3, C4, and C5 are fed into the neck network. In the neck network, they are processed sequentially through the SCJA module and the EMFR module to obtain enhanced fusion features P3 and P4 at each level. Then, P4 and C5 are processed by the SCJA module and the EMFR module to obtain Pt. Pt is then downsampled at multiple levels to obtain features P5, P6, and P7, which are then output. Finally, the features output by the neck network are input into the detection network. In the detection network: each level of features is processed by a DMS module to generate a hard mask for the key region at that scale, and this mask is used to guide sparse convolution to obtain enhanced sparse features; a bottom-up approach is adopted to downsample the lower-level sparse features and concatenate them with adjacent higher-level features to achieve multi-scale sparse feature fusion; classification and bounding box regression are performed on the fused final features to output the target detection results, and the detection boxes are drawn on the original image.

[0048] In this embodiment, the above-described high-resolution UAV multi-scale remote sensing image target detection method is evaluated on the VisDrone dataset, which is widely used in the field of UAV target detection. The VisDrone dataset contains a large number of complex images collected from the perspective of UAVs, and its targets (such as pedestrians and vehicles) are usually small in size and densely distributed, posing a dual challenge to the detector's recognition accuracy and computational efficiency.

[0049] Table 1

[0050] Where AP represents Average Precision, a metric that comprehensively measures the precision and recall of an object detection model; AP50 represents the average precision when the intersection-union ratio (IU) threshold is 0.5; FPS represents the number of frames processed per second, used to evaluate model speed; and s / img represents the number of seconds required to process a single image, another commonly used way to represent model efficiency. As shown in Table 1, the performance comparison results demonstrate that the method of this invention achieves the best overall performance, with its average precision AP (29.3) and AP50 (52.1) both surpassing other comparative methods. This proves the effectiveness of the collaborative work of OSK target region selection, the multi-scale feature enhancement of the SCJA and EMFR modules, and the sparse computation guided by the DMS module, maintaining a high processing speed (12.04 FPS) while improving accuracy.

[0051] Module ablation analysis: Comparing the data of OSK+DMS (containing only the target activation network and sparse detection module) and the method of this invention, it can be found that although the former achieves the fastest inference speed (15.38 FPS) due to its highly sparsity computation, its accuracy (AP 27.6) is significantly reduced. This clearly shows that the SCJA module and EMFR module introduced into the neck network are crucial for improving feature quality, especially enhancing the detail information of small targets, and are the key to improving accuracy.

[0052] Efficiency advantages: Compared with the traditional two-stage method ClusDet, all variants of the method of this invention have an order of magnitude improvement in speed; and compared with Yolc, which is also a single-stage design, this solution maintains higher accuracy while significantly leading in frame rate (FPS), demonstrating its excellent ability to balance accuracy and speed.

[0053] Based on the same inventive concept, the embodiments also provide a high-resolution UAV multi-scale remote sensing image target detection device, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the above-mentioned high-resolution UAV multi-scale remote sensing image target detection method, specifically including the following steps: S1, Construct a target detection network that includes a backbone network, a neck network, and a detection network; S2 utilizes the constructed target detection network to perform target detection on the input UAV remote sensing images.

[0054] The computing device provided in this embodiment, at the hardware level, includes not only a processor and memory, but also internal buses, network interfaces, memory, and other hardware required for business operations. The memory is non-volatile memory. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the high-resolution UAV multi-scale remote sensing image target detection method described in S1-S2 above. Of course, besides software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0055] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method, characterized in that, The method comprises the following steps: A target detection network is constructed, which comprises a backbone network, a neck network and a detection network. The target activation network is introduced into the backbone network to generate an object activation map. The object activation map is used to cut out the region blocks containing targets from the image features output by the backbone network through uniform cutting and mapping. The spatial-channel collaborative attention module and the enhanced multi-level feature reconstruction module are introduced into the neck network to process the region blocks to obtain the high-scale fusion features and the low-scale down-sampling features in the down-sampling process and output. The dynamic mask sparse calculation module is introduced into the detection network to generate the hard mask of the key region in the output features of the neck network. The hard mask is used to guide the sparse convolution calculation of the output features of the neck network to obtain the sparse features. The multi-scale sparse feature fusion is performed on the sparse features, and the classification and the bounding box regression are performed on the fused features through the detection head to output the target detection result. The constructed target detection network is used to perform target detection on the input unmanned aerial vehicle remote sensing image. 2.The high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method according to claim 1, characterized in that, The target activation network is composed of a depth separable convolution block and a convolution, and is used to perform forward propagation based on the last layer feature map output by the backbone network and predict an object activation map. During training, the axial independent Gaussian kernel is used to generate a Gaussian mask used as a pseudo label according to the bounding box label. The binary cross entropy loss is calculated between the predicted object activation map and the Gaussian mask. The network weight is updated through back propagation. The object activation map is used to cut out the region blocks containing targets from the image features output by the backbone network through uniform cutting and mapping. The object activation map is divided into grids based on a threshold, and the target regions are screened out from the grids. Then, the coordinates of the target regions are mapped to the image features output by the backbone network to cut out the region blocks containing targets. 3.The high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method according to claim 1, characterized in that, The spatial-channel collaborative attention module and the enhanced multi-level feature reconstruction module process two input features in parallel. In the up-sampling process, the two input features come from the last two layers of the region blocks output by the backbone network or come from the intermediate layer region blocks output by the backbone network and the fusion features of the previous scale in the up-sampling process. In the down-sampling process, the two input features come from the fusion features of the same scale in the up-sampling and the fusion features of the previous scale in the down-sampling process. The spatial-channel collaborative attention module models the mutual dependence between the spatial and channel dimensions of the two input features to obtain attention enhanced features. The enhanced multi-level feature reconstruction module adaptively separates, fuses and reconstructs the two input features according to the strength attributes of the features to obtain reconstructed features. In the down-sampling process, the attention enhanced features and the reconstructed features of each level are fused to obtain the high-scale fusion features of each level and output. The fusion features of the last level are further down-sampled to obtain the low-scale down-sampling features and output.

4. The high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method according to claim 3, characterized in that, The spatial-channel collaborative attention module models the mutual dependence between the spatial and channel dimensions of the two input features to obtain attention enhanced features, which comprises: The preliminary fusion result of the two input features is used as the input feature map of the spatial-channel collaborative attention module. The channel attention branch: the channel attention weight is calculated by global average pooling, multi-layer perceptron and activation function on the input feature map; The spatial attention branch: the mean and maximum of the input feature map are calculated respectively, then the channel dimension is spliced, and the spatial attention weight is generated after convolution and activation; Cooperative weighting and feature interaction: the input feature map is multiplied by the channel attention weight and the spatial attention weight element by element to obtain the feature after double attention modulation, a grouped convolution is introduced, the modulated feature is grouped in the channel dimension, then independent convolution is performed in each group, finally it is recombined, and then batch normalization is connected, the input feature map is added to the feature after grouping interaction through the residual connection controlled by the learnable scaling parameter, and the final attention enhanced feature is output.

5. The high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method according to claim 3, characterized in that, The enhanced multi-level feature reconstruction module adaptively separates, fuses and reconstructs the two input features according to the strength attribute of the features to obtain the reconstructed feature, including: Feature preprocessing and weight generation: the two input features are respectively aligned in the channel and preliminarily transformed by the convolution layer, then batch normalization is performed and the activation function is passed to generate two independent normalized weight maps; Feature separation: the preliminary fusion result of the two input features generates a dynamic adaptive threshold map through the threshold generator, and four binary masks are generated by comparing the two normalized weight maps to distinguish the strong features and weak features of each input feature and obtain the fused strong features and weak features through binary mask weighted summation; Feature transformation: the strong features and weak features are respectively transformed by the convolution layer to obtain the transformed strong features and weak features; Gated weighted fusion: the weak features are separated and the soft attention weight in the channel dimension is generated by the gate generator to weight the transformed weak features to obtain the final weak features; the transformed strong features and the final weak features are added to output the reconstructed feature. 6.The high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method according to claim 5, characterized in that, The threshold generator adopts global average pooling, convolution layer and Sigmoid function; The gate generator adopts global average pooling, convolution layer and Softmax function. 7.The high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method according to claim 1, characterized in that, The hard mask of the key region in the generated neck network output feature includes: The features of each scale output by the neck network are first extracted into the adaptive mask module after convolution, and the continuous soft mask representing the probability of each position being focused is generated after the extracted basic features are combined with random noise in the adaptive mask module. The soft mask is discretized into a hard mask that explicitly specifies important and unimportant regions through threshold processing and direct supervision of real mask labels. 8.The high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method according to claim 1, characterized in that, The sparse convolution calculation of the hard mask guided neck network output feature to obtain sparse features includes: The features of each scale output by the neck network are input into the sparse convolution after being processed by convolution, and the hard mask is used as a dynamic selector, so that the sparse convolution calculation can focus on the key region indicated by the hard mask to obtain the sparse feature. 9.The high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method according to claim 1, characterized in that, The multi-scale sparse feature fusion and classification and bounding box regression of the sparse feature at each level are performed to output the target detection result, including: In a bottom-up manner, lower-level sparse features are down-sampled and spliced with adjacent upper-level sparse features, so as to pass and fuse the detail and semantic information between multiple levels; finally, the multi-layer sparse features after the step-by-step fusion are sent into a detection head, and the classification and bounding box regression tasks are completed uniformly, and multi-level target detection results are output.

10. A high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection device, comprising a memory and one or more processors, wherein the memory stores executable code, and the executable code comprises the following steps: The one or more processors, when executing the executable code, are configured to implement the high-resolution unmanned aerial vehicle multi-scale remote sensing image target detection method in any one of claims 1-9. ​

Images