Full-scale semantic enhancement-based high-resolution small target detection method and system

Abstract

The application discloses a full-scale semantic enhancement type high-resolution small target detection method and system based on full-scale semantics, and belongs to the technical field of image or video recognition. The method comprises the following steps: extracting four groups of backbone feature maps of different scales from a remote sensing image; constructing a light four-cascade feature pyramid to generate a backbone feature map of a P2 layer detection branch, performing multi-path aggregation, and generating path-aggregation-enhanced P3 to P5 layer feature maps; generating a global context feature map containing full-scale optimal information; performing double-path fusion to generate an enhanced P2 layer feature map; and feeding the enhanced P2 layer feature map and the path-aggregation-enhanced P3 to P5 layer feature maps into a decoupled detection head together, respectively predicting the class probability and the bounding box coordinates of targets of different scales, and outputting a final small target detection result. The application effectively solves the problems of easy loss of micro and small target features, great scale difference and serious complex background interference in the remote sensing image.

Description

Technical Field

[0001] This invention relates to the field of image or video recognition technology, and in particular to a method and system for detecting small targets at high resolution based on full-scale semantic enhancement. Background Technology

[0002] Remote sensing image target detection is a core technology for automatically locating and identifying specific targets from satellite or aerial imagery. It features a wide detection range, rapid data updates, and strong objectivity. However, in practical applications (such as large-scale datasets like Dior), remote sensing image target detection still suffers from drawbacks such as high false negative rates for small targets, numerous false positives against complex backgrounds, and low accuracy due to large differences in target scale.

[0003] Compared to traditional manual visual interpretation, deep learning-based detection algorithms (such as the YOLO series) add automated feature extraction capabilities to the detection head, feature fusion network (Neck), and backbone network, enabling rapid end-to-end recognition. YOLOv8n, as a lightweight model, uses compact modules such as C2f with few parameters, resulting in fast inference speed and ease of deployment. During detection, YOLOv8n extracts features using the backbone network and fuses multi-scale feature maps (P3, P4, P5, etc.) through a feature pyramid network (such as PANet) to achieve prediction. However, in pursuit of speed, lightweight models like YOLOv8n often discard shallow, high-resolution feature maps such as the P2 layer (4x downsampling), causing the model to lose crucial spatial details for small targets in the early stages of feature extraction, thus reducing detection capability.

[0004] In existing literature, target detection in remote sensing images mainly employs the standard YOLOv8n model or fine-tunes it on the backbone network, using the standard PANet structure for multi-scale feature fusion. While this has achieved basic detection results on some general datasets, its performance (e.g., mAP) drops significantly when facing remote sensing scenes with complex backgrounds and densely packed small targets. In practical detection scenarios, this architecture design leads to two significant problems: First, for densely packed small targets (such as vehicles and ships) in remote sensing images, the receptive field of deep features is too large and the spatial resolution is insufficient, resulting in weak target feature responses and difficulty in improving recall. Second, when facing complex backgrounds (such as dense building clusters or water ripples), simple linear feature fusion is insufficient to effectively suppress background noise, leading to low feature discrimination between target features and background textures, making it difficult to guarantee a high average precision (mAP). Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a high-resolution small target detection method and system based on full-scale semantic enhancement, which effectively solves the problems of easy loss of small target features, huge scale differences and severe interference from complex backgrounds in remote sensing images.

[0006] This invention is achieved through the following technical solution:

[0007] The method for detecting small targets at high resolution based on full-scale semantic enhancement includes the following steps:

[0008] S1: After preprocessing the remote sensing images in the Dior remote sensing image dataset, input them into the backbone network. The backbone network performs convolution downsampling layer by layer to extract four sets of backbone feature maps at different scales in layers P2, P3, P4 and P5 respectively.

[0009] S2: Based on four sets of backbone feature maps of different scales, a lightweight four-level linkage feature pyramid is constructed, including a top-down semantic injection path and a bottom-up position enhancement path. In the top-down semantic injection path, a non-linear fusion mechanism is used to generate the backbone feature map of the P2 layer detection branch. In the bottom-up position enhancement path, the backbone feature map of the P2 layer detection branch is used as the starting point for downsampling at each level, and multi-path aggregation is performed with the backbone feature maps of the P3, P4 and P5 layers to generate path-aggregated and enhanced P3, P4 and P5 layer feature maps.

[0010] S3: The scale sequence feature extractor unifies the channels of four sets of backbone feature maps at different scales and aligns them to the resolution of the backbone feature map at layer P2. It then constructs a five-dimensional scale sequence tensor stacked along the scale dimension. The five-dimensional scale sequence tensor is subjected to 3D max pooling operation. While retaining the spatial dimension, the maximum response value is selected along the scale dimension. Then, an internal dimensionality reduction and compression operation is performed to obtain a global context feature map containing the optimal information at all scales.

[0011] S4: The global context feature map containing optimal information across the entire scale is fused with the backbone feature map of the P2 layer detection branch through dual-path fusion to generate an enhanced P2 layer feature map.

[0012] S5: The enhanced P2 layer feature map, along with the path-aggregated enhanced P3, P4, and P5 layer feature maps, are fed into the decoupled detection head to predict the class probability and bounding box coordinates of targets at different scales, and output the final small target detection results.

[0013] In the optimized step S1, adaptive image scaling and normalization methods are used to preprocess the remote sensing images in the Dior remote sensing image dataset.

[0014] Furthermore, the method for performing convolutional downsampling layer by layer in step S1 of the backbone network is as follows:

[0015] The backbone feature map of layer P2 was downsampled by 4 times, with a resolution of [missing value]. ;

[0016] The backbone feature map of layer P3 was downsampled by 8 times, with a resolution of [missing value]. ;

[0017] The backbone feature map of layer P4 was downsampled by 16 times, with a resolution of [resolution value missing]. ;

[0018] The backbone feature map of layer P5 was downsampled by 32 times, with a resolution of [missing information]. .

[0019] Furthermore, the method for generating the backbone feature map of the P2 layer detection branch in step S2 is as follows:

[0020] S211: Upsample the P5 layer backbone feature map output by the backbone network, and use the first Zoom_cat module to perform non-linear fusion with the P4 layer backbone feature map output by the backbone network to generate an intermediate feature map of P4 layer carrying the semantics of P5 layer.

[0021] S212: The intermediate feature map of layer P4 carrying the semantics of layer P5 is further upsampled, and nonlinear fusion is performed with the backbone feature map of layer P3 output by the second Zoom_cat module to generate the intermediate feature map of layer P3.

[0022] S213: Upsample the intermediate feature map of P3 layer and use the third Zoom_cat module to perform nonlinear fusion with the backbone feature map of P2 layer output by the backbone network to generate the backbone feature map of the detection branch of P2 layer.

[0023] Furthermore, the method for generating the path aggregation enhanced feature maps of layers P3, P4, and P5 in step S2 is as follows:

[0024] S221: Starting with the feature map of the main branch of the detection branch in the P2 layer, perform a convolutional downsampling operation with a stride of 2 to obtain a P3 layer downsampled feature map with the same resolution as the P3 layer backbone feature map extracted in step S1. Then, the P3 layer downsampled feature map and the P3 layer intermediate feature map are concatenated in the channel dimension and aggregated through the C2f module to generate a path-aggregated enhanced P3 layer feature map.

[0025] S222: Take the path-aggregated enhanced P3 layer feature map as input, perform a convolution downsampling operation with a stride of 2 to obtain a P4 layer downsampled feature map with the same resolution as the P4 layer backbone feature map extracted in step S1. Then, concatenate the P4 layer downsampled feature map with the P4 layer intermediate feature map in the channel dimension, and aggregate them through the C2f module to generate a path-aggregated enhanced P4 layer feature map.

[0026] S223: Using the path-aggregated enhanced P4 layer feature map as input, perform a convolutional downsampling operation with a stride of 2 to obtain a P5 layer downsampled feature map with the same resolution as the P5 layer backbone feature map extracted in step S1. Then, concatenate the P5 layer downsampled feature map with the P5 layer intermediate feature map in the channel dimension, and aggregate them through the C2f module to generate a path-aggregated enhanced P5 layer feature map.

[0027] Furthermore, the method for generating the global context feature map containing optimal information across all scales in step S3 is as follows:

[0028] S311: The scale sequence feature extractor receives four sets of backbone feature maps at different scales from layers P2, P3, P4, and P5 of the backbone network. It unifies the number of channels in the four sets of backbone feature maps at different scales and upsamples the three sets of backbone feature maps at different scales in layers P3, P4, and P5 to the same resolution size as the backbone feature map in layer P2, thus obtaining high-resolution aligned feature maps in layers P3, P4, and P5.

[0029] S312: Stack the backbone feature map of layer P2, the high-resolution aligned feature map of layer P3, the high-resolution aligned feature map of layer P4, and the high-resolution aligned feature map of layer P5 in the scale dimension to construct a 5-dimensional scale sequence tensor of shape [B,C,4,H,W], where: B represents the batch size, C represents the number of channels, 4 represents the scale dimension, H represents the feature map height, and W represents the feature map width;

[0030] S313: Perform 3D max pooling on the 5-dimensional scale sequence tensor to filter out the maximum response value across four scales at each spatial location, and obtain a feature map with the largest response value.

[0031] S314: The feature map with the largest response value is subjected to dimensionality reduction to obtain a global context feature map containing optimal information across the entire scale.

[0032] Furthermore, in step S314, global context features containing optimal information across the entire scale are extracted according to equation (1), resulting in a global context feature map containing optimal information across the entire scale:

[0033] (1);

[0034] in: This represents a global context feature that contains optimal information across the entire scale. This indicates a dimensionality reduction operation. Represents a high-dimensional scale sequence tensor. This represents the three-dimensional max pooling operation.

[0035] Furthermore, the method for generating the enhanced P2 layer feature map in S4 is as follows:

[0036] S411: The backbone feature map of the detection branch of the P2 layer is first passed through adaptive one-dimensional convolution to capture cross-channel interactions without dimensionality reduction, generate channel weights, and suppress background noise channels to obtain the refined P2 layer feature map.

[0037] S412: Add the global context feature map containing the optimal information across the entire scale to the refined P2 layer feature map element by element to form the fused feature map;

[0038] S413: The fused feature map is aggregated along the horizontal and vertical directions through the coordinate space attention mechanism to generate an accurate spatial position weight map, which is used to spatially relocate and focus on small targets, and generate an enhanced P2 layer feature map.

[0039] In the optimized step S411, channel weights are generated according to equation (2), and the refined feature map of layer P2 is obtained according to equation (3):

[0040] (2);

[0041] (3);

[0042] in: Indicates channel weight, This represents the Sigmoid activation function. This represents a one-dimensional convolution operation. Indicates global average pooling. Indicates the input features of the main path. This represents the refined features of layer P2. This indicates the adaptive one-dimensional convolution kernel size. This indicates element-wise multiplication.

[0043] The system is based on a full-scale semantic enhancement high-resolution small target detection system, used to execute the full-scale semantic enhancement high-resolution small target detection method as described above, which includes an image preprocessing unit, a backbone network, a lightweight four-level linkage feature pyramid construction unit, a scale sequence feature extractor, a dual-path attention fusion unit, and a decoupled detection head.

[0044] The image preprocessing unit is used to preprocess remote sensing images in the Dior remote sensing image dataset; the backbone network is used to perform convolutional downsampling on the preprocessed image layer by layer, and extract four sets of backbone feature maps of different scales at layers P2, P3, P4 and P5 respectively.

[0045] The lightweight four-level linkage feature pyramid construction unit is used to construct a lightweight four-level linkage feature pyramid, generating the main feature map of the P2 layer detection branch and the path aggregation enhanced feature maps of the P3, P4, and P5 layers.

[0046] The scale sequence feature extractor is used to extract features from four sets of backbone feature maps of different scales from layers P2, P3, P4 and P5 of the backbone network, and generate a global context feature map containing optimal information across all scales.

[0047] The dual-path attention fusion unit is used to fuse the global context feature map containing full-scale optimal information with the backbone feature map of the P2 layer detection branch to generate an enhanced P2 layer feature map.

[0048] The decoupled detection head predicts the class probability and bounding box coordinates of targets at different scales based on the enhanced P2 layer feature map and the path-aggregated enhanced P3, P4, and P5 layer feature maps, and outputs the final small target detection result.

[0049] Beneficial effects of the invention:

[0050] The method and system for detecting small targets based on full-scale semantic enhancement provided by this invention have the following advantages:

[0051] 1. It breaks through the physical bottleneck of perceiving tiny targets and solves the contradiction between "preserving high-frequency details" and "semantic consistency".

[0052] 2. A full-scale context interaction mechanism was established, which effectively suppressed noise interference from complex backgrounds.

[0053] 3. It achieves architectural lightweighting by "reversely reducing the number of parameters" and has excellent edge deployment potential.

[0054] 4. A "full-scale semantically enhanced" cascaded detection system was constructed to balance the detection needs of multi-scale targets. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the process of this invention.

[0056] Figure 2 This is a diagram showing the receptive field effect of the baseline model.

[0057] Figure 3 This is a rendering of the receptive field of the ASF-YOLO model.

[0058] Figure 4 This is a diagram showing the receptive field effect of the ASF-YOLO-4H model of the present invention.

[0059] Figure 5This is a comparison of the original image and the predicted result.

[0060] Figure 6 This is a heat map of the baseline model.

[0061] Figure 7 This is a heatmap of the ASF-YOLO model.

[0062] Figure 8 This is the thermal diagram of the ASF-YOLO-4H model of the present invention. Detailed Implementation

[0063] The flowchart of the full-scale semantically enhanced high-resolution small target detection method is as follows: Figure 1 As shown, it includes the following steps:

[0064] S1: After preprocessing the remote sensing images in the Dior remote sensing image dataset, input them into the backbone network. The backbone network performs convolution downsampling layer by layer to extract four sets of backbone feature maps at different scales in layers P2, P3, P4 and P5 respectively.

[0065] Specifically, adaptive image scaling and normalization methods can be used to preprocess the remote sensing images in the Dior remote sensing image dataset to prevent geometric distortion and ensure they meet the network input requirements. Because targets in remote sensing images, such as airplanes and storage tanks, are highly sensitive to shape, directly stretching them to 640x640 will cause circular storage tanks to become elliptical, leading to distorted model learning.

[0066] Furthermore, the method for performing convolutional downsampling layer by layer in the backbone network is as follows:

[0067] The backbone feature map of layer P2 was downsampled by 4 times, with a resolution of [missing value]. ;

[0068] The backbone feature map of layer P3 was downsampled by 8 times, with a resolution of [missing value]. ;

[0069] The backbone feature map of layer P4 was downsampled by 16 times, with a resolution of [resolution value missing]. ;

[0070] The backbone feature map of layer P5 was downsampled by 32 times, with a resolution of [missing information]. .

[0071] Among them: P2 layer is a fine-grained spatial perception layer, containing rich fine-grained spatial textures for the perception of small targets; P3 to P5 layers are deep semantic representation layers, with P5 layer containing highly abstract semantic representations.

[0072] S2: A lightweight four-level linked feature pyramid is constructed based on four sets of backbone feature maps at different scales, including a top-down semantic injection path and a bottom-up position enhancement path. In the top-down semantic injection path, a non-linear fusion mechanism is used to generate the backbone feature map of the P2 layer detection branch. In the bottom-up position enhancement path, the backbone feature map of the P2 layer detection branch is used as the starting point for downsampling at each level, and multi-path aggregation is performed with the backbone feature maps of the P3, P4, and P5 layers to generate path-aggregated and enhanced P3, P4, and P5 layer feature maps. Specifically, the method for generating the backbone feature map of the P2 layer detection branch is as follows:

[0073] S211: Upsample the P5 layer backbone feature map output by the backbone network, and use the first Zoom_cat module to perform non-linear fusion with the P4 layer backbone feature map output by the backbone network to generate an intermediate feature map of P4 layer carrying the semantics of P5 layer.

[0074] S212: The intermediate feature map of layer P4 carrying the semantics of layer P5 is further upsampled, and nonlinear fusion is performed with the backbone feature map of layer P3 output by the second Zoom_cat module to generate the intermediate feature map of layer P3.

[0075] S213: Upsample the intermediate feature map of P3 layer and use the third Zoom_cat module to perform nonlinear fusion with the backbone feature map of P2 layer output by the backbone network to generate the backbone feature map of the detection branch of P2 layer.

[0076] This invention adopts a top-down semantic injection path to construct the backbone feature map of the P2 layer detection branch, aiming to transfer strong semantic information from the deep layer to the shallow layer. The backbone feature map of the P2 layer detection branch undergoes three semantic injections, which not only has a high-resolution texture with 4 times downsampling, but also inherits the deep semantic guidance from the P5 layer.

[0077] Furthermore, the method for generating the path aggregation enhanced feature maps of layers P3, P4, and P5 in step S2 is as follows:

[0078] S221: Starting with the feature map of the main branch of the detection branch in the P2 layer, perform a convolutional downsampling operation with a stride of 2 to obtain a P3 layer downsampled feature map with the same resolution as the P3 layer backbone feature map extracted in step S1. Then, the P3 layer downsampled feature map and the P3 layer intermediate feature map are concatenated in the channel dimension and aggregated through the C2f module to generate a path-aggregated enhanced P3 layer feature map.

[0079] S222: Take the path-aggregated enhanced P3 layer feature map as input, perform a convolution downsampling operation with a stride of 2 to obtain a P4 layer downsampled feature map with the same resolution as the P4 layer backbone feature map extracted in step S1. Then, concatenate the P4 layer downsampled feature map with the P4 layer intermediate feature map in the channel dimension, and aggregate them through the C2f module to generate a path-aggregated enhanced P4 layer feature map.

[0080] S223: Using the path-aggregated enhanced P4 layer feature map as input, perform a convolutional downsampling operation with a stride of 2 to obtain a P5 layer downsampled feature map with the same resolution as the P5 layer backbone feature map extracted in step S1. Then, concatenate the P5 layer downsampled feature map with the P5 layer intermediate feature map in the channel dimension, and aggregate them through the C2f module to generate a path-aggregated enhanced P5 layer feature map.

[0081] This invention employs bottom-up location enhancement path reconstruction of the branches from layer P3 to layer P5, which can compensate for the shortcomings of deep features in spatial localization, enabling the generated path-aggregated and enhanced feature maps of layers P3, P4, and P5 to achieve details that combine both "semantic" and "spatial" aspects.

[0082] By employing the aforementioned full-scale path aggregation architecture, four sets of backbone feature maps of different scales (P2, P3, P4, and P5) are reconstructed through two paths: top-down and bottom-up. Unlike existing technologies that only use P3-P5 and have bloated inter-layer connections, this invention reconstructs a compact and efficient backbone feature map of the P2 layer detection branch in the neck network. During the fusion process, a channel compression strategy is implemented to establish a high-resolution, low-parameter information path. This ensures that the spatial information of small targets is not filtered out by the downsampling operation of deep networks, and effectively reduces feature channel redundancy while preserving texture details, enabling cross-scale deep fusion of features.

[0083] S3: The scale sequence feature extractor unifies the channels of four sets of backbone feature maps at different scales and aligns them to the resolution of the backbone feature map at layer P2. It then constructs a five-dimensional scale sequence tensor stacked along the scale dimension. The five-dimensional scale sequence tensor is subjected to 3D max pooling operation. While retaining the spatial dimension, the maximum response value is selected along the scale dimension. Then, an internal dimensionality reduction and compression operation is performed to obtain a global context feature map containing the optimal information at all scales.

[0084] The ScalSeq feature extractor is configured to build a full-dimensional feature interaction hub. Utilizing high-dimensional tensor stacking technology, it bridges the information gap between shallow details and deep semantics, achieving lossless interaction of information flow across the entire scale from P2 to P5. Its role is to compensate for the loss of information transmission over long distances in traditional feature pyramids, ensuring that small targets can acquire deep semantic support, while large-scale targets can acquire shallow boundary constraints.

[0085] Specifically, the method for generating a global context feature map containing optimal information across all scales is as follows:

[0086] S311: The scale sequence feature extractor receives four sets of backbone feature maps at different scales from layers P2, P3, P4, and P5 of the backbone network. It unifies the number of channels in the four sets of backbone feature maps at different scales and upsamples the three sets of backbone feature maps at different scales in layers P3, P4, and P5 to the same resolution size as the backbone feature map in layer P2, thus obtaining high-resolution aligned feature maps in layers P3, P4, and P5.

[0087] The scale sequence feature extractor receives four sets of backbone feature maps at different scales from layers P2, P3, P4, and P5 of the backbone network. First, it... Convolution unifies the feature maps of each layer to a preset number of channels, and aligns the spatial resolution to the target scale (such as the P2 scale) through upsampling or dynamic sampling.

[0088] S312: Stack the backbone feature map of layer P2, the high-resolution aligned feature map of layer P3, the high-resolution aligned feature map of layer P4, and the high-resolution aligned feature map of layer P5 in the scale dimension to construct a 5-dimensional scale sequence tensor of shape [B,C,4,H,W], where: B represents the batch size, C represents the number of channels, 4 represents the scale dimension, H represents the feature map height, and W represents the feature map width;

[0089] S313: Perform 3D max pooling on the 5-dimensional scale sequence tensor to filter out the maximum response value across four scales at each spatial location, and obtain a feature map with the largest response value.

[0090] S314: The feature map with the largest response value is subjected to dimensionality reduction to obtain a global context feature map containing optimal information across the entire scale.

[0091] Specifically, global context features containing optimal information across all scales can be extracted according to equation (1) to obtain a global context feature map containing optimal information across all scales:

[0092] (1);

[0093] in: This represents a global context feature that contains optimal information across the entire scale. This indicates a dimensionality reduction operation. Represents a high-dimensional scale sequence tensor. This represents the three-dimensional max pooling operation.

[0094] The above operations solve the feature coupling problem caused by the large target scale span in remote sensing images. A scale sequence extractor is introduced to select the optimal semantic features from the full scale space, in order to prepare for subsequent fusion.

[0095] S4: The global context feature map containing optimal information across the entire scale is fused with the backbone feature map of the P2 layer detection branch through dual-path fusion to generate an enhanced P2 layer feature map.

[0096] The dual-path attention fusion unit (asf_attention_model) aims to enhance the multidimensional saliency of features. Utilizing an asymmetric sequential processing flow, this unit first suppresses redundant background noise channels through channel attention, then introduces global context for feature supplementation, and finally performs precise calibration of the target location through spatial attention. Its role is to significantly improve the model's feature signal-to-noise ratio under complex background interference, ensuring that the model can accurately lock onto small targets in noisy environments.

[0097] Specifically, the method for generating the enhanced P2 layer feature map is as follows:

[0098] S411: The main feature map of the detection branch of the P2 layer is first passed through adaptive one-dimensional convolution to capture cross-channel interactions without dimensionality reduction, generate channel weights, and suppress background noise channels to obtain a refined P2 layer feature map. This step can further refine the path.

[0099] The backbone feature map of the P2 layer detection branch first aggregates spatial information using global average pooling (GAP) through adaptive channel attention subunits, and captures cross-channel interaction information through adaptive one-dimensional convolution.

[0100] Adaptive one-dimensional convolution kernel size Depend on Adaptive determination, where: Indicates the number of channels. This represents the linear offset, with a preferred value of 1. This represents the proportionality coefficient, with a preferred value of 2. This indicates the operation of taking the nearest odd number.

[0101] Preferably, channel weights can be generated according to equation (2), and the refined P2 layer feature map can be obtained according to equation (3):

[0102] (2);

[0103] (3);

[0104] in: Indicates channel weight, This represents the Sigmoid activation function. This represents a one-dimensional convolution operation. Indicates global average pooling. Indicates the input features of the main path. This represents the refined features of layer P2. This indicates element-wise multiplication.

[0105] This step aims to enhance the channel saliency of the backbone features and suppress redundant channel responses.

[0106] S412: The global context feature map containing the best information at all scales is added element by element to the refined P2 layer feature map to form a fused feature map. This operation can make up for the lack of semantic information in shallow features and realize the lossless injection of cross-scale context information.

[0107] The expression is , Indicates the characteristics after fusion. This represents a global context feature that contains optimal information across the entire scale. This indicates element-wise addition.

[0108] S413: The fused feature map is aggregated along the horizontal and vertical directions through the coordinate space attention mechanism to generate an accurate spatial position weight map, which is used to spatially relocate and focus on small targets, and generate an enhanced P2 layer feature map.

[0109] To address the difficulty of locating small targets, the fused features The feature is then entered into the coordinate space attention sub-unit, which performs one-dimensional feature aggregation along the horizontal and vertical directions according to the following formula:

[0110] ;in: Represents the feature aggregation vector in the height direction. This represents the feature aggregation vector along the width direction. This represents one-dimensional average pooling along the height direction. This represents one-dimensional average pooling along the width direction.

[0111] Then through shared convolutional layers Feature transformation and nonlinear activation are performed according to the following formula:

[0112] ,in: This represents an intermediate feature that integrates spatial information in the height and width directions. express Convolution kernel; Represents a non-linear activation function. Indicates batch normalization;

[0113] Subsequently, the layers split and pass through independent convolutional layers to generate spatial attention maps in two directions according to the following formula:

[0114] ;in: The spatial position weights in the height direction are represented. This represents a 1×1 convolutional layer along the height direction. Indicates intermediate features in the height direction. Indicates the spatial position weight in the width direction. This represents a 1×1 convolutional layer in the width direction. This represents the intermediate feature in the width direction.

[0115] Final output features for , This is the enhanced feature map of layer P2.

[0116] This step enables targeted enhancement of P2 layer features using a dual-path attention fusion unit (asf_attention_model). This unit performs non-linear adaptive alignment, improving feature saliency without significantly increasing computational burden.

[0117] S5: The enhanced P2 layer feature map, along with the path-aggregated enhanced P3, P4, and P5 layer feature maps, are fed into the decoupled detection head to predict the class probability and bounding box coordinates of targets at different scales, and output the final small target detection results.

[0118] Through the deep collaboration of steps S1 to S5 described above, this invention is not a simple linear stacking of existing modules, but rather creates a novel "full-scale semantically enhanced fine-grained perception" network architecture through organic integration and structural reconstruction. This architecture breaks through the limitations of traditional methods that treat attention mechanisms as independent plugins, innovatively embedding a dual-path attention module deeply into the topology of the high-resolution branch of layer P2, making it a necessary structural component connecting shallow details and deep semantics. This design achieves a "dialectical unity of addition and subtraction" at the architectural level: while introducing high-frequency information to improve detection accuracy, it achieves a reverse reduction in the overall number of model parameters through channel compression and efficient operator strategies. Ultimately, this invention can achieve precise capture of tiny targets with better parameter efficiency, achieving a perfect balance between "high performance and computational friendliness," providing a novel solution for high-precision remote sensing detection in resource-constrained environments.

[0119] The system is based on a full-scale semantic enhancement high-resolution small target detection system, used to execute the full-scale semantic enhancement high-resolution small target detection method as described above, which includes an image preprocessing unit, a backbone network, a lightweight four-level linkage feature pyramid construction unit, a scale sequence feature extractor, a dual-path attention fusion unit, and a decoupled detection head.

[0120] The image preprocessing unit is used to preprocess remote sensing images in the Dior remote sensing image dataset; the backbone network is used to perform convolutional downsampling on the preprocessed image layer by layer, and extract four sets of backbone feature maps of different scales at layers P2, P3, P4 and P5 respectively.

[0121] The lightweight four-level linkage feature pyramid construction unit is used to construct a lightweight four-level linkage feature pyramid, generating the main feature map of the P2 layer detection branch and the path aggregation enhanced feature maps of the P3, P4, and P5 layers.

[0122] The scale sequence feature extractor is used to extract features from four sets of backbone feature maps of different scales from layers P2, P3, P4 and P5 of the backbone network, and generate a global context feature map containing optimal information across all scales.

[0123] The dual-path attention fusion unit is used to fuse the global context feature map containing full-scale optimal information with the backbone feature map of the P2 layer detection branch to generate an enhanced P2 layer feature map.

[0124] The decoupled detection head predicts the class probability and bounding box coordinates of targets at different scales based on the enhanced P2 layer feature map and the path-aggregated enhanced P3, P4, and P5 layer feature maps, and outputs the final small target detection result.

[0125] This invention innovatively expands the scope and topology of the ASF module, adapting it to a four-level feature pyramid. Unlike conventional ASF modules that only operate between layers P3-P5, this invention adaptively reconstructs the ScalSeq scale sequence extractor, extending its input dimension to the full-scale space covering layer P2. This design constructs a high-dimensional scale sequence tensor spanning 4x to 32x downsampling, enabling full-link information flow interaction from P2 to P5. This transforms the ASF module from a standalone plugin into a core bridge connecting shallow details and deep semantics, effectively bridging the significant semantic gap between layer P2 and the backbone network.

[0126] Secondly, this invention achieves a breakthrough in optimizing model efficiency. Typically, introducing additional detection heads (P2) and ASF modules leads to a surge in the number of model parameters and computational cost. However, this invention, through an optimized channel reduction strategy and operator reorganization, achieves a reverse reduction in the number of model parameters while constructing a four-level linkage architecture. This invention is not a simple stacking of existing technologies, but rather, through system-level architectural reconstruction, it finds an optimal balance between improving detection accuracy, enhancing anti-interference capabilities, and maintaining lightweight design, exhibiting a significant synergistic effect.

[0127] Experimental data and effect verification

[0128] To verify the effectiveness of this invention in target detection in remote sensing images, detailed ablation experiments were conducted on the Dior remote sensing image dataset.

[0129] The specific experimental data are shown in Tables 1, 2, and 3 respectively:

[0130] Table 1 YOLOv8 Training Data

[0131]

[0132] Table 2 ASF-YOLO Training Data

[0133]

[0134] Table 3 ASF-YOLO-4H Training Data

[0135]

[0136] Table 4 shows the performance comparison data between the benchmark model and the embodiment of the present invention, and Table 5 shows the detailed comparison data on model complexity (Parameters, GFLOPs).

[0137] Table 4 Comparison of Ablation Test Performance

[0138]

[0139] Table 5 Model Complexity

[0140]

[0141] In the table, YOLOv8 is the baseline model, ASF-YOLO is the existing model, and ASF-YOLO-4H is the model of this invention.

[0142] As can be seen from Tables 4 and 5, the model of this invention (ASF-YOLO-4H) improved the mAP50 index by 4.3 percentage points. (794), improving precision by 2.2 percentage points. The baseline model's recall was only 0.698, while this invention improved it to over 0.742. This indicates that the combination of the P2 detection head and the ASF module significantly recovered a large number of tiny targets missed by the baseline model.

[0143] Regarding the number of parameters: Although the number of layers in this invention's model has increased to 316 (due to the introduction of the P2 branch and the ASF module), its number of parameters (2.96M) is slightly lower than the baseline model (3.01M). This is because this invention employs a carefully designed channel reduction strategy when constructing the P2 branch, effectively controlling the model size while ensuring feature representation. This demonstrates that this invention is highly storage-friendly and very suitable for deployment on storage-constrained edge remote sensing devices.

[0144] Regarding computational complexity: The computational complexity of the model in this invention increased from 8.2 GFLOPs to 13.0 GFLOPs. This increase is mainly due to the P2 detection head's processing of high-resolution feature maps. The processing of small targets in remotely sensed images is considered. Given the extreme difficulty of detecting tiny targets in remotely sensed images, a reasonable increase in computational cost (approximately 4.8 GFLOPs) is made in exchange for a significant improvement in recall (0.698 GFLOPs). The 0.742 (and accuracy gains) are perfectly in line with the needs of practical applications. The number of training epochs can be extended; higher computational cost means the model processes a greater information density, requiring more iterations for optimization.

[0145] Qualitative analysis

[0146] To further verify the improved feature representation capability of the proposed module from the perspective of feature extraction mechanism, this invention utilizes the gradient backpropagation algorithm to visualize the effective receptive fields of the three models at key levels. The comparison diagrams of the receptive field effects of the three models are shown below. Figure 2 , Figure 3 , Figure 4 As shown in the figure, the green area represents the feature saliency area that contributes highly to the final prediction, while the yellow area represents the edge transition area and background area where the contribution gradually decreases.

[0147] (1) Baseline model: Specifically, as shown in Figure 2 As shown:

[0148] Visual observation: The central green high-response area exhibits a significant grid-like discrete distribution pattern, with obvious gaps between activated pixels. Furthermore, the green area has a small coverage radius, rapidly transitioning to a large yellow background area, indicating a relatively limited focal range for effective features.

[0149] Mechanism Analysis: This phenomenon reveals the limitations of local inductive bias in standard CNN architectures. The grid-like morphology reflects the spatial discontinuities caused by multiple strided convolutions and downsampling operations. This means the model can only capture local, discontinuous pixel features, lacking the ability to model long-range dependencies on the overall structure of the target.

[0150] (2) ASF-YOLO model: Specifically as follows Figure 3 As shown:

[0151] Visual observation: After the introduction of ASF and ScalSeq, the thermal distribution of the effective receptive field ERF underwent a qualitative change.

[0152] Significant region expansion: The area of ​​the central green high-response zone has expanded significantly, and its shape has changed from a discrete grid to a continuous, full, clump-like distribution.

[0153] Smooth semantic transition: The boundary between the green area and the yellow background area becomes smoother and extends outward, indicating that the model pays attention to a wider range of surrounding information.

[0154] Mechanism Analysis: The significant expansion of ERF demonstrates that the ASF module successfully establishes global contextual relationships between features. Through ScalSeq's cross-scale pooling and channel attention mechanisms, the model overcomes the geometric limitations of local convolutional kernels, achieving effective aggregation of non-local semantic information. This continuous and extensive receptive field distribution indicates that the model can fully utilize the environmental contextual information surrounding the target, thereby effectively enhancing the semantic consistency of deep features.

[0155] (3) ASF-YOLO-4H model: such as Figure 4 As shown:

[0156] Visual observation: The preferred embodiment of the present invention shows the receptive field distribution in the P2 fine-grained sensing branch. The figure exhibits extreme spatial convergence characteristics: the green high-response area shrinks into an extremely small, high-intensity needle-like focal point, which is directly surrounded by a uniform yellow background area, without any extra diffusion or clutter.

[0157] Mechanism analysis: This morphology directly verifies that the P2 branch perfectly preserves high-frequency spatial information.

[0158] Pixel-level localization capability: Thanks to a shallow downsampling rate (4x), ERF does not exhibit the diffusion phenomenon common in deep networks, but instead maintains extremely high spatial resolution. This means that the network highly concentrates its attention weights within extremely small pixel neighborhoods.

[0159] Salience of small targets: This "laser-like" focusing mechanism proves that the model has the ability to accurately lock onto the geometric texture of small targets, effectively solving the problem of small target signals being submerged in deep features, and providing a solid foundation for high-precision localization and regression.

[0160] In summary, the three sets of ERF visualization results form a complete chain of evidence in terms of physical mechanism: the present invention achieves "breadth expansion of semantic perception" through the ASF module and "precision focusing of spatial details" through the P2 detection head. The two work together to fundamentally solve the scale and background problems in remote sensing image detection.

[0161] Category Activation Mapping (CAM) Heatmap Analysis

[0162] To further demonstrate the attention mechanism and feature focusing capability of the model in real-world detection scenarios, this invention provides a comparison of category activation heatmaps based on Grad-CAM technology, as shown below. Figures 5 to 8 As shown in the figure, the red areas represent the regions where the model shows the highest attention and strongest feature activation during object detection.

[0163] in Figure 5 The original image and the prediction result are shown. Figure 6 The baseline model heatmap, Figure 7 The heatmap for the ASF-YOLO model. Figure 8 The image shows a heatmap of the ASF-YOLO-4H model.

[0164] (1) Baseline model analysis: background interference and attention diffusion

[0165] observe Figure 6 It is evident that the heatmap of the YOLOv8 model exhibits a clear "attention diffusion" phenomenon.

[0166] Background misfocusing: In complex background areas (such as runway lines and building edges), the model exhibits unnecessary feature activation, with red patches scattered in non-target areas, indicating that the model has difficulty distinguishing foreground targets from background texture noise.

[0167] Weak target activation: When targeting a target object, the activation centers of the baseline model are often offset or incompletely covered, resulting in insufficient feature response strength. This is due to a lack of global contextual information, preventing the model from forming a complete understanding of the overall structure of the target.

[0168] (2) Validation of ASF module performance: semantic integrity and noise suppression

[0169] Depend on Figure 7 It is evident that after introducing ASF dual-path attention and the ScalSeq module, the thermal distribution exhibits a significant "semantic aggregation" characteristic.

[0170] Noise suppression: Compared with the baseline model, the cluttered red spots in the background region are significantly reduced, proving that the ASF module effectively suppresses the interference of background texture by introducing a global context.

[0171] Semantic integrity: For medium to large-scale targets, the heatmap can more completely cover the target body (the mask coverage is more uniform), proving that the model has established stronger feature consistency. However, for extremely small targets, the activation region still appears slightly broad, and there is still room for improvement in localization accuracy.

[0172] (3) Verification of the effect of P2 perception layer: precise positioning and fine-grained focusing

[0173] Figure 8 The heatmap of a preferred embodiment of the present invention is shown, which exhibits excellent "fine-grained focusing" capability and is the best of the three.

[0174] Tiny Target Locking: For tiny targets in an image, such as densely packed vehicles or ships, the heatmap displays a matrix of high-intensity activations. Each red hotspot falls precisely at the geometric center of the target and is independent of the others. This directly demonstrates that the P2 fine-grained perception layer successfully preserves high-frequency spatial information, enabling the model to achieve pixel-level localization capabilities.

[0175] Extremely high signal-to-noise ratio: the target area is reddish and bright, while the background area is almost completely cooled and appears deep blue. This extremely high characteristic signal-to-noise ratio indicates that the architecture of this invention can find tiny targets without introducing additional background noise, perfectly achieving the unity of "clear visibility" and "accurate judgment".

[0176] In summary, the full-scale semantically enhanced high-resolution small target detection method and system provided by this invention can more easily discover the potential weaknesses of the model under test, thereby more accurately reflecting the performance of the model under test in the real environment. It has high concealment and high flexibility, and can better meet the security assessment needs in practical applications.

[0177] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A high-resolution small target detection method based on full-scale semantic enhancement, characterized by: Includes the following steps: S1: After preprocessing the remote sensing images in the Dior remote sensing image dataset, input them into the backbone network. The backbone network performs convolution downsampling layer by layer to extract four sets of backbone feature maps at different scales in layers P2, P3, P4 and P5 respectively. S2: Based on four sets of backbone feature maps of different scales, a lightweight four-level linkage feature pyramid is constructed, including a top-down semantic injection path and a bottom-up position enhancement path. In the top-down semantic injection path, a non-linear fusion mechanism is used to generate the backbone feature map of the P2 layer detection branch. In the bottom-up position enhancement path, the backbone feature map of the P2 layer detection branch is used as the starting point for downsampling at each level, and multi-path aggregation is performed with the backbone feature maps of the P3, P4 and P5 layers to generate path-aggregated and enhanced P3, P4 and P5 layer feature maps. The method for generating the backbone feature map of the P2 layer detection branch is as follows: S211: Upsample the P5 layer backbone feature map output by the backbone network, and use the first Zoom_cat module to perform nonlinear fusion with the P4 layer backbone feature map output by the backbone network to generate an intermediate feature map of the P4 layer carrying the semantics of the P5 layer. S212: The intermediate feature map of layer P4 carrying the semantics of layer P5 is further upsampled, and nonlinear fusion is performed with the backbone feature map of layer P3 output by the second Zoom_cat module to generate the intermediate feature map of layer P3. S213: Upsample the intermediate feature map of P3 layer and use the third Zoom_cat module to perform nonlinear fusion with the backbone feature map of P2 layer output by the backbone network to generate the backbone feature map of the detection branch of P2 layer. S3: The scale sequence feature extractor unifies the channels of four sets of backbone feature maps at different scales and aligns them to the resolution of the backbone feature map at layer P2. It then constructs a five-dimensional scale sequence tensor stacked along the scale dimension. The five-dimensional scale sequence tensor is subjected to 3D max pooling operation. While retaining the spatial dimension, the maximum response value is selected along the scale dimension. Then, an internal dimensionality reduction and compression operation is performed to obtain a global context feature map containing the optimal information at all scales. S4: The global context feature map containing optimal information across the entire scale is fused with the backbone feature map of the P2 layer detection branch through dual-path fusion to generate an enhanced P2 layer feature map. S5: The enhanced P2 layer feature map, along with the path-aggregated enhanced P3, P4, and P5 layer feature maps, are fed into the decoupled detection head to predict the class probability and bounding box coordinates of targets at different scales, and output the final small target detection results.

2. The high-resolution small target detection method based on full-scale semantic enhancement according to claim 1, characterized in that: In step S1, adaptive image scaling and normalization methods are used to preprocess the remote sensing images in the Dior remote sensing image dataset.

3. The high-resolution small target detection method based on full-scale semantic enhancement according to claim 1, characterized in that: The method for performing convolutional downsampling layer by layer in step S1 of the backbone network is as follows: The backbone feature map of layer P2 was downsampled by 4 times, with a resolution of [resolution value missing]. ; The backbone feature map of layer P3 was downsampled by 8 times, with a resolution of [missing value]. ; The backbone feature map of layer P4 was downsampled by 16 times, with a resolution of [resolution value missing]. ; The backbone feature map of layer P5 was downsampled by 32 times, with a resolution of [missing information]. .

4. The high-resolution small target detection method based on full-scale semantic enhancement according to claim 1, characterized in that: The method for generating the path aggregation enhanced P3, P4, and P5 layer feature maps in step S2 is as follows: S221: Starting with the feature map of the main branch of the detection branch in the P2 layer, perform a convolutional downsampling operation with a stride of 2 to obtain a P3 layer downsampled feature map with the same resolution as the P3 layer backbone feature map extracted in step S1. Then, the P3 layer downsampled feature map and the P3 layer intermediate feature map are concatenated in the channel dimension and aggregated through the C2f module to generate a path-aggregated enhanced P3 layer feature map. S222: Take the path-aggregated enhanced P3 layer feature map as input, perform a convolution downsampling operation with a stride of 2 to obtain a P4 layer downsampled feature map with the same resolution as the P4 layer backbone feature map extracted in step S1. Then, concatenate the P4 layer downsampled feature map with the P4 layer intermediate feature map in the channel dimension, and aggregate them through the C2f module to generate a path-aggregated enhanced P4 layer feature map. S223: Using the path-aggregated enhanced P4 layer feature map as input, perform a convolutional downsampling operation with a stride of 2 to obtain a P5 layer downsampled feature map with the same resolution as the P5 layer backbone feature map extracted in step S1. Then, concatenate the P5 layer downsampled feature map with the P5 layer intermediate feature map in the channel dimension, and aggregate them through the C2f module to generate a path-aggregated enhanced P5 layer feature map.

5. The high-resolution small target detection method based on full-scale semantic enhancement according to claim 1, characterized in that: The method for generating the global context feature map containing optimal information across all scales in step S3 is as follows: S311: The scale sequence feature extractor receives four sets of backbone feature maps at different scales from layers P2, P3, P4, and P5 of the backbone network. It unifies the number of channels in the four sets of backbone feature maps at different scales and upsamples the three sets of backbone feature maps at different scales in layers P3, P4, and P5 to the same resolution size as the backbone feature map in layer P2, thus obtaining high-resolution aligned feature maps in layers P3, P4, and P5. S312: Stack the backbone feature map of layer P2, the high-resolution aligned feature map of layer P3, the high-resolution aligned feature map of layer P4, and the high-resolution aligned feature map of layer P5 in the scale dimension to construct a 5-dimensional scale sequence tensor of shape [B,C,4,H,W], where: B represents the batch size, C represents the number of channels, 4 represents the scale dimension, H represents the feature map height, and W represents the feature map width; S313: Perform 3D max pooling on the 5-dimensional scale sequence tensor to filter out the maximum response value across four scales at each spatial location, and obtain a feature map with the largest response value. S314: The feature map with the largest response value is subjected to dimensionality reduction to obtain a global context feature map containing optimal information across the entire scale.

6. The high-resolution small target detection method based on full-scale semantic enhancement according to claim 5, characterized in that: In step S314, global context features containing optimal information across the entire scale are extracted according to equation (1), resulting in a global context feature map containing optimal information across the entire scale: （1）； in: This represents a global context feature that contains optimal information across the entire scale. This indicates a dimensionality reduction operation. Represents a high-dimensional scale sequence tensor. This represents the three-dimensional max pooling operation.

7. The high-resolution small target detection method based on full-scale semantic enhancement according to claim 1, characterized in that: The method for generating the enhanced P2 layer feature map in S4 is as follows: S411: The backbone feature map of the detection branch of the P2 layer is first passed through adaptive one-dimensional convolution to capture cross-channel interactions without dimensionality reduction, generate channel weights, and suppress background noise channels to obtain the refined feature map of the P2 layer. S412: Add the global context feature map containing the optimal information across the entire scale to the refined P2 layer feature map element by element to form the fused feature map; S413: The fused feature map is aggregated along the horizontal and vertical directions respectively through the coordinate space attention mechanism to generate an accurate spatial position weight map, which is used to spatially relocate and focus on small targets, and generate an enhanced P2 layer feature map.

8. The high-resolution small target detection method based on full-scale semantic enhancement according to claim 7, characterized in that: In step S411, channel weights are generated according to equation (2), and the refined P2 layer feature map is obtained according to equation (3): （2）； （3）； in: Indicates channel weight, This represents the Sigmoid activation function. This represents a one-dimensional convolution operation. Indicates global average pooling. Indicates the input features of the main path. This represents the refined features of layer P2. This indicates the adaptive one-dimensional convolution kernel size. This indicates element-wise multiplication.

9. A high-resolution small target detection system based on full-scale semantic enhancement, used to execute the high-resolution small target detection method based on full-scale semantic enhancement as described in any one of claims 1 to 8, characterized in that, It includes an image preprocessing unit, a backbone network, a lightweight four-level linkage feature pyramid construction unit, a scale sequence feature extractor, a dual-path attention fusion unit, and a decoupled detection head; The image preprocessing unit is used to preprocess remote sensing images in the Dior remote sensing image dataset. The backbone network is used to perform convolutional downsampling on the preprocessed image layer by layer, and extract four sets of backbone feature maps of different scales in layers P2, P3, P4 and P5 respectively. The lightweight four-level linkage feature pyramid construction unit is used to construct a lightweight four-level linkage feature pyramid, generating the main feature map of the P2 layer detection branch and the path aggregation enhanced feature maps of the P3, P4, and P5 layers. The scale sequence feature extractor is used to extract features from four sets of backbone feature maps of different scales from layers P2, P3, P4 and P5 of the backbone network, and generate a global context feature map containing optimal information across all scales. The dual-path attention fusion unit is used to fuse the global context feature map containing full-scale optimal information with the backbone feature map of the P2 layer detection branch to generate an enhanced P2 layer feature map. The decoupled detection head predicts the class probability and bounding box coordinates of targets at different scales based on the enhanced P2 layer feature map and the path-aggregated enhanced P3, P4, and P5 layer feature maps, and outputs the final small target detection result.

Images