A micro target detection method based on implicit transformer

Abstract

The application discloses a kind of micro target detection methods based on implicit Transformer, belong to computer vision and target detection technical field, comprising: S1, input image is extracted high-level semantic feature by backbone network, obtains single scale feature representation and low resolution feature map;S2, single scale feature is enhanced semantic expression by global modeling of encoder;S3, initial query set containing feature representation and position information is generated by initializing query through center point guide dense query initialization module;S4, decoder realizes low resolution feature map continuous position query using implicit deformable attention module;S5, decoder directly outputs target class and position by multilayer optimization, completes end-to-end detection.The micro target detection method based on implicit Transformer provided in the application considers detection precision and efficiency, and is suitable for remote sensing, automatic driving and other scenes.

Description

Technical Field

[0001] This invention relates to the field of computer vision and target detection technology, and in particular to a method for detecting tiny targets based on implicit Transformer. Background Technology

[0002] In recent years, the end-to-end detection framework DETR, which incorporates global modeling capabilities, has gradually attracted attention. To compensate for its shortcomings in modeling local details and scale variations, multi-scale feature design has been introduced. However, the aforementioned multi-scale design has also revealed significant limitations in the scenario of small target detection. On the one hand, the extraction, fusion, and attention computation of multi-scale features significantly increase the computational complexity and memory consumption of the model; on the other hand, as the resolution of the input image increases, the computational overhead brought by high-resolution feature maps shows a rapid growth trend. This poses a severe challenge to computational resources and real-time performance under the ultra-high resolution input conditions commonly seen in small target detection. To alleviate the computational burden, some existing technologies attempt to reduce the number of regions involved in computation by cropping, sparsifying, or selectively processing high-resolution images or features. However, such methods usually require additional result fusion or post-processing steps, making it difficult to maintain the end-to-end detection process; at the same time, sparsification of features or direct modeling of high-resolution features can easily lead to the loss of key detail information, thereby reducing the detection accuracy of small targets. In addition, some studies have attempted to weaken or even eliminate multi-scale or local feature design, and instead rely on single-scale features or global modeling mechanisms to complete target detection. However, these methods have inherent limitations in receptive field adjustment, spatial sampling accuracy, and characterization of fine-grained features of small targets, especially in high-resolution small target detection tasks, where detection performance is often significantly limited.

[0003] In summary, existing technologies have made some progress in improving the performance of small target detection, but they still generally face problems such as high computational cost, strong dependence on high-resolution features, insufficient information utilization efficiency, and difficulty in balancing accuracy and efficiency in end-to-end detection. There is an urgent need for a technical solution that can effectively reduce computational complexity while ensuring detection accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a small target detection method based on implicit Transformer, which solves the problems of high computational complexity, strong dependence on high-resolution features, insufficient information utilization efficiency, and difficulty in balancing accuracy and efficiency in end-to-end detection in existing technologies for small target detection.

[0005] To achieve the above objectives, this invention provides a method for detecting small targets based on implicit Transformer, comprising the following steps: S1. The input image is processed by a backbone network to extract high-level semantic features, resulting in a single-scale feature representation. And use the low-resolution feature map output from the last layer of the backbone network. As a detection feature; S2. Representing single-scale features The input encoder module enhances the semantic expressive power of single-scale features through global modeling; S3, Low-resolution feature map output by the backbone network The query is initialized by the CGQI (Central Point Guided Query Initialization) module, which generates a reasonably distributed initial query set. The initial query set includes the feature representation of the initial query. and the initial query location information ; S4. An implicit deformable attention module iAttn is used inside the decoder to replace the original deformable attention mechanism of DETR, so as to realize feature query at any continuous position of the low-resolution feature map. S5, the decoder progressively optimizes the query through multi-layered stacked self-attention, implicit deformable attention, and feedforward networks, outputting the target's category and location information to complete end-to-end small target detection.

[0006] Preferably, the working process of the center point-guided dense query initialization module CGQI in step S3 specifically includes the following steps: S31. Low-resolution feature maps Feature expansion processing is performed, which transforms the original features into a dense set of candidate features through nonlinear mapping; S32. A scoring mechanism is used to filter the candidate features in the candidate feature set, and the highest-scoring candidate features are selected for query initialization. S33. An anchorless query generation strategy is adopted to decouple the modeling process of query location into two stages: center point prediction and target size regression. Before the target size regression, an implicit deformable attention module is introduced to refine the features corresponding to the center point.

[0007] Preferably, the center point prediction stage in step S33 specifically includes the following steps: S331. Based on the filtered candidate features, the prediction network outputs the coordinate offset values ​​between the spatial location of the candidate features and the center point of the real target. S332. Extract each candidate feature from the low-resolution feature map. The original spatial coordinates on; S333. Combine the original spatial coordinates with the predicted feature offset to obtain the final predicted position of the target center point.

[0008] Preferably, the target size regression stage specifically involves: predicting the target size parameters based on the refined center point features, the target size information corresponding to each center point, and combining this with the final predicted position of the target center point to form complete initial query location information. At the same time, the refined center point features are combined to form the feature representation of the initial query. .

[0009] Preferably, the implementation of the implicit deformable attention module iAttn in step S4 specifically includes the following steps: S41. Introduce a continuous sampling function based on implicit neural representation (INR), and continuously model the feature space through learning to achieve processing of low-resolution feature maps. Feature query at any consecutive positions; S42. A set of local implicit basis functions with the same structure but different parameters are used to model the features. Each local implicit basis function is parameterized by the latent code of the corresponding spatial location of the feature map. S43. The attention head-related dimension mapping process is directly integrated into the parameterization process of the local implicit basis function to achieve integrated modeling of sampling and dimensionality reduction.

[0010] Preferably, a feature recombination mechanism is introduced in the feature generation stage. By combining a 1×1 convolutional layer with an Unpixelshuffle operation, the feature channels and spatial distribution are rearranged so that the generated features retain sufficient spatial information at the target scale.

[0011] Preferably, in the multi-layer stacking process in step S5, each layer sequentially performs self-attention mechanism processing, implicit deformable attention mechanism processing, and feedforward network processing, and the output of the previous layer serves as the input of the next layer to achieve iterative optimization of the query.

[0012] Preferably, the micro-target is a target instance with fewer than 16×16 pixels.

[0013] Therefore, the present invention employs the above-mentioned method for detecting small targets based on implicit Transformer, which has the following beneficial effects: (1) Achieving high-precision small target detection using only single-scale features; This patented solution abandons the reliance on multi-scale feature fusion and can complete the target detection task using only the single-scale features output by the backbone network, effectively reducing the complexity of the model structure and computational cost, while still maintaining high detection accuracy for small targets. (2) Break through the limitations of discrete sampling and significantly improve the positioning accuracy under low-resolution features; By introducing a feature sampling mechanism based on implicit neural representation, the detection model can perform feature query at any continuous position in the low-resolution feature map, avoiding the positioning error caused by traditional interpolation sampling when the spatial resolution is limited, thereby significantly improving the spatial modeling capability and positioning accuracy of small targets. (3) Achieve dense and adaptive query initialization under single-scale conditions; In view of the problem that query initialization in the prior art is highly dependent on multi-scale features and predefined anchor boxes, this patent proposes a center point-guided dense query initialization strategy, which significantly increases the number of candidate queries and spatial coverage without relying on anchor box design, and is especially beneficial to improving the recall rate of small targets.

[0014] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the structure of a small target detection method based on implicit Transformer according to the present invention. Detailed Implementation

[0016] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0017] The implicit Transformer-based small target detection method of this invention (hereinafter referred to as the "iDETR method") follows an end-to-end detection logic, and the core process is as follows: Figure 1 As shown, the process involves "input image → backbone network feature extraction → encoder global modeling → CGQI module query initialization → decoder query optimization → target detection output." The entire process relies solely on single-scale features to detect small targets, eliminating the need for multi-scale feature fusion and additional post-processing steps. Here, a small target is defined as a target instance with fewer than 16×16 pixels.

[0018] I. Experimental Setup (a) Dataset This method utilizes the AI-TODv2 dataset, which contains 28,036 aerial images. In the experiments, these images were proportionally divided into 11,214 training images, 2,804 validation images, and 14,018 test images. The dataset covers eight categories and contains a total of 752,745 annotated object instances. The average object size is only 12.7 pixels, and 86% of the instances are smaller than 16 pixels, making it a highly challenging benchmark dataset in the field of small object detection.

[0019] (ii) Model variants and training configurations This method is an improvement on the latest D-FINE baseline, removing its multi-scale feature fusion design and replacing the original deformable attention and query initialization components with the proposed iAttn and CGQI modules, respectively. Four model variants were tested, as follows: iDETR-L: The basic version for large-scale models; iDETR-DC5-L: Large model scale, using an inflated C5 backbone (a common configuration for single-scale detectors); iDETR-X: The basic version for ultra-large model scale; iDETR-DC5-X: Ultra-large model size, using an inflated C5 backbone.

[0020] All models were trained on eight RTX 5880 GPUs, with a total batch size of 8. The AdamW optimizer was used for 120 epochs, with an initial learning rate of 1e-4, which was decayed by a factor of 0.8 at the 60th and 90th epochs. To ensure fairness in the comparison, the model architecture, loss function, and other hyperparameters were strictly aligned with the corresponding D-FINE-L and D-FINE-X baseline methods.

[0021] (III) Evaluation Indicators Following the standard COCO style protocol, Average Precision (AP) is used as the core evaluation metric. Additionally, based on the AI-TOD benchmark definition, AP metrics for specific dimensions are reported, including: APvt: Very small objects (2 to 8 pixels); APt: Small objects (8 to 16 pixels); APs: Smaller objects (16 to 32 pixels); APm: Medium object (greater than 32 pixels).

[0022] These refined metrics can more comprehensively evaluate the model's performance in the task of detecting extremely small objects, which aligns with the core requirements of TOD (Total Object Detection).

[0023] II. Specific Implementation Details of Each Step (a) Step S1: Backbone network feature extraction The input image is fed into the backbone network, and high-level semantic features are extracted through the feature extraction process of the backbone network. The backbone network outputs only the last layer of features, resulting in two key features: a single-scale feature representation and a low-resolution feature map. This low-resolution feature map is then used as the detection feature for the entire detection process. This process does not involve the extraction of multi-scale features, but only retains single-scale features for subsequent processing, thereby reducing computational complexity from the process perspective.

[0024] (ii) Step S2: Global modeling of the encoder The single-scale feature representation obtained in step S1 is input into the encoder module; The encoder module processes single-scale feature representations through a global modeling mechanism, enhancing the semantic expressive power of features and focusing on improving the distinguishability between semantic information of small targets and background information. After processing, the semantically enhanced global features are output, providing a feature basis for query optimization of the decoder.

[0025] (III) Step S3: CGQI module query initialization Step S3 initializes the query using the CGQI (Central Point Guided Dense Query Initialization) module, generating an initial query set containing the feature representation and location information of the initial query. The specific implementation is as follows: Step S31: Feature expansion processing takes the low-resolution feature map output from step S1 as input and performs feature expansion processing on the original features through nonlinear mapping, transforming the original features into a denser set of candidate features, providing multiple optional candidate representations for each potential spatial location.

[0026] Step S32: Candidate feature selection uses a preset scoring mechanism to select candidate features from the candidate feature set generated in step S31, and selects several candidate features with the highest scores for subsequent query initialization process, resulting in a subset of highly discriminative candidate features.

[0027] S33. An anchorless query generation strategy is adopted to decouple the modeling process of query location into two stages: center point prediction and target size regression. Before entering the target size regression stage, an implicit deformable attention module iAttn is introduced to refine the features corresponding to the center point, thereby improving the discriminativeness of the features at the center point.

[0028] Center point prediction: Center point prediction is based on the candidate features selected in step S32. The prediction network outputs the coordinate offset values ​​between the spatial position of the candidate feature and the real target center point; the original spatial coordinates of each candidate feature on the low-resolution feature map are extracted; the original spatial coordinates are combined with the predicted feature offset to obtain the final predicted position of the target center point.

[0029] Target size regression: Target size regression predicts the size parameters of the target based on the refined center point features; the target size information corresponding to each center point is combined with the final predicted position of the target center point to form complete initial query position information; at the same time, the refined center point features are used as the feature representation of the initial query, and the two together constitute the initial query set.

[0030] (iv) Step S4: iAttn module feature query Inside the decoder, an implicit deformable attention module iAttn is used to replace the original deformable attention mechanism of DETR; This module achieves feature querying at any continuous position in a low-resolution feature map through a specific design, breaking through the limitation of traditional deformable attention mechanisms that rely on discrete sampling and improving the spatial accuracy of feature querying; The specific implementation method is as follows: A continuous sampling function based on implicit neural representation (INR) is introduced to continuously model the feature space through learning, thereby enabling feature querying at any continuous position in the low-resolution feature map. The features are modeled using a set of local implicit basis functions with the same structure but different parameters. Each local implicit basis function is parameterized by the latent code of the corresponding spatial location of the feature map. By directly integrating the attention-related dimension mapping process into the parameterization process of local implicit basis functions, we can achieve integrated modeling of sampling and dimensionality reduction, thereby reducing redundant computational overhead.

[0031] (V) Step S5: Decoder query optimization and target output The decoder receives the global features output by the encoder and the initial query set generated by the CGQI module; The decoder employs a multi-layered stacked structure, with each layer sequentially performing self-attention processing, implicit deformable attention processing, and feedforward network processing. Self-attention is used to optimize dependencies between queries and avoid confusion between queries with different objectives. Implicit deformable attention processing is based on global features and performs precise sampling and enhancement of the features corresponding to the query. The feedforward network performs a non-linear transformation on the query features after attention processing, further enhancing the feature representation capability; The output of the previous layer serves as the input of the next layer, and the query is gradually optimized through multiple iterations. After optimization, the decoder directly outputs the target's category and location information without any additional post-processing steps, completing end-to-end detection of small targets.

[0032] III. Experimental Results and Analysis (I) Main Experimental Results Table 1 shows the performance comparison between the iDETR series models and the latest D-FINE baseline on the AI-TODv2 dataset. The results show that the iDETR model has achieved significant improvements in both performance and efficiency.

[0033] Table 1 Experimental Results

[0034] The specific analysis is as follows: 1. Compared with the single-scale D-FINE baseline, the overall average accuracy (AP) of iDETR-L and iDETR-X is improved by 6.5 and 6.6 percentage points, respectively, which fully demonstrates the effectiveness of the improvement strategy of this method; 2. Compared with multi-scale D-FINE (which integrates S2, S3, S4, and S5 features): iDETR-L's performance is comparable to D-FINE-L, but its computational cost is only one-third of it; iDETR-DC5-L has an AP that is 1.1 percentage points higher with only half the number of floating-point operations (FLOPs) of D-FINE-L; iDETR-X approaches D-FINE-X's performance with less than one-third of its computational cost, and iDETR-DC5-X has an AP that is 0.3 percentage points higher with only half the resource consumption of D-FINE-X; 3. Performance Advantages in Small Object Detection: The iDETR series models demonstrate outstanding performance in key small object detection metrics. Compared to D-FINE-L, iDETR-DC5-L improves APvt (for very small objects) by 2.9 percentage points and APt (for small objects) by 1.7 percentage points; compared to D-FINE-X, iDETR-DC5-X improves APvt and APt by 2.5 and 1.4 percentage points respectively, highlighting the core advantages of this method in small object detection scenarios.

[0035] (II) Ablation Experiment Results To verify the independent roles of the iAttn and CGQI modules, ablation experiments were conducted on the AI-TODv2 dataset with single-scale D-FINE-L as the baseline. The results are shown in Table 2. Table 2 Ablation Experiment Results

[0036] Experimental results show that: 1. By simply introducing the iAttn module to replace the deformable attention mechanism, the AP was improved by 4.3 percentage points, which verifies the important role of the continuous sampling mechanism in improving detection accuracy; 2. By simply adding the CGQI module to optimize query initialization, the AP improved by 1.4 percentage points, indicating that a dense and adaptive query initialization strategy can effectively improve the coverage and quality of candidate queries. 3. By combining the iAttn and CGQI modules (the complete iDETR model), the AP was improved by a total of 6.5 percentage points, proving that the two modules are complementary in function and their synergistic effect can maximize the detection performance.

[0037] Therefore, this invention adopts the aforementioned implicit Transformer-based micro-target detection method. Through an architecture design that relies solely on single-scale features, combined with the innovative application of a center-point guided dense query initialization module and an implicit deformable attention module, it not only solves the problems of high computational complexity and large memory consumption in existing multi-scale feature schemes, but also makes up for the shortcomings of traditional single-scale feature detection in terms of spatial sampling accuracy and query coverage. At the same time, it eliminates additional post-processing steps, maintaining the simplicity of the end-to-end detection process. Ultimately, while ensuring the accuracy of micro-target detection, it significantly reduces computational overhead, adapts to the real-time requirements of ultra-high resolution input scenarios, and provides an efficient and reliable technical solution for micro-target detection tasks in fields such as remote sensing, autonomous driving, and pedestrian detection.

[0038] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting small targets based on implicit Transformer, characterized in that, Includes the following steps: S1. The input image is processed by a backbone network to extract high-level semantic features, resulting in a single-scale feature representation. And use the low-resolution feature map output from the last layer of the backbone network. As a detection feature; S2. Representing single-scale features The input encoder module enhances the semantic expressive power of single-scale features through global modeling; S3, Low-resolution feature map output by the backbone network The query is initialized by the CGQI (Central Point Guided Query Initialization) module, which generates a reasonably distributed initial query set. The initial query set includes the feature representation of the initial query. and the initial query location information ; S4. An implicit deformable attention module iAttn is used inside the decoder to replace the original deformable attention mechanism of DETR, so as to realize feature query for any continuous position of low-resolution feature map. S5, the decoder progressively optimizes the query through multi-layered stacked self-attention, implicit deformable attention, and feedforward networks, outputting the target's category and location information to complete end-to-end small target detection.

2. The method for detecting small targets based on implicit Transformer according to claim 1, characterized in that, The working process of the center point-guided dense query initialization module CGQI in step S3 specifically includes the following steps: S31. Low-resolution feature maps Feature expansion processing is performed, which transforms the original features into a dense set of candidate features through nonlinear mapping; S32. A scoring mechanism is used to filter the candidate features in the candidate feature set, and the candidate feature with the highest score is selected for query initialization. S33. An anchorless query generation strategy is adopted to decouple the modeling process of query location into two stages: center point prediction and target size regression. Before the target size regression, an implicit deformable attention module is introduced to refine the features corresponding to the center point.

3. The method for detecting small targets based on implicit Transformer according to claim 2, characterized in that, The center point prediction stage in step S33 specifically includes the following steps: S331. Based on the filtered candidate features, the prediction network outputs the coordinate offset values ​​between the spatial location of the candidate features and the center point of the real target. S332. Extract each candidate feature from the low-resolution feature map. The original spatial coordinates on; S333. Combine the original spatial coordinates with the predicted feature offset to obtain the final predicted position of the target center point.

4. The method for detecting small targets based on implicit Transformer according to claim 3, characterized in that, The target size regression stage specifically involves: based on the refined center point features, predicting the target size parameters, the target size information corresponding to each center point, and combining this with the final predicted position of the target center point to form complete initial query location information. At the same time, the refined center point features are combined to form the feature representation of the initial query. .

5. The method for detecting small targets based on implicit Transformer according to claim 1, characterized in that, The implementation of the implicit deformable attention module iAttn in step S4 specifically includes the following steps: S41. Introduce a continuous sampling function based on implicit neural representation (INR), and continuously model the feature space through learning to achieve processing of low-resolution feature maps. Feature query at any consecutive positions; S42. A set of local implicit basis functions with the same structure but different parameters are used to model the features. Each local implicit basis function is parameterized by the latent code of the corresponding spatial location of the feature map. S43. The attention head-related dimension mapping process is directly integrated into the parameterization process of the local implicit basis function to achieve integrated modeling of sampling and dimensionality reduction.

6. The method for detecting small targets based on implicit Transformer according to claim 5, characterized in that, A feature recombination mechanism is introduced in the feature generation stage. By combining a 1×1 convolutional layer with an Unpixelshuffle operation, the feature channels and spatial distribution are rearranged so that the generated features retain sufficient spatial information at the target scale.

7. The method for detecting small targets based on implicit Transformer according to claim 1, characterized in that, In the multi-layer stacking process in step S5, each layer sequentially performs self-attention mechanism processing, implicit deformable attention mechanism processing, and feedforward network processing, with the output of the previous layer serving as the input of the next layer, thereby achieving iterative optimization of the query.

8. The method for detecting small targets based on implicit Transformer according to claim 1, characterized in that, Tiny targets are target instances with fewer than 16×16 pixels.

Images