Meta-DETR few-sample target detection method based on region guidance

Abstract

The invention discloses a Meta-DETR few-sample target detection method based on region guidance, and belongs to the technical field of computer vision. Comprising the following three modules: 1) a learnable attention fusion module LAF; 2) a double-flow prototype area memory module DPRM; 3) a region guidance self-attention module RGSA; compared with the existing method for fusing the attention weight by adopting a fixed rule (such as Attention Rollout), the LAF realizes dynamic refining and fusion of the attention weight through learnable head / point weight correction, a self-adaptive normalization strategy and bilinear bottleneck enhancement, and the generated space thermodynamic diagram can more accurately highlight a target area, suppress background clutters and improve the accuracy of the space thermodynamic diagram. The problem that a fixed rule is difficult to adapt to complex feature distribution is avoided, and higher-quality positioning prior is provided for subsequent regional prototype extraction. The method can effectively solve the core pain points of distraction and weak feature generalization ability of an existing few-sample detection method, and finally realizes double improvement of few-sample target detection precision and generalization ability.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a region-guided Meta-DETR method for target detection with few samples. It is suitable for general target detection tasks in scenarios with scarce samples and can be widely applied in fields such as security monitoring, industrial quality inspection, agricultural monitoring, and remote sensing exploration that require rapid adaptation to new target categories. Background Technology

[0002] Computer vision technology, by simulating the perceptual capabilities of the human visual system, enables the localization and recognition of targets in images, and has become one of the core application areas in the field of artificial intelligence. Object detection, as a key task of computer vision, aims to accurately extract the location information and category attributes of targets from complex images, providing core support for various intelligent applications.

[0003] With the rapid development of deep learning technology, object detection algorithms have evolved from traditional manual feature methods to end-to-end learning methods based on deep neural networks. Deep learning-based object detection algorithms can be divided into two categories: one-stage and two-stage. One-stage algorithms, such as the YOLO series, directly classify and regress bounding boxes from feature maps, offering fast detection speeds but limited accuracy. Two-stage algorithms, such as Faster R-CNN, generate proposal boxes and then fine-tune them, achieving higher accuracy but with higher computational costs.

[0004] Few-shot object detection, as an important branch of object detection, focuses on solving the problem of scarce labeled data. It enables models to quickly adapt to new object categories using a small number of labeled samples (usually 1-5 images per class). Existing few-shot detection methods mainly include fine-tuning-based methods (such as TFA) and meta-learning-based methods (such as Meta-DETR). Among them, Meta-DETR adopts a meta-learning paradigm of "support set-query set," abandons the traditional anchor box design, achieves end-to-end detection, and effectively utilizes inter-class correlations to improve few-shot detection performance, making it the current advanced few-shot detection model.

[0005] However, Meta-DETR still has significant limitations in practical applications: First, its self-attention mechanism is global and data-driven, making it difficult to automatically focus on the discriminative local regions of the target in scenarios with extremely scarce samples, and it is easily affected by complex backgrounds; Second, new categories have only a small number of samples, resulting in sparse and discriminative feature representations learned by the model, especially for categories with similar appearances, where the generalization ability is weak; Third, existing attention fusion methods mostly use fixed rules and cannot adaptively adjust according to the distribution of image features, making it difficult to generate high-quality region localization priors, which further limits the detection performance.

[0006] These problems are particularly prominent in complex real-world scenarios, severely hindering the practical application of few-shot object detection techniques. Therefore, there is an urgent need for a few-shot object detection method that can effectively guide attention focus, enrich feature representations, and improve model generalization capabilities. Summary of the Invention

[0007] This invention belongs to the field of computer vision and few-shot target detection technology, specifically involving a Transformer target detection method based on region guidance and memory enhancement, which aims to improve the target detection accuracy and generalization ability in few-shot scenarios.

[0008] To achieve the above objectives, this invention provides a target detection method that integrates learnable attention fusion, dual-stream prototype region memory, and region-guided self-attention mechanism, comprising the following three modules:

[0009] 1) Learnable Attention Fusion Module (LAF);

[0010] The learnable attention fusion module takes the attention weights output from each layer of the encoder as input and reweights the attention map using learnable head weights and point weights. A lightweight decision network is introduced to dynamically select a normalization strategy to achieve adaptive separation of foreground and background. Through inter-layer fusion, inter-head competition, and a bilinear bottleneck structure, a high-contrast spatial heatmap is generated to guide target region localization.

[0011] 2) Dual-stream prototype region memory module (DPRM);

[0012] In each episode, the dual-stream prototype region memory module extracts class-specific region information from the heatmap based on the current activation category, generating the region center, radius, and features for each class. A dual-stream progressive class prior calibration strategy is proposed, initially using bounding boxes to quickly locate target regions, and later gradually transitioning to self-supervised region modeling based on learnable prototypes. High-confidence region prototypes are generated through saliency sampling and temperature-softened K-means clustering, and weighted reduction and refinement are performed at the episode level to form stable class-specific region prior knowledge.

[0013] 3) Region-guided self-attention module (RGSA);

[0014] The region-guided self-attention module introduces a region memory enhancement mechanism during the decoding stage. Based on the spatial distance and semantic similarity between the reference point of the query and the region prototype, it calculates the position gating and category gating weights and adaptively fuses external region features.

[0015] Compared with existing technologies, the advantages of this invention are as follows: It constructs an attention refinement mechanism with a learnable attention fusion module (LAF). Compared to existing methods that use fixed rules (such as AttentionRollout) to fuse attention weights, LAF achieves dynamic refinement and fusion of attention weights through learnable head / point weight correction, adaptive normalization strategies, and bilinear bottleneck enhancement. The generated spatial heatmap can more accurately highlight the target region and suppress background clutter, avoiding the problem that fixed rules are difficult to adapt to complex feature distributions, and providing higher-quality localization priors for subsequent region prototype extraction. Furthermore, it designs a dual-stream prototype region memory module (DPRM). Compared to existing few-shot methods that only store global image features, DPRM can mine and memorize category-specific region prototypes (including geometric center, radius of action, and depth features) online for each training task (episode), condensing scarce sample information into "region-level" semantic priors. This effectively alleviates the problem of sparse feature representation in few-shot scenarios, enhances the model's perception of target regions, and especially better captures discriminative local cues for appearance-similar categories.

[0016] Simultaneously, a Region-Guided Self-Attention (RGSA) module is introduced. Compared to the original Meta-DETR's global self-attention mechanism, RGSA uses position gating (for spatial alignment), category gating (for semantic alignment), and adaptive gating to precisely inject the region knowledge stored in the DPRM into the decoder's query vector. This guides attention to focus on key regions that are highly coupled with the target category's spatial and semantic aspects, avoiding the problem of global self-attention drifting to the background in few-shot scenarios and improving the accuracy of target localization. Furthermore, the three modules work collaboratively under strict task isolation, retaining the advantages of Transformer's global modeling while effectively addressing the core pain points of existing few-shot detection methods—namely, scattered attention and weak feature generalization ability—through the implantation of local discriminative anchors. Ultimately, this achieves a dual improvement in the accuracy and generalization ability of few-shot target detection. Attached Figure Description

[0017] Figure 1 This is a structural diagram of the Meta-DETR target detection model based on region guidance.

[0018] Figure 2 This is a structural diagram of the Learnable Attention Fusion (LAF) module of the present invention.

[0019] Figure 3 This is a structural diagram of the Dual Stream Prototype Region Memory Module (DPRM) of the present invention.

[0020] Figure 4 This is a structural diagram of the Region Guided Self-Attention Module (RGSA) of the present invention.

[0021] Figure 5This is a schematic diagram comparing the experimental results of the present invention with those of three other methods.

[0022] Figure 6 This is a schematic diagram comparing the experimental results of the present invention with those of three other methods. Detailed Implementation

[0023] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] This invention provides a target detection method that integrates learnable attention fusion, dual-stream prototype region memory, and region-guided self-attention mechanisms, such as... Figure 1 It mainly includes the following three modules:

[0025] 1) Learnable Attention Fusion (LAF) module;

[0026] This module takes the attention weights output from each layer of the encoder as input and reweights the attention map using learnable head weights and point weights to solve the gradient ambiguity problem caused by the "egalitarianism" in the initial stage of attention distribution. A lightweight decision network is introduced to dynamically select a normalization strategy (sigmoid or softmax) to achieve adaptive separation of foreground and background. Furthermore, through inter-layer fusion, inter-head competition, and a bilinear bottleneck structure, a high-contrast spatial heatmap is generated to guide target region localization, such as... Figure 2 .

[0027] 2) Dual-Stream Prototype-Evolving Regional Memory (DPRM).

[0028] In each episode, class-specific region information is extracted from the heatmap based on the current activation category, generating the region center, radius, and features for each class. A "dual-stream progressive class prior calibration" strategy is proposed, initially using bounding boxes to quickly locate target regions, and later gradually transitioning to self-supervised region modeling based on learnable prototypes. High-confidence region prototypes are generated through saliency sampling and temperature-softened K-means clustering, and weighted reduction and refinement are performed at the episode level to form stable class-specific region prior knowledge, such as... Figure 3 .

[0029] 3) Region-Guided Self-Attention (RGSA) module;

[0030] This module introduces a region memory enhancement mechanism during the decoding stage. Based on the spatial distance and semantic similarity between the query's reference point and the region prototype, it calculates positional and category gating weights and adaptively fuses external region features. By controlling the strength of memory introduction through an adaptive gating mechanism, it achieves a flexible enhancement strategy of "using when needed and discarding when not needed," significantly improving the query's expressive power and detection accuracy. Figure 4 .

[0031] The implementation of a few-shot remote sensing target detection method based on region-guided Meta-DETR can be divided into two parts: constructing a region-guided Meta-DETR network model and training the region-guided Meta-DETR network model. In the model construction stage, a learnable attention fusion module (LAF), a two-stream prototype region memory module (DPRM), and a region-guided self-attention module (RGSA) are inserted into the Meta-DETR to achieve parameter-free coupling between region priors and the Transformer detection head.

[0032] For ease of understanding, the specific steps of the embodiments of the present invention will be described below with reference to the accompanying drawings.

[0033] Step 1: Construct a region-guided Meta-DETR network model;

[0034] Step 1. Backbone Network;

[0035] ResNet101 is used as the backbone network for feature extraction in the region-guided Meta-DETR network model, outputting only the layer 4 (C5) feature map with a spatial resolution of 1 / 32 of the input image and 1024 channels. The backbone network shares weights with the support and query images to ensure feature space consistency, facilitating subsequent prototype matching and region guidance.

[0036] Step 2. Feature output and TransformerEncoder configuration;

[0037] The C5 feature map output from the backbone network is directly fed into the TransformerEncoder. The encoder consists of six stacked DeformableDETR encoder layers, each containing an 8-head self-attention mechanism with a hidden dimension of 256. Each layer outputs an attention weight tensor. The shape is (B, Lq, 8, 4), where B represents the batch size, Lq = H × W is the flattened length of the feature map, 8 is the number of attention heads, and 4 is the number of deformable reference points. The A... l As an input source for the LAF module, it can be directly embedded into the existing Meta-DETR framework without modifying the trunk or detection head, achieving structural compatibility.

[0038] Step 3. Define the module insertion point;

[0039] LAF module: Inserted after TransformerEncoder and before TransformerDecoder, it receives all 6 layers of deformable attention weights A. l By learning-based reweighting, adaptive normalization, inter-layer fusion, and bilinear bottleneck enhancement, a single-channel spatial saliency map M∈R^{B×H×W} is generated; M is also used for DPRM sampling and detection head masking, realizing information reuse of "one map for two purposes".

[0040] The DPRM module is instantiated at the start of each episode, without adding any new learnable parameters, and only allocating cache space. Using M as the probability map, it generates 64 region prototype triples {c,r,f} for the current activation category through saliency sampling, temperature-softened K-means, and 90th percentile radius estimation, and dynamically accumulates them as the episode propagates forward to avoid cross-task information leakage.

[0041] The RGSA module is a self-attention layer embedded in the TransformerDecoder, which performs region memory injection before multi-head self-attention computation. Its input and output tensor shapes, data types, and residual connection methods are completely consistent with the standard Multi-HeadAttention, without requiring modification of FFN, LayerNorm, or subsequent cross-attention processes.

[0042] Step 4. Construction of the Learnable Attention Fusion Module (LAF);

[0043] Step 4.1 allows learning reweighted units;

[0044] To address the issue of effective attention being drowned out by background noise due to symmetric initialization with multiple heads and multiple reference points, LAF first introduces learnable scalar weights w_head∈R. 8 with w_point∈R 4 Multiply element-wise with the original attention map:

[0045] (1);

[0046] in, This indicates element-wise multiplication, with weights. Broadcast along the head dimension, Broadcast along the reference point dimension, with both initial values ​​set to 1 to ensure that the weights of each head and each point are approximately equal in the early stages of training; as the loss propagates back, the weights of heads that make high contributions to detection and reference points that fall at the center of the target automatically approach 1, while the weights of the rest approach 0, thereby achieving adaptive filtering of "enhancing high contributions and suppressing low contributions" without the need for manual threshold setting.

[0047] Step 4.2, Adaptive Normalization Unit;

[0048] After reweighting, the amplitude distribution varies greatly. Direct softmax can easily suppress weak response regions permanently in the early stages of training. LAF calculates the point statistics μ∈R32 of the channels along the batch and query dimensions, and outputs the decision factor d∈[0,1] after two fully connected layers:

[0049] (2);

[0050] Where W1: 32→64, W2: 64→1, softmax is used when d>0.5 to enhance the competition between foreground and background; sigmoid is used when d≤0.5 to preserve weak responses and maintain gradient flow. The formula is as follows:

[0051] (3);

[0052] The learnable decision factor dynamically adjusts the normalization strategy during the training phase, enabling the network to maintain gradient stability in the early stages and achieve high contrast in the later stages, without the need for manual threshold setting.

[0053] Step 4.3, Secondary calibration of head importance;

[0054] After adaptive normalization, some heads may still carry redundant texture information. LAF introduces a head importance vector η∈R. 8 The normalization results are then weighted again:

[0055] (4);

[0056] In back propagation It automatically converges to a soft-gated distribution where "discriminant head is close to 1 and noise head is close to 0", which is more flexible than hard deletion and can avoid accidentally deleting potential heads that are effective in small target scenarios.

[0057] Step 4.4, inter-layer learning and fusion;

[0058] The 6-layer encoder features focus on local details and global semantics respectively. Manually specifying the top layer or skipping connections is difficult to adapt to targets of different scales. LAF first divides each layer into layers... l Flattened into a tensor of shape (B, Lq, 32); then passed through a learnable vector w_layer ∈ R 6 The layer weights α∈R are obtained after softmax normalization. 6, where α_l represents the weight of the l-th layer. The flattened features of the 6 layers are weighted and summed by α, and the fused feature F is output with shape (B, Lq×32). This weighted summation completes the "soft layer selection", where the weights of lower layers are automatically increased for small target scenes and the weights of higher layers are increased for large target scenes, without the need for manual design of connection rules, and the gradient path remains consistent with the original network.

[0059] Step 4.5, Bilinear bottleneck enhancement;

[0060] To model the nonlinear collaboration between the head and reference point without introducing expensive computations, LAF employs a bilinear bottleneck with a compression ratio of 2:

[0061] (5);

[0062] Among them, FC↓:32→16, FC↑:16→32, LayerNorm normalizes the 16-dimensional intermediate features, and the residual connection preserves the gradient path, thereby improving the cross-dimensional feature interaction capability with almost no increase in inference latency.

[0063] Step 4.6: Spatial saliency map generation;

[0064] The enhancement feature E is mean-meaned along the channel dimension and reshaped to obtain a saliency map M of shape (B,H,W). M serves as the sampling probability map for DPRM to guide the prototype seed to fall into the high-response region. It can also be multiplied point-by-point with the front-end features to achieve foreground enhancement and background suppression without altering the detection head weights and structure. This design allows LAF to be shared between the localization prior and detection mask tasks without requiring additional parameters or nodes.

[0065] Step 5, Dual-Stream Prototype Region Memory Module (DPRM) Construction;

[0066] Step 5.1, activate class filtering;

[0067] To avoid inactive regions interfering with the construction of region knowledge, DPRM first performs effective category activation on the saliency map M output by LAF based on the current episode's supported label set, resulting in a class-specific heatmap M_c with shape (B, H, W). This filtering step ensures that subsequent sampling and clustering focus only on the categories to be detected in this episode, without introducing additional parameters or computation nodes.

[0068] Step 5.2, Dual-stream progressive calibration;

[0069] In the early stages of training, the model's localization of the target region is not accurate, and relying entirely on the self-supervised similarity map can easily lead to sampling bias. To address this, DPRM constructs a strongly supervised binary mask M_gt and a self-supervised signal M_proto, and progressively fuses them using linear coefficients λ.

[0070] (6);

[0071] Where λ increases linearly with the training period, and its value ranges from [0,1]: the current number of periods divided by the preset maximum number of periods, the result is limited to the interval between 0 and 1. M_gt generates a binary mask based on the support box coordinates, the support box coordinates are directly given by the annotation information gt, and mapped onto the class-specific heatmap M_c; M_proto is generated by calculating the cosine similarity between the learnable class prototype q_c and the Encoder features:

[0072] (7);

[0073] Where τ=0.1, q_c∈R256, f_{i,j}∈R256, is the feature vector of the Encoder at position (i,j), softmax is calculated along the spatial dimension (i,j), and the output shape is the same as M.

[0074] In the early stages, λ is close to 0, and the target is quickly locked by relying mainly on label masks. In the later stages, λ is close to 1, and the target is driven by prototype similarity, gradually getting rid of the dependence on the bounding boxes. In the inference stage, there are no labels, and M_proto is used directly to achieve prior localization with zero label dependence.

[0075] Step 5.3, significance-weighted sampling;

[0076] After normalizing M_fused into a probability distribution along the spatial dimension, multinomial sampling is used to extract k=64 initial seeds. The higher the significance, the easier it is to be selected, ensuring that the seeds fall into the high-density area of ​​the target. If M_fused is all zero (extremely weak response or no target in the image), it degenerates into uniform sampling to avoid the module from crashing due to empty pool and improve robustness.

[0077] Step 5.4, Temperature softening K-means clustering;

[0078] DPRM uses 64 initial seeds as the center and performs three rounds of temperature-softened K-means (temperature T=0.1). In each round, the Euclidean distance from the pixel to the center is calculated first, and then the soft membership degree is obtained by exponential weighting and temperature normalization. Finally, the center position is updated using a weighted average. This soft assignment causes the center to continuously drift towards the salient high-density region, avoiding the boundary error of hard assignment. It converges in three rounds, and the computational cost is negligible.

[0079] Step 5.5, Radius and Feature Extraction;

[0080] To balance "covering the target subject" and "suppressing background intrusion", DPRM selects the 64 pixels with the highest membership degree for each drift center, calculates their Euclidean distance to the center, takes the 90th percentile value as the initial radius, then maps the center back to the encoder feature map, uses nearest neighbor sampling to extract features within the radius, and after average pooling, obtains f_j∈R256, forming 64 high-confidence prototype triples {c_j,r_j,f_j}, without adding any new learnable parameters.

[0081] Step 5.6, episode-level reduction and refinement;

[0082] The 64 prototypes generated from a single image may have redundancy or offset. DPRM merges the same prototypes from all images into a shared buffer pool within the episode, performs radius-weighted K-means, and prioritizes the smaller radius prototypes as new centers. The 90% quantile radius is estimated again and updated to obtain the final 64 high-confidence representatives.

[0083] Step 6: Construction of Region Guided Self-Attention Module (RGSA);

[0084] Step 6.1, read the region memory;

[0085] To avoid invalid interactions between the DecoderQuery and the background Key during initialization, RGSA first reads the activation region knowledge of the current episode from the DPRM shared cache before computation of each self-attention layer: centers∈R^{A×64×2} are the normalized center coordinates, radius∈R^{A×64} is the normalized radius (used for Gaussian decay), and features∈R^{A×64×256} are the region features (consistent with the Query dimension). This reading process only involves tensor indexing and copying, without adding any new learnable parameters, ensuring zero latency during the inference phase.

[0086] Step 6.2, position gating weights;

[0087] To achieve "only spatially prototypal prototypes participate in enhancement", RGSA calculates the Gaussian kernel distance for each Query reference point t∈R² in the Decoder:

[0088] (8);

[0089] Where t∈R² is the normalized coordinate of the current Query reference point on the feature map; c_{a,j}∈R² is the normalized center coordinate of the j-th region knowledge of class a, with the same scale as t; ‖t−c_{a,j}‖² is the squared Euclidean distance between the two points; r_{a,j}∈R⁺ is the normalized radius of the prototype, which controls the decay rate (steep for small radii, gentle for large radii); the kernel function outputs w_pos∈R^{A×64}, which only gives significant weight to neighboring prototypes, suppresses long-distance interference, and achieves spatial gating.

[0090] Step 6.3, Category gating weights;

[0091] To prevent the query response semantics from being out of prototype, RGSA employs a layered gating strategy.

[0092] Layer 0 (cold start) uses zero-parameter cosine similarity:

[0093] (9);

[0094] Where Q_n∈R256 is the current Query vector; q_a∈R256 is the learnable class prototype of class a, with the same dimension as Q_n; cos is the vector cosine similarity, with an output range of [-1,1]; the coefficient 20 is used to amplify the similarity difference, avoid random initialization perturbation, and ensure that the first layer produces a meaningful response.

[0095] Layer 1 and above (hot refine) use learnable embeddings:

[0096] (10);

[0097] Where W_l∈R^{256×A} is the learnable projection matrix, A is the number of activation classes, which is updated during training; tanh is the hyperbolic tangent activation, with an output range of [-1,1], maintaining the same scale as Equation (10); the coefficient 20 is consistent with the cold start, ensuring that the magnitude of logits remains unchanged. The two types of logits are expanded along the category dimension after softmax to 64 prototypes w_cls∈[0,1] for each class, ensuring that the Query only interacts with prototypes of the same semantic class, reducing inter-class confusion.

[0098] Step 6.4, Adaptive Gating

[0099] To prevent the background query from being misdirected, RGSA introduces a single hidden layer MLP to generate adaptive gating:

[0100] (11);

[0101] Where W1:256→64, W2:64→1. When g_n is close to 0, the regional memory is completely masked and the module degenerates into standard self-attention; when g_n is close to 1, the regional memory is fully injected, making full use of external priors; intermediate values ​​realize soft switching, without the need to manually set thresholds, realizing dynamic enhancement of "use when needed, discard when not needed".

[0102] Step 6.5, Weighted Fusion of Regional Features and Residual Injection

[0103] RGSA multiplies position, category, and adaptive triple gating element-wise:

[0104] (12);

[0105] The weight approaches 1 only when the conditions of "spatial proximity + semantic similarity + self-need" are met simultaneously, ensuring accurate injection of regional memory.

[0106] The prototype features are weighted and summed using w_n as the weight to obtain r_n∈R256, which is then added to the residual of the original query:

[0107] (13);

[0108] The Q'_n dimension is exactly the same as Q_n, and can directly replace the original input from the attention layer. No modifications are needed in subsequent processes such as FFN and LayerNorm.

[0109] Step 2: Train the region-guided Meta-DETR network model;

[0110] Step 1. Training Strategy and Task Setting

[0111] This invention adopts the 5-way 5-shot standard protocol: each episode randomly selects 5 categories, with 5 support images and 10 query images for each category, and the images are uniformly scaled to 640×640. The support boxes are only used for prototype construction and loss matching, without adding extra annotation overhead; the query image labels are only used for loss calculation and do not introduce gradient backpropagation, ensuring realistic deployment conditions with few samples.

[0112] Step 2. Loss Function and Optimizer

[0113] The total loss is consistent with Meta-DETR, including sigmoidfocal loss (classification), L1+GIoU loss (boundary regression) and bipartite graph matching cost. LAF, DPRM and RGSA do not add new learnable loss terms to avoid the risk of overfitting in small sample scenarios.

[0114] The optimizer uses AdamW with an initial learning rate of 1×10⁻ 4 Weight decay 1×10⁻ 4The batch size was 8, the total number of iterations was 47475, and cosine annealing plus warm-up was used for 500 iterations to ensure stability in the early stages of training.

[0115] Step 3. Two-stage training process

[0116] Phase 1 (Basic Pre-training): Train for 50 epochs on the basic class to allow the parameters of modules such as LAF gate weights and RGSA embedding matrix to converge fully;

[0117] The second stage (joint fine-tuning): The base class and the new class jointly participate in 5-shot fine-tuning for 25 epochs, retaining the base class samples to prevent catastrophic forgetting. DPRM and RGSA participate throughout the process but with zero new parameters, ensuring that the model capacity is controllable in small sample scenarios.

[0118] The experimental setup and analysis are briefly described below. Through comparative analysis of the experimental results, the actual detection effect has been improved.

[0119] 1. Experimental conditions

[0120] The hardware testing platform for this invention consists of an Intel(R) Xeon(R) CPU E5-2678v3 processor with a clock speed of 2.50GHz, 126GB of memory, and an Nvidia GeForce RTX 3090 graphics card. The software platform is an Ubuntu 18.04 operating system, and the development environment is PyCharm. The programming language is Python, and the deep learning network architecture is PyTorch.

[0121] 2. Experimental Data

[0122] The performance evaluation of this invention was conducted on the DIOR remote sensing target detection dataset. The DIOR dataset contains 20 common target categories (such as aircraft, ships, vehicles, bridges, airports, etc.), with images sourced from Google Earth and Gaofen satellites. The spatial resolution is 0.5m–1.0m, and the image size is 800×800 pixels, covering various scenes such as cities, ports, airports, and farmland.

[0123] The model was trained and evaluated using the training set (5862 images), validation set (5863 images), and test set (11738 images) provided by DIOR. All images were uniformly scaled to 640×640 pixels before being input into the network, maintaining the original aspect ratio, and the short sides were treated with symmetrical zero padding.

[0124] 3. Performance Comparison

[0125] This invention is compared with the following three existing techniques on the DIOR remote sensing target detection dataset:

[0126] (1) Meta-DETR: The meta-learning detection framework proposed by Zhang et al. adopts a support-query dual-path Transformer and makes direct predictions at the image level without the need for region proposal. It is the closest existing technology with the same architecture and task as this invention. It does not introduce region priors and its attention is easily affected by background interference, which is the starting point for the improvement of this invention.

[0127] (2) P-CNN: Prototype-guided CNN proposed by Cheng et al. generates class prototypes through global average pooling and guides RPN to generate foreground proposals;

[0128] (3) Meta-RCNN: A meta-learning detector proposed by Yan et al., which uses class attention to reweight the supporting features and then embeds the query branch.

[0129] In the experiment, the method proposed in this invention is denoted as RG-MetaDETR. The experiment adopted the DIOR official 4-split partitioning, with each split containing 15 basic classes and 5 novel classes; during the testing phase, both basic and novel classes participated in the evaluation. The evaluation metric was the mean accuracy (mAP) of the 5 novel classes at 0.5, calculated according to the PASCALVOC protocol: each predicted bounding box was sorted in descending order of confidence, and an IoU was calculated with the ground truth bounding box. A true positive (TP) was recorded when the IoU was greater than or equal to 0.5; otherwise, it was recorded as a false positive (FP). After plotting the PR curve, the AP was calculated, and finally, the average AP of the 5 novel classes was taken as mAP. At the same threshold, a higher mAP indicates better overall detection performance of the model.

[0130] Table 1 shows the average mAP@0.5 for each method on the 4-split test set for novel classes (5-shot).

[0131] Model Split1 Split2 Split3 Split4 average Meta-DETR 36.3 33.4 30.1 21.2 30.3 P-CNN 22.8 14.9 18.8 17.5 18.5 Meta-RCNN 13.9 10.9 12.3 15.8 13.2 RG-MetaDETR 39.2 35.7 32.0 23.9 32.7

[0132] As shown in Table 1, the present invention achieves a 2.4 percentage point improvement over the closest prior art Meta-DETR, outperforming other comparative methods. The improvement mainly comes from the enhancement of region-level features of novel classes with few samples by the Learnable Attention Fusion Module (LAF), the Dual-Stream Prototype Region Memory Module (DPRM), and the Region Guided Self-Attention Module (RGSA), verifying the substantial progress of the present invention without increasing the learnable parameters.

[0133] Experimental results comparison Figure 5 , Figure 6As shown, Meta-DETR, lacking region priors, suffers from global self-attention drifting to the background, leading to increased missed detections of novel classes; P-CNN relies solely on global average prototypes, lacking local anchors, resulting in blurred boundaries between similar-looking classes; Meta-RCNN's class attention overfits to supporting samples in 5-shot tests, exhibiting significant localization bias. In contrast, this invention generates high-contrast region saliency maps through LAF, extracts and caches "center-radius-feature" triples online using DPRM, and injects region knowledge into the decoder query using position gating and category gating, enabling high coupling between attention weights and target space-semantics, thus improving the accuracy of novel class detection.

[0134] In summary, this invention, based on the Meta-DETR image-level framework, introduces a region-guided trilogy: learnable attention fusion, dual-stream prototype region memory, and region-guided self-attention. Under the condition of 5-way 5-shot and evaluation of only novel classes, the mAP@0.5 is improved by 2.4 percentage points compared to the Meta-DETR detection method, which improves the localization accuracy and class discrimination ability in low-sample scenarios. It can be widely used in practical applications such as remote sensing images that require rapid adaptation to new classes.

Claims

1. A region-guided Meta-DETR method for few-shot target detection, characterized in that: It includes the following three modules: 1) Learnable Attention Fusion Module (LAF); The learnable attention fusion module takes the attention weights output from each layer of the encoder as input and reweights the attention map through learnable head weights and point weights; it introduces a lightweight decision network to dynamically select a normalization strategy to achieve adaptive separation of foreground and background; and it generates a high-contrast spatial heatmap through inter-layer fusion, inter-head competition and bilinear bottleneck structure to guide the localization of the target region. 2) Dual-stream prototype region memory module (DPRM); In each episode, the dual-stream prototype region memory module extracts class-specific region information from the heatmap based on the current activated category, and generates the region center, radius and features for each category. The dual-stream progressive prior calibration strategy uses bounding boxes to quickly locate the target region in the early stage of training, and gradually transitions to self-supervised region modeling based on learnable prototypes in the later stage. High-confidence region prototypes are generated through saliency sampling and temperature-softened K-means clustering, and weighted reduction and refinement are performed at the episode level to form stable prior knowledge of category regions. 3) Region-guided self-attention module (RGSA); The region-guided self-attention module introduces a region memory enhancement mechanism during the decoding stage. Based on the spatial distance and semantic similarity between the reference point of the query and the region prototype, it calculates the position gating and category gating weights and adaptively fuses external region features.

2. The region-guided Meta-DETR few-shot target detection method according to claim 1, characterized in that, Constructing a region-guided Meta-DETR network model involves the following steps: Step 1, Backbone Network; Step 2, Feature output and TransformerEncoder configuration; Step 3, Define the module insertion point; Step 4 allows you to learn how to build the Attention Fusion Module (LAF). Step 5: Construction of the Dual-Stream Prototype Region Memory Module (DPRM); Step 6: Construction of the Region-Guided Self-Attention Module (RGSA).

3. The region-guided Meta-DETR few-shot target detection method according to claim 2, characterized in that, Step 1 uses ResNet101 as the backbone network for extracting features from the region-guided Meta-DETR network model, outputting a layer 4 feature map with a spatial resolution of 1 / 32 of the input image and 1024 channels.

4. The region-guided Meta-DETR few-sample target detection method according to claim 2, characterized in that, Step 2 includes directly feeding the C5 feature map output from the backbone network into the TransformerEncoder. The encoder consists of 6 stacked DeformableDETR encoder layers, each containing an 8-head self-attention mechanism with a hidden dimension of 256. Each layer outputs an attention weight tensor A. l The shape is (B,Lq,8,4), where B represents the batch size, Lq=H×W is the flattened length of the feature map, 8 is the number of attention heads, and 4 is the number of deformable reference points.

5. The region-guided Meta-DETR few-shot target detection method according to claim 2, characterized in that, Step 3 includes, LAF module: Inserted after TransformerEncoder and before TransformerDecoder, it receives all 6 layers of deformable attention weights A. l By learning-based reweighting, adaptive normalization, inter-layer fusion, and bilinear bottleneck enhancement, a single-channel spatial saliency map M∈R^{B×H×W} is generated. DPRM module: Instantiated at the start of each episode; Using M as the probability map, 64 region prototype triples {c,r,f} are generated for the current activation category through saliency sampling, temperature-softened K-means, and 90th percentile radius estimation, and are dynamically accumulated as the episode propagates forward; The RGSA module is a self-attention layer embedded in the TransformerDecoder, which performs region memory injection before multi-head self-attention computation. Its input and output tensor shapes, data types, and residual connection methods are completely consistent with the standard Multi-HeadAttention, without requiring modification of FFN, LayerNorm, or subsequent cross-attention processes.

6. The region-guided Meta-DETR few-shot target detection method according to claim 2, characterized in that, Step 4 includes, step 4.1, learning reweighted units; LAF first introduces learnable scalar weights w_head∈R 8 with w_point∈R 4 Multiply element-wise with the original attention map: （1）； in, This indicates element-wise multiplication, with weights. Broadcast along the head dimension, Broadcast along the reference point dimension, with both initial values ​​set to 1 to ensure that the weights of each head and each point are approximately equal in the early stages of training; Step 4.2, Adaptive Normalization Unit; LAF calculates the point statistics μ∈R32 of the channels along the batch and query dimensions, and outputs the decision factor d∈[0,1] after two fully connected layers: （2）； Where W1: 32→64, W2: 64→1, when d>0.5, softmax is used to enhance the competition between foreground and background; when d≤0.5, sigmoid is used to preserve weak responses and maintain gradient flow, as shown in the following formula: （3）； Step 4.3, Secondary calibration of head importance; After adaptive normalization, some heads may still carry redundant texture information; LAF introduces a head importance vector η∈R 8 The normalization results are then weighted again: （4）； In back propagation Automatically converges to a soft-gated distribution where the discrimination head is close to 1 and the noise head is close to 0; Step 4.4, inter-layer learning and fusion; The 6-layer encoder features focus on local details and global semantics respectively. Manually specifying the top layer or skipping connections is difficult to adapt to targets of different scales; LAF first divides each layer into layers... l Flattened into a tensor of shape (B, Lq, 32); then passed through a learnable vector w_layer ∈ R 6 The layer weights α∈R are obtained after softmax normalization. 6 , where α_l represents the weight of the l-th layer; the 6-layer flattened features are weighted and summed according to α, and the fused feature F is output with shape (B, Lq×32); Step 4.5, Bilinear bottleneck enhancement; To model the nonlinear collaboration between the head and reference point without introducing expensive computations, LAF employs a bilinear bottleneck with a compression ratio of 2: （5）； Among them, FC↓:32→16, FC↑:16→32, LayerNorm normalizes the 16-dimensional intermediate features, and residual connections preserve gradient paths, thereby improving the cross-dimensional feature interaction capability with almost no increase in inference latency. Step 4.6: Spatial saliency map generation; The saliency map M of shape (B,H,W) is obtained by averaging the enhanced feature E along the channel dimension and reshaping it. M is used as the sampling probability map of DPRM to guide the prototype seed to fall into the high response region, and can be multiplied point by point with the front-end features to achieve foreground enhancement and background suppression without changing the weight and structure of the detection head.

7. The region-guided Meta-DETR few-sample target detection method according to claim 2, characterized in that, Step 5 This includes step 5.1, activation-based filtering; To avoid inactive regions interfering with the construction of region knowledge, DPRM first performs effective category activation on the saliency map M output by LAF based on the current episode's supported label set, resulting in a class-specific heatmap M_c with the shape (B, H, W). Step 5.2, Dual-stream progressive calibration; In the early stages of training, the model's localization of the target region is not accurate, and relying entirely on the self-supervised similarity map can easily lead to sampling bias. To address this, DPRM constructs a strongly supervised binary mask M_gt and a self-supervised signal M_proto, and progressively fuses them using linear coefficients λ. （6）； Where λ increases linearly with the training period, and its value ranges from [0,1]: the current number of periods divided by the preset maximum number of periods, the result is limited to the interval between 0 and 1; M_gt generates a binary mask based on the support box coordinates, the support box coordinates are directly given by the annotation information gt, and mapped onto the class-specific heatmap M_c; M_proto is generated by calculating the cosine similarity between the learnable class prototype q_c and the Encoder features: （7）； Where τ=0.1, q_c∈R256, f_{i,j}∈R256, is the feature vector of the Encoder at position (i,j), softmax is calculated along the spatial dimension (i,j), and the output shape is the same as M; In the early stages, λ is close to 0, and the system mainly relies on label masks to quickly lock onto the target. In the later stages, λ is close to 1, and the system is driven by prototype similarity, gradually getting rid of the dependence on the bounding boxes. In the inference stage, there are no labels, and M_proto is used directly to achieve prior localization with zero label dependence. Step 5.3, significance-weighted sampling; After normalizing M_fused into a probability distribution along the spatial dimension, multinomial sampling is used to extract k=64 initial seeds, which fall into the target high-density region; if M_fused is all zero, it degenerates into uniform sampling to avoid the module crashing due to empty pool and improve robustness. Step 5.4, Temperature softening K-means clustering; DPRM uses 64 initial seeds as the center and performs three rounds of temperature-softened K-means. In each round, the Euclidean distance from the pixel to the center is calculated first, and the soft membership degree is obtained by exponential weighting and temperature normalization. Then, the center position is updated by weighted average. Step 5.5, Radius and Feature Extraction; To balance covering the target subject and suppressing background intrusion, DPRM selects the 64 pixels with the highest membership for each drift center, calculates their Euclidean distance to the center, takes the 90th percentile value as the initial radius, then maps the center back to the encoder feature map, uses nearest neighbor sampling to extract features within the radius, and after average pooling, obtains f_j∈R256, forming 64 high-confidence prototype triples {c_j,r_j,f_j}, without adding any new learnable parameters; Step 5.6, episode-level reduction and refinement; The 64 prototypes generated from a single image may have redundancy or offset. DPRM merges the same prototypes from all images into a shared buffer pool within the episode, performs radius-weighted K-means, and prioritizes the smaller radius prototypes as new centers. The 90% quantile radius is estimated again and updated to obtain the final 64 high-confidence representatives.

8. The region-guided Meta-DETR few-sample target detection method according to claim 2, characterized in that, Step 6 includes, step 6.1, reading the region memory; Before calculating each self-attention layer, RGSA first reads the activation class region knowledge of the current episode from the DPRM shared cache: centers∈R^{A×64×2} are normalized center coordinates, radius∈R^{A×64} is the normalized radius, and features∈R^{A×64×256} are region features. Step 6.2, position gating weights; To ensure that only spatially prototypical prototypes participate in enhancement, RGSA calculates the Gaussian kernel distance for each Query reference point t∈R² in the Decoder: （8）； Where t∈R² is the normalized coordinate of the current Query reference point on the feature map; c_{a,j}∈R² is the normalized center coordinate of the j-th region knowledge of class a, with the same scale as t; ‖t−c_{a,j}‖² is the squared Euclidean distance between the two points; r_{a,j}∈R⁺ is the normalized radius of the prototype, and the kernel function outputs w_pos∈R^{A×64}; Step 6.3, Category gating weights; To prevent the query response from having a semantically heterogeneous prototype, RGSA employs a layered gating strategy; Layer 0 uses zero-parameter cosine similarity: （9）； Where Q_n∈R256 is the current Query vector; q_a∈R256 is the learnable class prototype of class a, with the same dimension as Q_n; cos is the vector cosine similarity, with an output range of [-1,1]; the coefficient 20 is used to amplify the similarity difference; Learnable embeddings are used in layers 1 and above: （10）； Where W_l∈R^{256×A} is the learnable projection matrix, A is the number of activation classes, which is updated during training; tanh is the hyperbolic tangent activation, with an output range of [-1,1], maintaining the same scale as Equation (10); the coefficient 20 is consistent with the cold start, ensuring that the magnitude of logits remains unchanged; the two types of logits are expanded along the category dimension after softmax to 64 prototypes w_cls∈[0,1] for each class, ensuring that the Query only interacts with prototypes of the same semantic class, reducing inter-class confusion; Step 6.4, Adaptive Gating; To prevent the background query from being misdirected, RGSA introduces a single hidden layer MLP to generate adaptive gating: （11）； Where W1:256→64, W2:64→1; Step 6.5: Weighted fusion of regional features and residual injection; RGSA multiplies position, category, and adaptive triple gating element-wise: （12）； The prototype features are weighted and summed using w_n as the weight to obtain r_n∈R256, which is then added to the residual of the original query: （13）； The dimensions of Q'_n are exactly the same as those of Q_n.

9. The region-guided Meta-DETR few-shot target detection method according to claim 1, characterized in that, The training region-guided Meta-DETR network model includes the following: Step 1. Training Strategy and Task Setting The 5-way 5-shot standard protocol is adopted: 5 categories are randomly selected for each episode, with 5 supporting images and 10 query images for each category. The images are uniformly scaled to 640×640. Step 2. Loss Function and Optimizer The total loss remains consistent with Meta-DETR, including sigmoidfocalloss, L1+GIoUloss, and bipartite graph matching cost. LAF, DPRM, and RGSA do not add any new learnable loss terms. The optimizer uses AdamW with an initial learning rate of 1×10⁻ 4 Weight decay 1×10⁻ 4 Batch size 8, total iterations 47475, cosine annealing + warm-up 500 iterations to ensure stability in the early stages of training; Step 3. Two-stage training process Phase 1: Train for 50 epochs on the base class to allow the LAF gated weights and RGSA embedding matrix module parameters to fully converge; Phase 2: The basic class and the new class jointly participate in 5-shot fine-tuning for 25 epochs.

Images