A Cross-Domain Small Sample Target Detection Method Based on Dynamic Information Fusion

Abstract

This invention discloses a cross-domain small-sample target detection method based on dynamic information fusion, applicable to fields such as industrial defect detection, aiming to solve problems such as inter-domain distribution differences and scarce annotations in cross-domain small-sample scenarios. Specifically, it involves first dividing the source / target datasets, constructing a DIC-ViT model with two branches, and sharing the DINOv2 backbone network to extract features; then generating anchor boxes through a region proposal network, and optimizing sample partitioning using both IoU value and class center distance as criteria; subsequently, adaptive feature fusion is achieved through a dynamic information coupling module, and a contrastive learning module generates discriminative class prototypes; finally, the model is optimized using an objective function containing classification, regression, and contrastive losses. This method improves the robustness and accuracy of target detection, performs excellently in multi-dataset tests, and is suitable for detection scenarios with scarce samples.

Description

Technical Field

[0001] This invention relates to the field of industrial defect detection, and in particular to a cross-domain small-sample target detection method based on dynamic information fusion. Background Technology

[0002] Few-shot object detection (FSOD) aims to address the critical issue of scarce labeled data in practical applications, particularly in scenarios such as rare disease identification and defect detection in medical imaging. Its core challenge lies in achieving accurate localization and classification of new object categories using a very small number of labeled samples. Current research mainly revolves around two paradigms: transfer learning-based methods employ a "pre-training-fine-tuning" strategy, first training on base class datasets (such as PASCAL VOC and MSCOCO), and then fine-tuning the detection head using a small number of new class samples to balance the performance of old and new classes and suppress overfitting; meta-learning-based methods learn cross-task generalization capabilities through task-driven episode training (such as support set-query set mechanisms), enabling rapid adaptation to new classes. However, FSOD still faces many difficulties and challenges: overfitting and insufficient generalization, with the model being overly sensitive to low-quality samples due to the scarcity of samples, resulting in high-confidence false detections; cross-domain shift and data bias, with the difference in distribution between the base class and the new class causing negative transfer, and small samples making it difficult to represent the diversity of real data; prototypes generated through meta-learning cannot adaptively adjust according to the query image and lack sufficient representative information; small and blurry targets account for a small proportion and have low resolution in the image, and their subtle features are easily ignored, increasing the difficulty of detection.

[0003] Cross-domain few-sample object detection is a cutting-edge research direction in computer vision, aiming to address the challenge of training and testing data coming from different domains with only a small number of labeled samples in object detection tasks. In real-world scenarios, cross-domain characteristics of data are ubiquitous. For example, in surveillance video object detection, data collected during the day and night differ in terms of illumination and color distribution; medical image data from different hospitals may also differ significantly in imaging equipment, resolution, and contrast. In these cross-domain scenarios, traditional object detection methods often experience a significant performance degradation due to inter-domain distribution differences. Therefore, conducting research on cross-domain few-sample object detection has significant practical implications. The main challenges facing cross-domain few-sample object detection are as follows:

[0004] (1) Differences in feature distribution between the source and target domains: The data in the source and target domains differ significantly in terms of features such as color, texture, shape, and background. For example, the appearance of a target in a natural image may vary due to factors such as shooting conditions and seasonal changes, and targets in indoor and outdoor scenes may also differ greatly in terms of lighting and background complexity. This inconsistency in feature distribution makes it difficult for the feature representations learned by the model in the source domain to be directly applied to the target domain, resulting in a decrease in the model's generalization ability in the target domain.

[0005] (2) Scarcity of labeled information: In cross-domain small sample target detection, there are only a few labeled samples in the target domain, which is difficult to fully cover the distribution and changes of the target domain data, thus limiting the model's adaptability and learning effect to the target domain. At the same time, the limited number of labeled samples also makes the model prone to overfitting to these few samples and unable to generalize well to unseen samples.

[0006] (3) Class shift: The source domain and the target domain may involve different target categories, or the same category may have different representations in different domains. For example, in traffic sign detection, traffic signs in different countries and regions differ in shape, color, pattern, etc. This makes it difficult for the model to accurately identify the target category when crossing domains, resulting in class shift and affecting the accuracy of target detection.

[0007] (4) Model adaptability challenge: Due to the complexity of cross-domain problems, models need to be more adaptable to handle changes between different domains. However, many existing few-sample object detection models mainly focus on how to learn effective feature representations using a small number of labeled samples, and give little consideration to cross-domain problems, which makes it difficult for them to achieve good performance in cross-domain scenarios. The adaptability and robustness of the models face huge challenges. Summary of the Invention

[0008] The purpose of this invention is to provide a cross-domain small sample target detection method based on dynamic information fusion, which aims to solve problems such as inter-domain distribution differences and scarce annotations in cross-domain small sample scenarios.

[0009] To achieve the above objectives, this invention provides a cross-domain small-sample target detection method based on dynamic information fusion, the steps of which are as follows:

[0010] S1. Define the cross-domain small sample object detection task: Divide the dataset into source datasets containing sufficient labeled samples. and target dataset containing K-shot samples Clearly define the source dataset Data distribution PS and target dataset The data distribution is PT, and the test task adopts an N-way K-shot setting. Each task contains N categories, and r labeled instances are selected for each category as the support set S, and the remaining unlabeled samples are used as the query set Q.

[0011] S2. Construct the DIC-ViT network model and extract basic features: The DIC-ViT network model includes query branches and support branches, which share the weights of the DINOv2 backbone network based on the ViT architecture; given a query image... and each category Supporting images S, the query image Input query branches, supporting image input support branches, and extracting query features separately through the DINOv2 backbone network. and supporting features ;

[0012] S3. Candidate anchor boxes are generated using the region proposal network built into the DIC-ViT network model, and then the sample partitioning is optimized by the region proposal re-screening module. The region proposal network is embedded in the candidate region generation process of the DIC-ViT network model, based on the query features extracted in S2. and supporting features The system generates several candidate anchor boxes that may contain the target by using a preset multi-scale and multi-aspect ratio anchor point mechanism; the region proposal re-screening module calculates the center vector of each category based on the support features extracted by S2, and divides positive and negative samples by combining the "IoU value between the anchor box and the real target box" and the "metric distance from the anchor box to the corresponding category center" as dual criteria, and dynamically updates the category center vector during the training process;

[0013] S4. Feature adaptive fusion is achieved through the dynamic information coupling module: The dynamic information coupling module receives the query features output by S2. and supporting features In addition to the candidate region information generated by the region proposal network and filtered by the region proposal re-filtering module in S3, sparse local attention is first used to couple query feature information to focus on key regions, and then gated attention is used to support feature information fusion for dynamic weighting to generate customized query features. and coupled support features ;

[0014] S5. Generate a discriminative prototype through the contrastive learning module, using the coupled support features output from S4. As input, the vector is first transformed into a compact discriminative vector through a feature embedding layer, and then a class prototype is generated through iterative optimization by a contrastive prototype generator. ;

[0015] S6. Optimize the model parameters using an objective function L, the formula for which is:

[0016]

[0017] in, This indicates that the detection head is based on customized query features of S4. Category prototypes of S5 Calculated classification loss, Indicates that the detection head is based on The calculated regression loss, This represents the contrastive loss generated by the contrastive learning module of S5.

[0018] Preferably, S2 specifically includes the following sub-steps:

[0019] S2.1 Construct a dual-branch network structure: The query branch and the support branch are symmetrical structures. The number of network layers, convolution kernel size, and number of channels of the query branch and the support branch are completely identical. The query branch is input with a single query image, and the support branch is input with r support images grouped by category in the support set S.

[0020] S2.2 Determine the parameters of the DINOv2 backbone network: The DINOv2 backbone network is based on the ViT architecture, uses pre-trained weights for initialization, and the pre-training dataset is ImageNet-21K. During training, the parameters of the first 12 layers of the DINOv2 backbone network are frozen, and the parameters of the last 4 layers are fine-tuned. The feature output layer of the DINOv2 backbone network is selected as layer C4, the feature map resolution is 100×100, and the number of channels is 256.

[0021] S2.3 Image Preprocessing and Feature Extraction: The input query image and support images undergo unified preprocessing, including resizing them to 800×800 pixels and normalizing them according to the mean and variance. The preprocessed query image is then input into the query branch, and after forward propagation through the DINOv2 backbone network, the C4 layer features are output, denoted as the query features. The preprocessed support images are grouped by category and input into the support branch. After forward propagation through the DINOv2 backbone network, the corresponding C4 layer features are output, which are denoted as support features. ;

[0022] S2.4 Achieving Dual-Branch Weight Sharing: The DINOv2 backbone network parameters of the query branch and the support branch are fully bound. During training, only one set of DINOv2 backbone network parameters is updated. Both branches synchronously call the updated parameters for feature extraction, ensuring that the query features... and supporting features They exist in the same feature space.

[0023] Preferably, S3 specifically includes the following sub-steps:

[0024] S3.1: Region Proposal Network Generates Candidate Anchor Boxes: The region proposal network is a fully convolutional network structure, embedding the candidate region generation process of DIC-ViT, based on the output of S2. and supporting features The corresponding feature map with a resolution of 100×100 is used to generate 9 types of anchor boxes by setting 3 scales and 3 aspect ratios of anchor points. The classification branch of the region proposal network predicts the confidence that each anchor box contains the target, and the regression branch predicts the position offset of the anchor box. The preset number of candidate anchor boxes are generated by combining confidence filtering and position correction.

[0025] S3.2, The region proposal re-selection module calculates the initial category center vector: for each category in the support set S Extract the support features of all supporting images in this category after S2 processing. Global average pooling is performed on each support feature map to obtain a D-dimensional feature vector. The mean of the feature vectors of all support samples for that class is calculated, and this mean is used as the class. Initial class center vector ;

[0026] S3.3, the region proposal re-filtering module calculates the metric feature vector of the anchor box: For each candidate anchor box generated in S3.1, based on its coordinate position on the feature map, the query feature output from S2 is used. Or supporting features The features corresponding to the anchor frame are cropped out, compressed to D dimensions by 1×1 convolution, and then the metric feature vector of the anchor frame is obtained by global average pooling. ;

[0027] S3.4, The region proposal re-screening module initially screens samples based on IoU values: setting an IoU threshold. Calculate the IoU value between each candidate anchor box and all ground truth bounding boxes in the image. IoU = intersection area of ​​anchor box and ground truth bounding box / union area. If the IoU value of a candidate anchor box with any ground truth bounding box is greater than 1, the candidate anchor box is considered to have an IoU value greater than 1. If the IoU value between a candidate anchor box and all ground truth target boxes is less than or equal to 1, it is included in the candidate positive sample pool; if the IoU value between a candidate anchor box and all ground truth target boxes is less than or equal to 1, it is included in the candidate positive sample pool. These are directly identified as negative samples and will not proceed to further screening.

[0028] S3.5, The region proposal re-screening module filters samples a second time based on metric distance: For the j-th anchor box in the candidate positive sample pool obtained in S3.4, the metric value is calculated using the following formula:

[0029]

[0030] in, Indicates the first The anchor frame corresponds to the first The measure of each category, It is the first The metric feature vector of each anchor box. Indicates the number of categories, Indicates the number of anchor frames;

[0031] Take all The minimum value in the value is used as the minimum metric value of the anchor frame. The calculation formula is:

[0032]

[0033] Set measurement threshold ,like The anchor frame is determined to be a positive sample, and... Corresponding category The label is assigned to the anchor box; if The anchor frame was re-evaluated as a negative sample.

[0034] S3.6, The region proposal re-selection module updates the class center vector: After each training mini-batch, for each class included in the current mini-batch... Count the number N of samples that are judged as positive by S3.5 in this category. i Extract the metric feature vectors of these positive samples. The class center vector is updated using the following formula:

[0035]

[0036] in, It is the update step size, which is a constant between 0 and 1.

[0037] Preferably, S4 specifically includes the following sub-steps:

[0038] S4.1, Query Feature Information Coupling

[0039] S4.1.1 Support Feature Region Division: The support features output from S2 are... Divide the local window into P×P steps and extract the features of the 1×1 region at the center of each window as a local query.

[0040] S4.1.2, Query Feature Decomposition and Local Key Extraction: The query features output from S2 are... The vector is split into a key vector K and a value vector V. The K features of the corresponding region of each local window are extracted and obtained by average pooling.

[0041] S4.1.3 Calculate the local similarity weights using the following formula:

[0042]

[0043] in, For window indexing, , Indicates the first Average pooling of queries and keys within each window This means retaining the A largest values;

[0044] S4.1.4 Window-level Custom Feature Generation: Generate window features using the following formula:

[0045]

[0046] in, It is the projection matrix of the Value vector V, used to project the original query features. Mapped to a new feature space, After projection matrix After transformation, the first The re-encoded value vector of the query feature within a local window;

[0047] S4.1.5, Customized Query Feature Generation: Concatenating All The obtained feature map is upsampled to 100×100 through 1×1 convolution and bilinear interpolation to generate the similarity weight SW, as shown in the following formula:

[0048]

[0049] Then, V is weighted and summed using similarity weights to generate customized query features. The calculation formula is:

[0050]

[0051] S4.2 Supports feature information fusion

[0052] S4.2.1, Gating Weight Prediction: Predicting query features Global average pooling vector and support features The components are concatenated position by position, and then gated weights are generated using lightweight convolutional layers and activation functions. The formula is:

[0053]

[0054] S4.2.2, Hybrid Weight Generation: Calculating Query Features and supporting features cosine similarity Set adjustment parameters Generate the mixed weights using the following formula. :

[0055]

[0056] S4.2.3, Coupling Support Feature Generation: This involves generating the recombined query features. Support features are injected using a hybrid weighting method to generate coupled support features. The formula is as follows:

[0057] .

[0058] Preferably, S5 specifically includes the following sub-steps:

[0059] S5.1: Feature embedding layer transforms discriminative vectors:

[0060] Post-coupling support features of S4 output As input, a feature map is generated through a 1×1 convolution. The importance weights at each location are quantized using Sigmoid activation to obtain a spatial attention map. This attention map is then coupled with the resulting support features. Element-wise multiplication is performed for spatial weighting, and the weighted features are compressed into spatially independent vectors using global average pooling. The compressed vectors are then reduced in dimensionality using a linear projection layer to generate discriminative embedding vectors. The formula is expressed as:

[0061]

[0062] in, It is the projection transformation matrix in the feature embedding layer. These are the 1×1 convolutional kernel weights of the spatial attention submodule in the feature embedding layer;

[0063] S5.2: Initialize the prototype using the comparison prototype generator: for each category Determine its sample index set Calculate the initial prototype using the following formula:

[0064]

[0065] in, Indicate category The number of samples, Indicates the first Embedding vectors of each sample;

[0066] S5.3: Iteratively Optimize Prototypes by Comparing Prototype Generators

[0067] S5.3.1: Definition of positive and negative samples: For each iteration step , Each sample Positive sample pairs are , among which Indicates sample The category belongs to the iteration step The prototype; the negative sample set is defined as ,in Represents a heterogeneous prototype sampled randomly;

[0068] S5.3.2: Comparison Loss Calculation: The calculation formula is as follows:

[0069]

[0070]

[0071] in, For similarity, For temperature parameters, The total number of samples;

[0072] S5.3.3: Prototype Gradient Update: Move the prototype along the negative gradient direction and update it according to the following formula to cluster similar prototypes and separate dissimilar prototypes. The formula is expressed as:

[0073]

[0074] in, Indicates the iteration step prototype vector, Indicates the optimization step size, the default is 0.1; Indicates the loss to the prototype The gradient operator;

[0075] S5.4: Output the final category prototype: After completing T iterations, collect the final prototypes of all categories. The category prototypes are obtained by concatenating them in category order:

[0076]

[0077] in, This represents the total number of categories.

[0078] Preferably, the contrast loss weights described in S6 The value is 0.5; the classification loss The cross-entropy loss is used to calculate the cross-entropy between the class probability output by the detection head and the true class label of the positive sample; the regression loss Using smoothed L1 loss, the coordinates of the target bounding box predicted by the detector are calculated. Smoothing L1 error between the actual target bounding box coordinates and the true target bounding box coordinates.

[0079] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0080] (1) A proposal region re-screening strategy is proposed: a feature space similarity measure is introduced in the RPN stage, and a dual criterion is constructed by combining IoU and the distance from the proposal region to the class center, which significantly alleviates the misjudgment of positive and negative samples and improves the robustness to occluded / blurred targets;

[0081] (2) A dynamic information coupling mechanism is proposed: by calculating query-support image similarity, adaptively fusing features and enhancing the response of the target region, suppressing background interference, and generating a general representation that is adaptive to the task;

[0082] (3) A contrast-driven prototype generation strategy is proposed: contrast learning is used to explicitly optimize intra-class aggregation and inter-class separation, enhance the discriminativeness of prototypes, effectively address the problem of large intra-class variance and high inter-class similarity in low-sample scenarios, and especially improve the detection accuracy of small targets and fine-grained categories.

[0083] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0084] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0085] Figure 1 This is a schematic diagram of the DIC-ViT network model structure according to an embodiment of the present invention;

[0086] Figure 2 Embodiments of the present invention Schematic diagram of modular network structure;

[0087] Figure 3 This is a cross-domain small sample target detection dataset used for training and validation in embodiments of the present invention;

[0088] Figure 4 This is a comparison chart of the test results of the method of the present invention and the current mainstream small sample target detection methods on multiple novel target datasets;

[0089] Figure 5 This is a test result diagram of an embodiment of the present invention;

[0090] Figure 6 This is a comparison chart of the test performance of different modules in this embodiment of the invention on ArTaxOr, Clipart1k, and DIOR.

[0091] Figure 7A comparison chart of the test performance of different modules on ArTaxOr under the 10-shot settings of this invention embodiment;

[0092] Figure 8 This is a comparison diagram of the effects of the RPR module before and after processing in an embodiment of the present invention;

[0093] Figure 9 Different thresholds for embodiments of the present invention Performance comparison chart on ArTaxOr. Detailed Implementation

[0094] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0095] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0096] Example

[0097] This embodiment discloses a cross-domain small-sample target detection method based on dynamic information fusion, including:

[0098] S1. Defining the Task. Few-shot object detection aims to detect new categories of objects using a small number of sample images (called support images). Specifically, in a typical few-shot object detection task, an object detection dataset C consists of a base class Cbase and a new class Cnovel, i.e. Among them, Cbase and Cnovel do not overlap, that is... Cbase contains ample labeled samples for model training, while CNovel contains only a small number of K-shot samples (labeled instances). This task aims to simulate the needs of low-data-volume object detection in real-world applications. In the cross-domain few-shot object detection task, the source dataset Cbase has a data distribution of PS, and the target dataset CNovel has a data distribution of PT, where PS ≠ PT. The model needs to be trained on the source dataset and then tested on the target dataset. To evaluate the model's FSL capability, the testing task uses an N-way K-shot configuration, where each task contains N classes, and each class provides... There are 1 labeled instances, also known as the support set S, and the other unlabeled samples are used as the query set Q.

[0099] S2. Cross-Domain Small Sample Object Detection Dataset Setup and Evaluation. The core metrics for cross-domain datasets focus on quantifying the differences between the source and target domains (inter-domain differences) and the complexity of the data's inherent structure (intra-class / inter-class structure). Inter-domain differences are commonly assessed using style differences (measuring the distribution distance of low-level visual attributes such as texture and color, often using Gram matrices or low-level feature statistics) and semantic differences (measuring changes in high-level concepts such as class proportions or context). The key metric for intra-class structure is inter-class variance (ICV), which measures the feature dispersion of samples of the same class in different domains; a higher ICV indicates greater difficulty in learning domain-invariant features. Inter-class structure focuses on undefined boundaries (IB), reflecting the degree of feature space overlap between classes due to visual similarity or domain shift; a higher IB means more blurred classification boundaries and a greater risk of confusion. These metrics collectively reveal the difficulty of transferring the dataset and guide algorithm selection and design. These metrics are primarily used to quantify the degree of difference between the source and target domains, thereby evaluating the challenge and applicability of transfer learning or domain adaptation algorithms.

[0100] S201: Dataset Setup. Under N-way K-shot conditions, training is first performed on the MS-COCO dataset, followed by evaluation on widely used FSOD benchmarks such as PASCAL VOC, as well as public datasets including ArTaxOr, DIOR, UODD, Clipart1k, DeepFish, and NEU-DET. Each new class has r object instances, with r set to 1, 5, and 10 respectively.

[0101] S202: Evaluation of Cross-Domain Dataset Dissimilarity. The following is a comprehensive evaluation of the core metrics (style dissimilarity, inter-class variance (ICV), and undefined boundary (IB)) for each dataset in cross-domain tasks. Evaluation levels are divided into three categories: Low, Medium, and High, with specific meanings shown in the table below:

[0102] Table 1 Evaluation results for each dataset

[0103]

[0104] S203: Baseline Method Selection. To more comprehensively evaluate the performance of our proposed model, we also conducted comparative tests with a variety of methods, such as typical FSOD methods like Meta-RCNN, TFA, FSCE, DeFRCN, and Distill-cdfsod, as well as newer methods based on the Vision Transformer, such as ViTDeT-FT, DE-ViT, DE-ViT-FT, and CD-FSOD.

[0105] S3. Construct the DIC-ViT network model, such as Figure 1As shown. The specific steps include:

[0106] S301. Construct two branches with the same backbone network, namely the query branch and the support branch, and share the backbone network weights of the two branches.

[0107] S302, Construct a dual-filter module for regional proposals. Specifically, S includes:

[0108] (1) Calculate the class center vector. Assume the class center vector is... ,in Let represent the category index, and D be the dimension of the metric features. For each category, calculate the mean of the feature vectors of all supporting samples in that category, and use this as the category center vector. .

[0109] (2) Calculate the metric from each anchor box to each category. For each anchor box, calculate the metric based on its metric feature vector and the center vectors of each category. The metric can be calculated using the following formula:

[0110]

[0111] in, Indicates the first The anchor frame corresponds to the first The measure of each category, It is the first The metric feature vector of each anchor box. Indicates the number of categories, Indicates the number of anchor frames.

[0112] (3) Initial sample segmentation based on IoU value. Positive and negative samples are re-segmented based on the metric and IoU value. For each anchor box, its IoU value with the ground truth target box is first calculated using the traditional RPN method. Then, anchor boxes with IoU values ​​greater than a certain threshold are considered candidate positive samples. To reduce the false negative rate, a relatively low threshold is set here. For example, 0.3.

[0113] (4) Re-divide the samples based on the metric values. For each candidate positive sample, calculate its metric value to each class center vector. To improve computational efficiency and reliability, we only select the smallest metric value among all supporting class center vectors as the basis for further judgment:

[0114]

[0115] Then we set a threshold. ,if If so, then the sample is classified as a positive sample for training, and... The corresponding support category label is assigned; otherwise, it is reclassified as a negative sample to reduce the false detection rate.

[0116] For IoU values ​​less than the threshold The anchor box is directly classified as a negative sample.

[0117] (5) Category center vector The class center vector is dynamically updated. During training, as the network learns more deeply from the samples, the class center vector... It will be updated according to certain rules. Specifically, after each mini-batch training, the mean of the feature vectors of all samples in that class is calculated, and then this mean is compared with the current class center vector. After weighted fusion according to a certain ratio, the updated category center vector formula is:

[0118]

[0119] in, It is the update step size (usually a constant between 0 and 1, such as 0.9). It belongs to the category The number of samples, It is the metric feature vector for each sample.

[0120] S303, Constructing a dynamic information coupling module It includes two parts: query feature information coupling and support for feature information fusion, such as Figure 2 As shown.

[0121] S303.1, Construct a query feature information coupling module. Specifically, S includes:

[0122] (1) Region division: Supporting features Divide the area into P×P local windows (e.g., 3×3), and extract the central region features for each window:

[0123] (2) Calculate local similarity: Calculate the similarity between the query features and the central region of each window, then sort the similarity scores within each window, and only retain the top A (e.g., A=4) high-response regions for aggregation. Calculate the local similarity weights of Q and K using matrix multiplication and Softmax:

[0124]

[0125] For window indexing, , Indicates the first Average pooling of queries and keys within each window This means retaining the largest A values.

[0126] (3) Weighted aggregation: The query feature values ​​are weighted and summed using local similarity weights to generate window-level customized features:

[0127]

[0128] It is the projection matrix of the value vector, and its function is to project the original query features. Mapping to a new feature space generates feature representations that are more suitable for attention-weighted processing.

[0129] (4) Calculate similarity weights. Through cross-window stitching and convolution fusion, using 1×1 convolution to keep the number of channels unchanged, and using bilinear interpolation upsampling to restore the original resolution H×W, the final similarity weights are generated. :

[0130]

[0131] (5) Feature Coupling: V is weighted and summed using similarity weights to generate customized query features. .final, It integrates key semantics of supporting features to achieve deep alignment between query and supporting information.

[0132]

[0133] Through the above S, the query image features (containing rich context) are used to actively "query" support set features to find the most relevant target information, thus achieving query feature guidance.

[0134] S303.2, Construct a module that supports feature information coupling. Specific steps include:

[0135] (1) Gated weight prediction: The global average pooling vector of the query features is compared with the support features. The components are concatenated position by position, and then gated weights are generated using lightweight convolutional layers and activation functions. :

[0136]

[0137] (2) Hybrid weight generation: combining the cosine similarity between query features and supporting features. With gate weights Generate mixed weights .in This is an adjustable parameter, initialized to 0.5.

[0138]

[0139] (3) Coupling information generation: The recombined query features By injecting support features with mixed weights, the position and intensity of the coupling region can be adaptively adjusted, thereby enhancing the robustness of the model to complex scenes (such as occlusion and viewpoint changes).

[0140]

[0141] S304. Construct a contrastive learning module to generate a discriminative prototype. Specific steps include:

[0142] S304.1 Spatial Feature Transformation. First, a spatial attention map is generated using a 1×1 convolution. This map uses a Sigmoid activation layer to quantize the importance weight (range 0-1) of each location. Then, this weight is used to spatially weight the input features, enhancing the target region and suppressing the background. The weighted features are then compressed into spatially independent vectors using global average pooling. Finally, a linear projection layer reduces the channel dimension. The entire process achieves efficient transformation from spatial features to discriminative embedding vectors, expressed by the formula:

[0143]

[0144] in, It is the projection transformation matrix in the feature embedding layer. It represents the 1×1 convolutional kernel weights of the spatial attention submodule in the feature embedding layer.

[0145] S304.2, Generate Discriminative Contrast Prototypes. A discriminative prototype generator based on iterative optimization drives intra-class aggregation and inter-class separation through contrastive loss.

[0146] (1) First, calculate the mean of the embeddings of similar supporting samples as the initial prototype. :

[0147]

[0148] in Indicate category The sample index set, Indicate category The number of samples, Indicates the first The embedding vectors of each sample are then used. Subsequently, for each sample, a three-step optimization process is performed: defining positive and negative samples, calculating the loss, and iterating the gradient.

[0149] (2) Definition of positive and negative samples. A positive sample pair is defined as... ,in Indicates sample The category belongs to the iteration step The prototype. The negative sample set is defined as... ,in Represents a randomly sampled heterogeneous prototype, by default. .

[0150] (3) Calculate the comparative loss.

[0151]

[0152] in:

[0153]

[0154] (4) Gradient iteration. Move the prototype along the negative gradient direction to make positive samples closer and negative samples farther away.

[0155]

[0156] in, Indicates the iteration step prototype vector, Indicates the optimization step size, the default is 0.1; Indicates the loss to the prototype The gradient operator.

[0157] (5) Final prototype output:

[0158]

[0159] S4. Model training and validation. For example... Figure 3 As shown, the dataset was first trained on the MS-COCO dataset, and then validated on nine datasets: PASCAL VOC (Novel Split 1, Novel Split 2, Novel Split 3), ArTaxOr, DIOR, UODD, Clipart1k, DeepFish, and NEU-DET. Table 2 and... Figure 4 The validation results of several methods on the above seven public datasets are presented. As shown in Table 2, the DIC-ViT method proposed in this invention outperforms the original DE-ViT on all target datasets. In addition, in most cases, the method of this invention also outperforms other mainstream few-shot target detection methods.

[0160] Table 2 Test results on multiple novel target datasets

[0161]

[0162] S5. Model Application. The trained model is applied to small sample target datasets such as industrial defect detection, power grid defect detection, and underwater target detection. Figure 5The test results of the method of the present invention on multiple datasets are shown. The first row, from left to right, shows the test results of the method of the present invention on datasets such as wild insects, cartoons, remote sensing images, and power targets. The second row, from left to right, shows the test results of the method of the present invention on datasets such as underwater fish, industrial defects, sea cucumbers and sea urchins, and work and life scenarios.

[0163] S6. Validation of Model Structure. To further verify the effectiveness of each module of the proposed method, we conducted step-by-step experiments on three public datasets: ArTaxOr, Clipart1k, and DIOR. The results are shown in Table 3.

[0164] Table 3. Performance of different modules in the 10-shot fine-tuning stage on ArTaxOr, Clipart1k, and DIOR.

[0165]

[0166] (1) The effects of different modules.

[0167] Table 3 illustrates the effectiveness of the proposed modules. Compared to the DE-ViT-FT baseline, the performance of the RPR module alone is significantly improved across all datasets, demonstrating that the proposed region re-selection strategy can effectively improve the target detection rate. Further addition of the DIC module enhances the class discrimination ability of the features, indicating that dynamic information fusion can significantly improve the feature representation ability of the prototype, thereby improving the model's generalization ability and detection accuracy for new categories. With the introduction of the contrastive learning module, the model achieves performance of 62.0%, 45.7%, and 33.4% on the ArTaxOr, Clipart1k, and DIOR datasets, respectively, representing a significant improvement over the baseline model. Overall, on the three public datasets, the synergistic effect of the three modules improves the model's performance by 12.8%, 4.9%, and 7.8% compared to the baseline, respectively. Figure 6 , Figure 7 As shown.

[0168] (2) The effect of proposed regional re-screening.

[0169] After adding the RPR module, the model's detection accuracy on ArTaxOr, Clipart1k, and DIOR improved by 4.4%, 1.9%, and 2.9% respectively compared to the baseline, demonstrating the effectiveness of the proposal region re-selection mechanism in object detection networks under small sample conditions. We visualized the proposal regions before and after adding the RPR module, as shown below. Figure 8 As shown.

[0170] (3) The effect of dynamic information coupling.

[0171] To verify the effectiveness of the dynamic information coupling mechanism, we further added the DIC module and retested it on three datasets. As shown in Table 3, the model performance improved by 4.7, 1.2, and 2.2 percentage points relative to the benchmark, respectively. The conclusions indicate that the dynamic information coupling module can extract more significant and useful information into the generated prototype.

[0172] (4) Compare the effects of the learning modules.

[0173] After introducing the contrastive learning module, the model's performance on the ArTaxOr, Clipart1k, and DIOR datasets improved by 3.7, 1.8, and 2.7 percentage points respectively compared to the baseline. This indicates that the contrastive learning mechanism can significantly improve the detector's utilization efficiency of scarce positive class samples when labeled samples are extremely scarce.

[0174] (5) Different thresholds The effect.

[0175] In RPR, in order to reasonably select the threshold ,Evaluate To assess the impact on model performance, we conducted ablation experiments on the DIOR dataset under different shot settings, and the results are as follows: Figure 9 As shown. Threshold The choice of IoU directly affects whether positive and negative samples can be correctly distinguished. Reducing the model's reliance on a single IoU judgment method improves the detection rate of true positive samples and reduces the false positive rate, thus ensuring that the model is trained under positive guidance. Experimental results show that under low sample settings (e.g., 1 shot), Its performance is slightly better, but at high sample settings (such as 5-shot and 10-shot), The performance is better. This may be because relaxing the threshold can improve the detection rate when the sample size is small, while appropriately lowering the threshold helps reduce the false positive rate when the sample size is large. In summary, setting the threshold is more efficient throughout the training process. This ensures that the model performs optimally under all shot settings.

[0176] The above experiments show that the proposed method achieves state-of-the-art performance on nine cross-domain small-sample target detection datasets, including PASCAL VOC and ArTaxOr, verifying the innovation of the invention and the effectiveness of the proposed method. Furthermore, it has broad application prospects in application scenarios where samples are scarce, such as industrial / power defect detection and underwater target recognition.

[0177] The remaining technical features in the above embodiments can be flexibly selected by those skilled in the art to meet different specific practical needs according to actual circumstances. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims. In the above description, numerous specific details have been set forth to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to implement the present invention. In other instances, to avoid obscuring the present invention, well-known techniques, such as specific construction details, operating conditions, and other technical conditions, have not been specifically described.

[0178] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A cross-domain small-sample target detection method based on dynamic information fusion, characterized in that, The steps are as follows: S1. Define the cross-domain small sample object detection task: Divide the dataset into source datasets containing sufficient labeled samples. and target dataset containing K-shot samples Clearly define the source dataset Data distribution PS and target dataset The data distribution is PT, and the test task adopts an N-way K-shot setting. Each task contains N categories, and r labeled instances are selected for each category as the support set S, and the remaining unlabeled samples are used as the query set Q. S2. Construct the DIC-ViT network model and extract basic features: The DIC-ViT network model includes a query branch and a support branch, which share the weights of the DINOv2 backbone network based on the ViT architecture. Given a query image q and r support images s for each category, where query image q belongs to query set Q and support images s belong to support set S, input query image q into the query branch and support images s into the support branch, and extract query features through the DINOv2 backbone network. and supporting features ; S3. Candidate anchor boxes are generated using the region proposal network built into the DIC-ViT network model, and then the sample partitioning is optimized by the region proposal re-screening module. The region proposal network is embedded in the candidate region generation process of the DIC-ViT network model, based on the query features extracted in S2. and supporting features The system generates several candidate anchor boxes that may contain the target by using a preset multi-scale and multi-aspect ratio anchor point mechanism; the region proposal re-screening module calculates the center vector of each category based on the support features extracted by S2, and divides positive and negative samples by combining the "IoU value between the anchor box and the real target box" and the "metric distance from the anchor box to the corresponding category center" as dual criteria, and dynamically updates the category center vector during the training process; S4. Feature adaptive fusion is achieved through the dynamic information coupling module: The dynamic information coupling module receives the query features output by S2. and supporting features In addition to the candidate region information generated by the region proposal network and filtered by the region proposal re-filtering module in S3, sparse local attention is first used to couple query feature information to focus on key regions, and then gated attention is used to support feature information fusion for dynamic weighting to generate customized query features. and coupled support features ; S5. Generate a discriminative prototype through the contrastive learning module, using the coupled support features output from S4. As input, the vector is first transformed into a compact discriminative vector through a feature embedding layer, and then a class prototype is generated through iterative optimization by a contrastive prototype generator. ; S6. Optimize the model parameters using an objective function L, the formula for which is: ； in, This indicates that the detection head is based on customized query features of S4. Category prototypes of S5 Calculated classification loss, Indicates that the detection head is based on The calculated regression loss, This represents the contrastive loss generated by the contrastive learning module of S5.

2. The cross-domain small-sample target detection method based on dynamic information fusion according to claim 1, characterized in that, S2 specifically includes the following sub-steps: S2.1 Construct a dual-branch network structure: The query branch and the support branch are symmetrical. The number of network layers, convolution kernel size, and number of channels of the query branch and the support branch are completely identical. The query branch is input with a single query image, and the support branch is input with k support images grouped by category in the support set S. S2.2 Determine the parameters of the DINOv2 backbone network: The DINOv2 backbone network is based on the ViT architecture, uses pre-trained weights for initialization, and the pre-training dataset is ImageNet-21K. During training, the parameters of the first 12 layers of the DINOv2 backbone network are frozen, and the parameters of the last 4 layers are fine-tuned. The feature output layer of the DINOv2 backbone network is selected as layer C4, the feature map resolution is 100×100, and the number of channels is 256. S2.3 Image Preprocessing and Feature Extraction: The input query image and support images undergo unified preprocessing, including resizing both and normalizing them according to mean and variance. The preprocessed query image is then input into the query branch, and after forward propagation through the DINOv2 backbone network, the C4 layer features are output, denoted as the query features. ; The preprocessed support images are grouped by category and input into the support branch. After forward propagation through the DINOv2 backbone network, the corresponding C4 layer features are output, which are denoted as support features. ; S2.4 Achieving Dual-Branch Weight Sharing: The DINOv2 backbone network parameters of the query branch and the support branch are fully bound. During training, only one set of DINOv2 backbone network parameters is updated. Both branches synchronously call the updated parameters for feature extraction, ensuring that the query features... and supporting features They exist in the same feature space.

3. The method for cross-domain small-sample target detection based on dynamic information fusion according to claim 2, characterized in that: S3 specifically includes the following sub-steps: S3.1: Region Proposal Network Generates Candidate Anchor Boxes: The region proposal network is a fully convolutional network structure, embedding the candidate region generation process of DIC-ViT, based on the output of S2. and supporting features The corresponding feature map with a resolution of 100×100 is used to generate 9 types of anchor boxes by pre-setting 3 scales and 3 aspect ratios of anchor points. The classification branch of the region proposal network predicts the confidence that each anchor box contains the target, and the regression branch predicts the position offset of the anchor box. By combining confidence filtering and position correction, a preset number of candidate anchor boxes are generated. S3.2, The region proposal re-selection module calculates the initial category center vector: For each category i in the support set S, extract the support features of all support images of that category after processing by S2. Global average pooling is performed on each support feature map to obtain a D-dimensional feature vector. The mean of the feature vectors of all support samples for that class is calculated, and this mean is used as the initial class center vector for class i. ; S3.3, the region proposal re-filtering module calculates the metric feature vector of the anchor box: For each candidate anchor box generated in S3.1, based on its coordinate position on the feature map, the query feature output from S2 is used. Or supporting features The features corresponding to the anchor frame are cropped out, compressed to D dimensions by 1×1 convolution, and then the metric feature vector of the anchor frame is obtained by global average pooling. ; S3.4, The region proposal re-screening module initially screens samples based on IoU values: setting an IoU threshold. Calculate the IoU value between each candidate anchor box and all ground truth bounding boxes in the image. IoU = intersection area of ​​anchor box and ground truth bounding box / union area. If the IoU value of a candidate anchor box with any ground truth bounding box is greater than 1, the anchor box is considered to have a lower IoU value than the ground truth bounding box. If the IoU value between a candidate anchor box and all ground truth target boxes is less than or equal to 1, it is included in the candidate positive sample pool; if the IoU value between a candidate anchor box and all ground truth target boxes is less than or equal to 1, it is included in the candidate positive sample pool. These are directly identified as negative samples and will not proceed to further screening. S3.5, The region proposal re-screening module filters samples a second time based on metric distance: For the j-th anchor box in the candidate positive sample pool obtained in S3.4, the metric value is calculated using the following formula: ； in, Indicates the first The anchor frame corresponds to the first The measure of each category, It is the first The metric feature vector of each anchor box. Indicates the number of categories, Indicates the number of anchor frames; Take all The minimum value in the value is used as the minimum metric value of the anchor frame. The calculation formula is: ； Set measurement threshold ,like The anchor frame is determined to be a positive sample, and... Corresponding category The label is assigned to the anchor box; if The anchor frame was re-evaluated as a negative sample. S3.6, The region proposal re-selection module updates the class center vector: After each training mini-batch, for each class included in the current mini-batch... Count the number of samples that are judged as positive by S3.5 in this category. Extract the metric feature vectors of these positive samples. The class center vector is updated using the following formula: ； in, It is the update step size, which is a constant between 0 and 1.

4. The cross-domain small sample target detection method based on dynamic information fusion according to claim 3, characterized in that: S4 specifically includes the following sub-steps: S4.1, Query Feature Information Coupling S4.1.1 Support Feature Region Division: The support features output from S2 are... Divide the local window into P×P steps and extract the features of the 1×1 region at the center of each window as a local query. S4.1.2, Query Feature Decomposition and Local Key Extraction: The query features output from S2 are... The vector is split into a key vector K and a value vector V. The K features of the corresponding region of each local window are extracted and obtained by average pooling. S4.1.3 Calculate the local similarity weights using the following formula: ； in, For window indexing, , Indicates the first Average pooling of queries and keys within each window This means retaining the A largest values; S4.1.4 Window-level Custom Feature Generation: Generate window features using the following formula: ； in, It is the projection matrix of the Value vector V, used to project the original query features. Mapped to a new feature space, Indicates the projection matrix After transformation, the first The re-encoded value vector of the query feature within a local window; S4.1.5, Customized Query Feature Generation: Concatenating All The obtained feature map is upsampled to 100×100 through 1×1 convolution and bilinear interpolation to generate the similarity weight SW, as shown in the following formula: ； Then, V is weighted and summed using similarity weights to generate customized query features. The calculation formula is: ； S4.2 Supports feature information fusion S4.2.1, Gating Weight Prediction: Predicting query features Global average pooling vector and support features The components are concatenated position by position, and then gated weights are generated using lightweight convolutional layers and activation functions. The formula is: ， S4.2.2, Hybrid Weight Generation: Calculating Query Features and supporting features cosine similarity Set adjustment parameters Generate the mixed weights using the following formula. : ； S4.2.3, Coupling Support Feature Generation: This involves generating the recombined query features. Support features are injected using a hybrid weighting method to generate coupled support features. The formula is as follows: 。 5. The cross-domain small-sample target detection method based on dynamic information fusion according to claim 4, characterized in that, S5 specifically includes the following sub-steps: S5.1: Feature embedding layer transforms discriminative vectors: Post-coupling support features of S4 output As input, a feature map is generated through a 1×1 convolution. The importance weights at each location are quantized using Sigmoid activation to obtain a spatial attention map. This attention map is then coupled with the resulting support features. Element-wise multiplication is performed for spatial weighting, and the weighted features are compressed into a spatially independent vector using global average pooling. The compressed vector is then dimensionality-reduced through a linear projection layer to generate a discriminative embedding vector e, expressed by the formula: ； in, It is the projection transformation matrix in the feature embedding layer. These are the 1×1 convolutional kernel weights of the spatial attention submodule in the feature embedding layer; S5.2: Initialize the prototype using the comparison prototype generator: for each category Determine its sample index set Calculate the initial prototype using the following formula: ； in, Indicate category The number of samples, Indicates the first Embedding vectors of each sample; S5.3: Iteratively Optimize Prototypes by Comparing Prototype Generators S5.3.1: Definition of positive and negative samples: For each iteration step , Each sample Positive sample pairs are ,in Indicates sample The category belongs to the iteration step The prototype; the negative sample set is defined as ,in Represents a heterogeneous prototype sampled randomly; S5.3.2: Comparison Loss Calculation: The calculation formula is as follows: ； ； in, For similarity, For temperature parameters, The total number of samples; S5.3.3: Prototype Gradient Update: Move the prototype along the negative gradient direction and update it according to the following formula to cluster similar prototypes and separate dissimilar prototypes. The formula is expressed as: ； in, Indicates the iteration step prototype vector, Indicates the optimization step size, the default is 0.1; Indicates the loss to the prototype The gradient operator; S5.4: Output the final category prototype: After completing T iterations, collect the final prototypes of all categories. The category prototypes are obtained by concatenating them in category order: ； in, This represents the total number of categories.

6. The cross-domain small-sample target detection method based on dynamic information fusion according to claim 1, characterized in that, The contrastive loss weights described in S6 The value is 0.5; the classification loss The cross-entropy loss is used to calculate the cross-entropy between the class probability output by the detection head and the true class label of the positive sample; the regression loss Using smoothed L1 loss, the coordinates of the target bounding box predicted by the detector are calculated. Smoothing L1 error between the actual target bounding box coordinates and the true target bounding box coordinates.

Images