High-performance unmanned aerial vehicle detection method based on visual cue words

Abstract

The invention discloses a high-performance unmanned aerial vehicle detection method based on a visual cue word, and relates to the technical field of unmanned aerial vehicle detection, and the method comprises the steps: collecting diversified images containing a target to form a training data set, collecting a few-sample cross-domain image as cross-domain information input, inputting the training image into a pre-trained and parameter-locked backbone network, and extracting basic features; a few-sample cross-domain image is input into a cross-domain network, after features are extracted, a learnable visual prompt vector is generated, noise is added, then the prompt vector and basic features are input into a neck module through a cross attention mechanism to be integrated to obtain fusion features, and finally a detection head is input to output a classification and position regression result of a target. By reducing the training cost and improving the cross-domain generalization performance of an unmanned aerial vehicle algorithm, an efficient solution is provided for low-resource cross-domain detection tasks such as industrial quality inspection and geographical remote sensing, and meanwhile, a solution is provided for training a high-performance unmanned aerial vehicle through extremely few training parameters.

Description

Technical Field

[0001] The present invention relates to the field of drone detection technology, and more particularly to a high-performance drone detection method based on visual cue words. Background Art

[0002] With the rapid development of unmanned aerial vehicle (UAV) remote sensing technology, its applications in agricultural monitoring, disaster assessment, urban planning, and other fields are becoming increasingly widespread. However, UAV remote sensing imagery is characterized by dense distribution of multi-scale targets (such as crops and buildings), complex background interference (such as clouds and shadows), and significant cross-domain data differences (e.g., across seasons, sensors, and geographic regions). These characteristics pose a severe challenge to the generalization of object detection algorithms. In particular, when target domain data is scarce (such as in post-disaster emergency monitoring scenarios) or significantly differs from the source domain distribution (such as cross-modality from visible light to infrared), the performance of traditional detection models declines dramatically, making it difficult to meet practical needs.

[0003] Cross-Domain Few-Shot Detection (CD-FSOD) has become a key research direction in this context. This detection task aims to process data from different domains in object detection while maintaining good detection performance even with small sample sizes. Unlike traditional few-shot object detection (FSOD), CD-FSOD not only has to deal with the small number of samples but also with the disparity in data distribution between the source and target domains, making it even more challenging. This type of detection differs from the scenario in FSOD, where base classes and novel classes share the same data domain, transforming previously labeled "unknown" objects into targets to be detected. Cross-domain detection, however, requires processing entirely new classes from different domains. Even with extensive prior knowledge, humans struggle to identify such vastly different classes, posing a significant challenge for CD-FSOD.

[0004] Among the existing methods, CD-FSOD introduces a distillation method to prevent the student model from forgetting during fine-tuning through a teacher model. In addition, CD-ViTO uses additional learnable instance features and reweights them on the target domain to guide the prediction of the detection head. Existing methods require a lot of pre-training, and support images from the source domain are required during pre-training. In addition, in order to prevent catastrophic forgetting, the model needs to be trained using both source domain and cross-domain images when transferred to the target domain. CD-ViTO expands from three datasets to six cross-domain datasets. In order to maintain the consistency of semantic predictions under domain perturbations, it introduces a loss function to ensure semantic consistency, while requiring the support branch to have strong feature fusion capabilities. However, this approach makes the support branch too heavy and lacks flexibility, resulting in performance degradation when migrating to other cross-domain datasets.

[0005] Therefore, how to make up for the shortcomings of existing cross-domain small sample detection methods, such as supporting branch redundancy and low computational efficiency, is an urgent problem that those skilled in the art need to solve. Summary of the Invention

[0006] In view of this, the present invention provides a high-performance drone detection method based on visual prompt words to solve the problems existing in the background technology.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A high-performance drone detection method based on visual cue words, including:

[0009] Collect diverse images containing the target to form a training dataset, and collect a small number of cross-domain images as cross-domain information input;

[0010] The training data set is input into the pre-trained backbone network, and the basic features are extracted using its learned feature representation capabilities, while the backbone network parameters are locked;

[0011] Input a few-sample cross-domain image into the cross-domain network, extract features and generate a learnable visual cue vector; and add noise to the learnable visual cue vector;

[0012] Through the cross-attention mechanism, the cross-domain learnable visual cue vector and the basic features in the pre-trained backbone network are simultaneously input into the neck module for integration to obtain the fused features;

[0013] The fused features are input into the detection head, and the classification and position regression results of the target are output.

[0014] Optionally, the cross-domain network adopts a dynamic feature update mechanism driven by visual cues, replacing traditional support branches with lightweight cue modules, and visual cues interact with basic features of the backbone network through learnable visual cue vectors;

[0015] During the feature update process, the input feature of the i-th layer is F i ∈R b×l×d , b is the batch size, l is the sequence length, d is the feature dimension, and the learnable visual cue vector is P i ∈R n×d , n is the prompt length, and the feature update formula is as follows:

[0016]

[0017] Δ=δMLP(P i )

[0018] F i+1 =F i +MLP(Δ+F i )

[0019] Among them, the feature F is calculated by normalizing the dot product i With prompt P i The semantic association weights of the target domain are captured; the MLP (Δ) is superimposed on the original features in the form of residuals to ensure that the hint injection process does not destroy the general representation ability of the backbone network pre-training; an independent hint module is embedded in each layer of the backbone network to refine the cross-domain feature response layer by layer.

[0020] Optionally, it also includes decoupling the learnable visual cue vector from the ObjectQuery, which is a set of learnable parameters used to locate the target and decode the bounding box; extracting visual cues from the last 9 layers of the backbone network {P i ,…,P layers}, spliced ​​along the channel dimension into a composite query:

[0021] Q=Concat([ P i,…,P layers ])∈R nk×d

[0022] Where k = 9 is the number of layers, n = 100 is the length of a single layer prompt, P layers The total number of layers representing the backbone dynamically generates target-aware queries through hierarchical hints to reduce redundant queries. In the decoder, the composite query and encoder features are cross-attended to explicitly associate the target position with the global context.

[0023] Optionally, the neck module includes an encoder and a decoder, the encoder encodes and compresses the features, and the decoder decodes and integrates the features.

[0024] Optionally, the adding of noise is specifically: adopting a semantically guided noise generation mechanism, the core of which is: modeling cross-domain noise as a high-frequency perturbation component of the target domain features relative to the source domain, and by binding the noise to the semantic space of visual cues, achieving adaptive alignment of the noise pattern with task requirements.

[0025] Optionally, the process of the semantically guided noise generation mechanism is as follows:

[0026] Given an initial visual cue matrix, it is first expanded to twice the length along the sample dimension to generate an extended cue. After expansion, the first n dimensions retain the original semantic information, and the last n dimensions are used to carry noise perturbations.

[0027] To simulate the discretization characteristics of cross-domain interference, a binary random signal is regenerated. This signal is randomly sampled with a uniform probability distribution, and its polarity is flipped to simulate the positive and negative offsets of the cross-domain characteristics.

[0028] Introduce the intensity coefficient that obeys the truncated normal distribution to control the noise amplitude;

[0029] Multiply the binary signal and the intensity coefficient element by element to generate the final noise matrix;

[0030] Synthetic noise is injected into the extended cue space in the form of residuals to achieve fine-grained fusion of noise and semantic cues. Positive cue: the original semantic body is retained, and the noise is used as a slight perturbation to enhance local invariance; negative cue: the noise is dominant, simulating the interference pattern of features not seen in the target domain.

[0031] Optionally, during the training phase, the feature distances between positive and negative examples are constrained by contrastive learning:

[0032]

[0033] Where α is the boundary threshold, forcing the model to distinguish semantically relevant cues from noise interference, thereby implicitly learning cross-domain invariant features.

[0034] Optionally, the noise enhancement hint NP is used as the decoder input to interact with the encoder features; in the Hungarian algorithm, a noise robustness measure is introduced:

[0035] C match =λ1C IoU +λ2C cls +λ3||NP-F enc ||2

[0036] Among them, λ1 is a weight coefficient used to balance C IoUContribution to the overall matching cost, C IoU represents the cost associated with the intersection-and-union ratio; C cls is the classification-related cost; λ3 is the weight coefficient of this noise robustness metric; NP is the meaning of the noise prompt; F enc Output features for the encoder; ||NP-F enc ||2 represents the Euclidean distance between the noise cue feature and the encoder output feature, which is used to measure the degree of difference between the two feature vectors.

[0037] Through the above technical solutions, it can be seen that compared with the existing technology, the present invention discloses a high-performance drone detection method based on visual cue words, collects a variety of images containing targets to form a training data set, collects a few-sample cross-domain images as cross-domain information input, inputs the training images into a pre-trained and parameter-locked backbone network to extract basic features, inputs the few-sample cross-domain images into the cross-domain network, generates a learnable visual cue vector after feature extraction and adds noise, and then integrates the cue vector and the basic features into the neck module through the cross-attention mechanism to obtain fused features, and finally inputs the detection head to output the classification and position regression results of the target. The present invention provides an efficient solution for low-resource cross-domain detection tasks such as industrial quality inspection and geographic remote sensing by reducing training costs and improving the cross-domain generalization performance of drone algorithms. At the same time, very few training parameters provide a solution for training high-performance drones. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 A schematic flow chart of the method provided by the present invention;

[0040] Figure 2a Visualization diagram of the baseline and DN-VP provided by the present invention on the ArTaxOr dataset;

[0041] Figure 2b Visualization diagram of the baseline and DN-VP provided by the present invention on the Clipart1k dataset;

[0042] Figure 2c Visualization of the baseline and DN-VP provided by the present invention on the DeepFish dataset;

[0043] Figure 2dVisualization of the baseline and DN-VP provided by the present invention on the NEU-DET dataset;

[0044] Figure 2e Visualization of the baseline and DN-VP provided by the present invention on the UODD dataset. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0046] The embodiment of the present invention discloses a high-performance drone detection method based on visual prompt words, such as Figure 1 Shown, including:

[0047] Collect diverse images containing the target to form a training dataset, and collect a small number of cross-domain images as cross-domain information input;

[0048] The training data set is input into the pre-trained backbone network, and the basic features are extracted using its learned feature representation capabilities, while the backbone network parameters are locked;

[0049] Input a few-sample cross-domain image into the cross-domain network, extract features and generate a learnable visual cue vector; and add noise to the learnable visual cue vector;

[0050] Through the cross-attention mechanism, the cross-domain learnable visual cue vector and the basic features in the pre-trained backbone network are simultaneously input into the neck module for integration to obtain the fused features;

[0051] The fused features are input into the detection head, and the classification and position regression results of the target are output.

[0052] In a specific embodiment, a new detector DN-VP is proposed. During the fine-tuning process, the detector freezes all backbone parameters to prevent catastrophic forgetting and only uses the detection head for prediction. DN-VP requires fewer prediction branches but can achieve better results. In order to address the inconsistency problem between the main branch and the support branch, DN-VP removes redundant support branches and introduces visual fine-tuning to replace the model's adaptation to cross-domain novel classes. On this basis, DN-VP also adds a noise detection head, removes the domain prompter, and does not need to input the classification detection head through reweighted instances to supervise the main branch for classification and regression prediction.

[0053] This implementation re-examines the requirements of cross-domain detection for the target domain, removing support branches, redundant loss functions, and prediction heads, enabling a unified detection process for both the source and target domains. By freezing the backbone parameters, visual cues are introduced as a method for adapting to the target domain, while noise is added to further improve the model's robustness, enabling the detector to remain stable under domain perturbations.

[0054] This embodiment addresses the problems of redundant support branches and low computational efficiency in existing cross-domain small sample detection methods, and proposes a dynamic feature update mechanism driven by visual prompts, replacing traditional support branches with lightweight prompt modules. Traditional methods such as CD-ViTO rely on support branches of multi-stage feature fusion, which leads to a surge in model complexity and limited generalization during cross-domain migration. In contrast, visual prompts dynamically inject target domain prior knowledge during the fine-tuning phase through the interaction of learnable parameters and backbone features, which not only avoids parameter expansion (only increases 0.5% of parameters) but also achieves semantic alignment of cross-domain features through the attention mechanism.

[0055] During the feature update process, assuming that the input feature of the i-th layer is F i ∈R b×l×d (b is the batch size, l is the sequence length, and d is the feature dimension), the learnable visual cue is P i ∈R n×d (n is the prompt length). The feature update formula is as follows:

[0056]

[0057] Δ=δMLP(P i )

[0058] F i+1 =F i +MLP(Δ+F i )

[0059] Among them, the feature F is calculated by normalizing the dot product i With prompt P i The semantic association weights of the target domain are captured (e.g., low-contrast objects in foggy scenes). The MLP (Δ) is superimposed on the original features in residual form to ensure that the cue injection process does not destroy the general representation capabilities of the pre-trained backbone network (e.g., underlying features such as edges and textures). Independent cue modules are embedded in each layer of the backbone network to refine cross-domain feature responses layer by layer. For example, shallow modules (e.g., ResNetBlock1) focus on enhancing illumination invariance, while deep modules (e.g., Block4) focus on target semantic alignment.

[0060] Object Query is a set of learnable parameters used to locate the object and decode the bounding box. This embodiment innovatively decouples the visual cue from the Object Query. Visual cues are extracted from the last 9 layers of the backbone network. i ,…,P layers}, spliced ​​along the channel dimension into a composite query:

[0061] Q=Concat([P i ,…,P layers ])∈R nk×d

[0062] Where k = 9 is the number of layers, n = 100 is the length of a single layer prompt, P layers Represents the total number of layers in the backbone. Compared to DINO's fixed 900 queries, this method dynamically generates object-aware queries through hierarchical cues, reducing redundant queries. In the decoder, the composite query and encoder features undergo cross-attention calculations, explicitly associating object locations (e.g., "vehicles in fog") with global context (e.g., road structure), alleviating the problem of missed object detection in cross-domain scenarios.

[0063] Denoising training with visual cues

[0064] To address the model overfitting and generalization deficiencies caused by inter-domain distribution differences in cross-domain small sample detection tasks, this embodiment proposes a noise-coupled visual cue enhancement strategy. Through structured noise injection and self-supervised denoising training, this strategy simulates feature perturbations in cross-domain scenarios and improves model robustness. The innovation of this embodiment lies in combining the noise generation process with semantic guidance from visual cues to construct a training framework from noise simulation and feature enhancement to contrast denoising.

[0065] Existing research shows that the essence of cross-domain distribution differences can be regarded as a nonlinear offset between the source domain and the target domain feature space, which often manifests as high-frequency noise interference (such as sudden changes in illumination, sensor noise). Traditional methods enhance robustness by injecting random noise into the detection box coordinates or classification confidence, but their noise patterns deviate significantly from the statistical characteristics of the true domain differences (such as long-tail distribution and spatial correlation). Inspired by this, this embodiment proposes a semantically guided noise generation mechanism, the core assumption of which is that cross-domain noise can be modeled as a high-frequency perturbation component of the target domain features relative to the source domain. By binding the noise to the semantic space of the visual cue, adaptive alignment of the noise pattern with the task requirements is achieved.

[0066] The process is given an initial visual cue matrix P∈R n×d , first expand along the sample dimension to twice the length, generating an expansion prompt

[0067] P expand=repeat(P,2,1)

[0068] This operation aims to reserve redundant space for noise injection to avoid excessive damage to the original hint information. After expansion, the first n dimensions retain the original semantic information, and the last n dimensions are used to carry noise perturbations.

[0069] In order to simulate the discrete characteristics of cross-domain interference (such as sensor quantization error), a binary random signal S∈{-1,1} is generated. 2n×d ,S∈[-1,1]:

[0070] S = randint_like(P expand ,low=0,high=2)×2.0-1.0.

[0071] The signal is randomly sampled with a uniform probability distribution, and its polarity is flipped (±1) to simulate the positive and negative shifts of cross-domain features.

[0072] Introduce the intensity coefficient R∈[0,2) that obeys the truncated normal distribution 2n×d ,R∈[0,2), control the noise amplitude:

[0073] R=rand_like(S),R[half:]+=1.0.

[0074] The first n dimensions correspond to the noise intensity of the original cue (R∈[0,1)), and the second n dimensions correspond to the enhanced noise (R∈[1,2)). This design makes the noise intensity gradually increase with the cue position, simulating the cumulative effect of cross-domain interference.

[0075] Multiply the binary signal and the intensity coefficient element by element to generate the final noise matrix Noisy∈[-2,-1)∪[1,2) 2n×d :

[0076] Noisy=S⊙R

[0077] where ⊙ represents the Hadamard product. This noise matrix has asymmetric, multi-peaked statistical characteristics, which is closer to the disturbance distribution in real cross-domain scenarios.

[0078] Inject synthetic noise into the extended hint space in the form of residual:

[0079] NP=P expand +Noisy⊙P expand ⊙C

[0080] where C∈R dThe learnable channel attention weights are used to dynamically adjust the noise sensitivity of each feature dimension. This operation realizes the fine-grained fusion of noise and semantic cues. The positive example prompt (the first n dimensions): the original semantic subject is retained, and the noise is used as a slight disturbance to enhance local invariance. The negative example prompt (the last n dimensions): the noise is dominant, simulating the interference mode of unseen features in the target domain.

[0081] In the training phase, the feature distance of the positive and negative example prompts is constrained by contrastive learning:

[0082]

[0083] where a is a boundary threshold that forces the model to distinguish between semantic-related prompts and noise interference, thereby implicitly learning cross-domain invariant features.

[0084] In the bipartite graph matching phase of the DETR framework, the noise-enhanced prompt NP is input to the decoder and interacts with the encoder features. In the Hungarian algorithm, a noise robustness measurement term is introduced:

[0085] C match = λ1C IoU + λ2C cls + λ3||NP-F enc ||2

[0086] where λ1 is a weight coefficient used to balance the contribution of C IoU in the overall matching cost. C IoU represents the cost related to the intersection over union, which is an index that measures the degree of overlap between two bounding boxes (e.g., predicted boxes and real boxes in object detection); C cls is the classification-related cost; λ3 is the weight coefficient of this noise robustness measurement term; NP is the meaning of the noise prompt; F enc is the encoder output feature, which improves the robustness of the matching process to domain shifts by minimizing the difference between the noise prompt and the encoded feature; ||NP-F enc ||2 represents the Euclidean distance (L2 distance) between the noise prompt feature and the encoder output feature, which is used to measure the difference between the two feature vectors. By minimizing this distance, the noise prompt and the encoded feature can be made closer, thereby improving the matching robustness in the presence of noise. Gradually reduce the noise intensity (R ← R · γ) in the later training stage to make the model transition from strong noise adaptation to fine feature alignment, balancing convergence stability and generalization accuracy.

[0087] This noise-added indicator simulates real-world cross-domain interference through structured noise generation, overcoming the blinding limitations of traditional random noise. Noise injection and denoising learning are completed end-to-end, requiring no additional pre-training or complex data augmentation pipelines. Through structured noise injection and self-supervised denoising training, it effectively addresses model overfitting and generalization deficiencies caused by inter-domain distribution differences in cross-domain small-sample detection tasks.

[0088] The following specific experiments are introduced to further illustrate this method

[0089] Experimental settings and dataset description

[0090] DN-VP was trained on the MSCOCO source domain and evaluated on six validation datasets from different domains. Three validation datasets, ArTaxOr, DIOR, and UODD, were proposed by Distill-cdfsod. These datasets cover insect recognition, drone imagery, and underwater object detection. Three additional validation datasets, Clipart1k, DeepFish, and NEU-DET, were proposed by CD-ViTO. Specifically, ArTaxOr has 7 classes containing 13,991 training images and 1,383 test images; UODD is an underwater object detection dataset, containing 3 classes, 3,194 training images and 506 test images; the aerial remote sensing dataset DIOR contains 20 classes, 18,463 training images and 5,000 test images; Clipart1k contains 20 categories, all of which are ink and cartoon images for weakly supervised cross-domain object detection, including training and test images covering 205 categories of scenes; DeepFish obtains 40,000 training and test images from formal waters; NEU-DET is an industrial defect detection dataset, containing 6 defect categories and a total of 1,800 defect images.

[0091] To ensure fair comparison, we used the same number of training and fine-tuning rounds as previous work on all datasets, with only 12 epochs trained on MSCOCO. Furthermore, our model implementation used an input resolution of 640×640, based on the mmdetection codebase, and no additional image augmentation was performed during training. We performed detection on each dataset with a small sample size of 1, 5, or 10, and reported the mean Average Precision (mAP) for each item. mAP@0.95 was used for this experiment.

[0092] To better evaluate the effectiveness of our method, we report the accuracy of these methods on the CD-FSOD dataset, following the fine-tuning settings of CD-ViTO. Several FSOD methods, including Meta-RCNN, TFA, FSCE, and DeFRCN, use Distill as their results. Object detection methods and open-set methods using ViT as their backbone include DE-ViT and Detic, where "FT" indicates that the detector was fine-tuned on the dataset, and their results are reported on CD-ViTO. Existing cross-domain methods for six different datasets include Distill, MoFSOD, and CD-ViTO. Finally, we report the results of DN-VP on the CD-FSOD dataset, as shown in Tables 1 and 2, respectively.

[0093] Comparative experiments on cross-domain object detection datasets

[0094] To systematically validate the generalization capabilities of our cross-domain few-shot object detection methods, we employed a two-stage evaluation framework: first, we trained the base model on the source domain MSCOCO dataset (12 training epochs), followed by validation of the transfer performance on three cross-domain detection benchmarks. Notably, the experimental design strictly adhered to the cross-domain few-shot detection protocol, which maintains complete category and scene isolation between the source domain (MSCOCO) and the target domain.

[0095] For target domain evaluation, three benchmark datasets covering typical cross-domain challenges were selected: ArTaxOr (cross-species insect classification, simulating biometric shift), DIOR (multi-target detection in remote sensing images, addressing scale variation and low-resolution challenges), and UODD (underwater open domain detection, addressing optical distortion and low-contrast interference). For each target domain, experiments included three small sample configurations: 1, 5, and 10-shot to comprehensively evaluate model performance. As shown in Table 1, the DN-VP method demonstrates excellent cross-domain adaptability. Its core advantages are reflected in two aspects: effectively capturing cross-domain invariant features through a noise coupling mechanism; and maintaining stable category discriminability with limited samples.

[0096] DN-VP achieves state-of-the-art performance on all three datasets in both 5-shot and 10-shot settings. Taking the most challenging UODD dataset as an example, DN-VP achieves mAPs of 18.0% and 24.6% in 5-shot and 10-shot detection, respectively, representing absolute performance improvements of 165.2% and 251.4% over the current best method, CD-ViTO (6.8% / 7.0%). This breakthrough demonstrates the robustness of our proposed noise-coupled visual cueing mechanism to complex domain shifts, such as underwater optical distortion and low-contrast objects.

[0097] Under limited sample conditions, DN-VP demonstrates exceptional cross-domain adaptability. When the number of annotated samples is increased to 5-shot, DN-VP achieves leading detection accuracy on the ArTaxOr, DIOR, and UODD benchmarks. Further increasing the number to 10-shot, its performance advantage continues to expand, demonstrating the method's ability to effectively utilize sample increments. Particularly noteworthy is that on the challenging UODD dataset (which simulates real underwater scenes), which contains complex domain shifts caused by light scattering and color distortion, DN-VP achieves mAP values ​​of 18.0% and 24.6% in the 5-shot and 10-shot configurations, respectively. This represents a 165.2% and 251.4% improvement over the state-of-the-art method CD-ViTO (6.8% / 7.0%). These results demonstrate that the proposed noise-coupled visual cueing mechanism effectively addresses the core challenge unique to underwater environments: the confusion of high-frequency features between low-contrast targets and backgrounds. This is achieved by dynamically injecting domain-dependent noise and subjecting the cue words to denoising training. This mechanism forces the model to focus on semantic structural features that are invariant across domains rather than domain-sensitive local texture information, thereby maintaining robustness under complex domain shift conditions.

[0098] Notably, in extremely low-sample (1-shot) scenarios, while DN-VP slightly lags behind CD-ViTO in absolute accuracy (17.1% vs. 21.0% mAP on the ArTaxOr dataset), a difference likely due to the explicit knowledge distillation properties of the ViTO architecture. However, its performance improves significantly with increasing sample size, surpassing traditional methods. For example, on the UODD dataset, which exhibits significant domain shift, DN-VP's mAP improves by 6.6% (18.0% → 24.6%) when the sample size increases from 1-shot to 10-shot, while DE-ViT-FT only improves by 0.4% (5.0% → 5.4%). This phenomenon suggests that under conditions of extreme data scarcity, models must compensate for insufficient prior knowledge through generative approaches. The proposed noise hint mechanism effectively overcomes the limitation of overfitting to the source domain by generating virtual samples in the feature space that match the target domain distribution. Compared to methods based on pure parameter fine-tuning (such as DE-ViT-FT), it demonstrates stronger cross-domain generalization potential.

[0099] In relatively simple cross-domain scenarios (such as the insect classification task of ArTaxOr, where domain shift is mainly reflected in the texture difference between specimens and wild individuals), DN-VP's semantic fidelity advantage is even more significant. When the sample size is increased to 10-shot, the method breaks the field record with 66.4% mAP, a 4.9% improvement over CD-ViTO's 60.5%. Through visual analysis, we found that Figures 2a-2eAs shown, the noise injection mechanism effectively suppresses the shift in category prototypes caused by domain differences by imposing constraints in the low-frequency semantic space. For example, the morphological characteristics of Coleoptera insects (elytra proportions and antennal structure) remain highly distinguishable across domains. This demonstrates the unique advantage of this method in balancing domain adaptation and category discriminability.

[0100] DN-VP demonstrates significant stability advantages in addressing the complex challenges unique to cross-modal tasks, such as multi-scale object detection in UAV remote sensing imagery and object detection with progressive optical distortion in underwater scenes. Experimental data shows that DN-VP's performance fluctuation is only 6.5% across the UODD (24.6% underwater 10-shot mAP) and DIOR (31.1% remote sensing 10-shot mAP) tasks, while the traditional method DeFRCN exhibits fluctuations of up to 18.4% under the same conditions. This is primarily due to its sensitivity to background complexity and prone to false detection in sparsely textured underwater scenes. Experiments demonstrate that adaptive noise injection suppresses domain-specific high-frequency interference, cloud artifacts, and underwater bubble noise in remote sensing imagery, effectively decoupling the coupling effect between object scale diversity and background complexity, breaking through the performance bottleneck of traditional single-stage adaptation methods.

[0101] Visualizing the feature distribution clearly demonstrates that DN-VP exhibits tighter intra-class clustering and sharper inter-class boundaries in target domains (e.g., UODD underwater scenes). This phenomenon strongly suggests that noise cues implicitly model high-frequency components that vary across domains, prompting the model to deeply learn essential domain-invariant features, such as object shape and topology, rather than simply relying on shallow texture information that is susceptible to domain variations. This learning of essential features makes the model more adaptable and robust across domains, enabling more accurate object recognition and classification.

[0102] Table 1 Cross-domain object detection 1 / 5 / 10-shot accuracy on ArTaxOr, DIOR and UODD datasets

[0103]

[0104] More generalized cross-domain object detection experiments

[0105] To further systematically evaluate the generalization capabilities of DN-VP in heterogeneous, cross-domain scenarios, experiments were extended to the supplementary benchmarks proposed by CD-ViTO: Clipart1k (cross-modal comics detection, which needs to address the style gap between hand-drawn lines and real objects); DeepFish (underwater microscopic biological detection, which needs to cope with the interference of tiny fish schools and turbid media); and NEU-DET (industrial metal surface defect detection, which requires sub-pixel texture recognition). These datasets cover three challenging scenarios: style transfer, complex media interference, and industrial precision inspection. They fully test the model's cross-modal, cross-scale, and cross-distribution generalization capabilities. The results are shown in Table 2.

[0106] Table 2 Cross-domain detection results of 1 / 5 / 10-shot accuracy of cross-domain object detection on Clipart1k, DIOR and NEU-DET datasets

[0107]

[0108]

[0109] In the one-shot weakly supervised detection task of Clipart1k, a cross-modal art domain, DN-VP significantly surpassed CD-ViTO (17.7%) with a mAP of 29.0%. Furthermore, when trained on a larger number of samples (5 / 10-shot), DN-VP achieved significant improvements of 7.6% (from 41.1% to 48.7%) and 4.5% (from 44.3% to 48.8%). In-depth analysis shows that DN-VP, through its noise coupling mechanism, effectively expands the decision boundary by generating adversarial virtual samples that address the unique abstract line expressions of comics (such as semantic ambiguity caused by simplified character outlines), reducing false positives and significantly suppressing false positives compared to the compared methods on average.

[0110] The 1-shot task of NEU-DET in the industrial quality inspection scenario exposes serious limitations of traditional methods: Distill and Detic-FT suffer model collapse due to sample scarcity (mAP ≤ 3.8%), while DN-VP still maintains a mAP of 3.9%, verifying the strong adaptability of noise perturbation to annotation sparsity; when the sample size is increased to 5-shot, although DN-VP is not the highest method, it is 4.1% higher than CD-ViTO and is close to the highest accuracy; when the sample size is increased to 10-shot, DN-VP approaches the current optimal method Distill (21.1%) with a mAP of 19.2%, indicating that its hierarchical decoupling mechanism can effectively separate micron-level defect features (such as crack depth gradient) from industrial background noise (such as metal oxidation artifacts), achieving a balance between accuracy and efficiency while maintaining a lightweight architecture.

[0111] On the challenging DeepFish underwater detection dataset (average target size 32×32 pixels), DN-VP achieved 38.3% and 38.6% mAP in the 5-shot and 10-shot settings, respectively, representing absolute improvements of 16.0 and 16.3 percentage points over CD-ViTO (22.3% and 22.3%). On the minimal 1-shot metric, DN-VP achieved an accuracy of 26.7%, significantly outperforming existing methods (6.4%). Feature visualization revealed that the visual cue module enhances low-resolution target recognition by intensifying the spectral response of coral fish textures, while noise modeling effectively compensates for the loss of high-frequency information caused by underwater scattering.

[0112] Table 2 comprehensively evaluates the average mAP across three new cross-domain datasets, showing that DN-VP leads CD-ViTO (21.8%) by 8.1% at 29.9%. More notably, during cross-domain task switching (e.g., from Clipart1k to NEU-DET), DN-VP's performance fluctuation (25.1%) is significantly lower than DE-ViT-FT (32.2%), demonstrating the universal robustness of its hierarchical adaptation mechanism to domain shift heterogeneity. This feature is crucial for practical deployments, as real-world scenarios often require models to rapidly switch between different detection domains.

[0113] To address the challenges of high model training costs and limited generalization capabilities in cross-domain object detection, this paper proposes a lightweight transfer learning framework based on denoised visual cues (DN-VP). By constructing a fine-tuning architecture and freezing the backbone network parameters, a learnable visual cue module is designed to achieve cross-domain feature adaptation: 1) A denoising training mechanism adds positive and negative sample noise to the visual cue words, and comparative training suppresses noise interference; 2) A hierarchical feature fusion strategy is adopted to integrate shallow visual cues with deep semantic cues at multiple scales and input them into the detection head. Experiments on cross-domain datasets such as DIOR and DeepFish show that DN-VP only needs to fine-tune 2% of the parameters to achieve an average detection accuracy of 29.9%, with the performance of drone scenes (DIOR) reaching 31.1%. By reducing training costs and improving the cross-domain generalization performance of drone algorithms, this paper provides an efficient solution for low-resource cross-domain detection tasks such as industrial quality inspection and geographic remote sensing. At the same time, it provides a solution for training high-performance drones with very few training parameters.

[0114] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0115] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A high-performance drone detection method based on visual cue words, characterized by: include: Collect diverse images containing the target to form a training dataset, and collect a small number of cross-domain images as cross-domain information input; The training data set is input into the pre-trained backbone network, and the basic features are extracted using its learned feature representation capabilities, while the backbone network parameters are locked; Input a few-sample cross-domain image into the cross-domain network, extract features and generate a learnable visual cue vector; and add noise to the learnable visual cue vector; Through the cross-attention mechanism, the cross-domain learnable visual cue vector and the basic features in the pre-trained backbone network are simultaneously input into the neck module for integration to obtain the fused features; The fused features are input into the detection head, and the classification and position regression results of the target are output.

2. A high-performance drone detection method based on visual prompt words according to claim 1, characterized in that: The cross-domain network adopts a dynamic feature update mechanism driven by visual cues, replacing the traditional support branch with a lightweight cue module. The visual cues interact with the basic features of the backbone network through a learnable visual cue vector. During the feature update process, the input feature of the i-th layer is F i ∈R b×l×d , b is the batch size, l is the sequence length, d is the feature dimension, and the learnable visual cue vector is P i ∈R n×d , n is the prompt length, and the feature update formula is as follows: Δ=δMLP(P i ) F i+1 =F i +MLP(Δ+F i ) Among them, the feature F is calculated by normalizing the dot product i With prompt P i The semantic association weights of the target domain are captured; the MLP (Δ) is superimposed on the original features in the form of residuals to ensure that the hint injection process does not destroy the general representation ability of the backbone network pre-training; an independent hint module is embedded in each layer of the backbone network to refine the cross-domain feature response layer by layer.

3. A high-performance drone detection method based on visual prompt words according to claim 2, characterized in that: It also includes the decoupling design of the learnable visual cue vector from the Object Query, which is a set of learnable parameters for locating the target and decoding the bounding box; extracting visual cues from the last 9 layers of the backbone network {P i ,…,P layers }, spliced ​​along the channel dimension into a composite query: Q=Concat([P i ,…,P layers ])∈R nk×d Where k = 9 is the number of layers, n = 100 is the length of a single layer prompt, P layers The total number of layers representing the backbone dynamically generates target-aware queries through hierarchical hints to reduce redundant queries. In the decoder, the composite query and encoder features are cross-attended to explicitly associate the target position with the global context.

4. The high-performance drone detection method based on visual prompt words according to claim 1 is characterized in that: The neck module includes an encoder and a decoder. The encoder encodes and compresses features, while the decoder decodes and integrates features.

5. The high-performance drone detection method based on visual prompt words according to claim 1 is characterized in that: The added noise specifically adopts a semantically guided noise generation mechanism, the core of which is: cross-domain noise is modeled as a high-frequency perturbation component of the target domain features relative to the source domain, and by binding the noise to the semantic space of visual cues, the adaptive alignment of the noise pattern with the task requirements is achieved.

6. A high-performance drone detection method based on visual prompt words according to claim 5, characterized in that: The process of the semantically guided noise generation mechanism is as follows: Given an initial visual cue matrix, it is first expanded to twice the length along the sample dimension to generate an extended cue. After expansion, the first n dimensions retain the original semantic information, and the last n dimensions are used to carry noise perturbations. To simulate the discretization characteristics of cross-domain interference, a binary random signal is regenerated. This signal is randomly sampled with a uniform probability distribution, and its polarity is flipped to simulate the positive and negative offsets of the cross-domain characteristics. Introduce the intensity coefficient that obeys the truncated normal distribution to control the noise amplitude; Multiply the binary signal and the intensity coefficient element by element to generate the final noise matrix; Synthetic noise is injected into the extended cue space in the form of residuals to achieve fine-grained fusion of noise and semantic cues. Positive cue: the original semantic body is retained, and the noise is used as a slight perturbation to enhance local invariance; negative cue: the noise is dominant, simulating the interference pattern of features not seen in the target domain.

7. A high-performance drone detection method based on visual prompt words according to claim 6, characterized in that: During the training phase, the feature distances of positive and negative examples are constrained by contrastive learning: Where α is the boundary threshold, forcing the model to distinguish semantically relevant cues from noise interference, thereby implicitly learning cross-domain invariant features.

8. The high-performance drone detection method based on visual prompt words according to claim 4 is characterized in that: The noise enhancement hint NP is used as the decoder input to interact with the encoder features; in the Hungarian algorithm, a noise robustness measure is introduced: C match =λ1C IoU +λ2C cls +λ3||NP-F enc ||2 Among them, λ1 is a weight coefficient used to balance C IoU Contribution to the overall matching cost, C IoU represents the cost associated with the intersection-and-union ratio; C cls is the classification-related cost; λ3 is the weight coefficient of this noise robustness metric; NP is the meaning of the noise prompt; F enc Output features for the encoder; ||NP-F enc ||2 represents the Euclidean distance between the noise cue feature and the encoder output feature, which is used to measure the degree of difference between the two feature vectors.

Images