Anti-unmanned aerial vehicle tracking method and device based on multi-modal diffusion feature enhancement and entropy-oriented adaptive sampling
The anti-drone tracking method, which uses a diffusion model and entropy-guided adaptive search region adjustment, solves the problems of feature loss at night and under smoke cover, as well as the easy escape of small targets from the field of view, and achieves high-precision multimodal information fusion and target tracking.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-02-05
- Publication Date
- 2026-05-12
AI Technical Summary
Existing anti-drone tracking methods fail at night or under smoke cover, lose high-frequency information in feature extraction, are prone to small targets escaping the field of view, and have low regression accuracy. Multimodal fusion leads to mutual exclusion of features, and existing local search trackers rely on a single confidence level and do not consider environmental imaging quality.
A two-stream feature extraction network based on a diffusion model is adopted, combined with entropy-guided adaptive search region adjustment and dense attention masking module. Through multimodal feature enhancement and information entropy evaluation, the target location is optimized using a coverage-type minimum point distance intersection-union loss function.
It improves the feature recognition of micro-drones, reduces the target loss rate, enhances environmental robustness and small target regression accuracy, solves the problem of multimodal information imbalance, and avoids feature mutual exclusion.
Smart Images
Figure CN122024104A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of computer vision, deep learning and optoelectronic signal processing technology, and relates to an anti-drone tracking method and device based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling. Background Technology
[0002] With the rapid development of the low-altitude economy, micro unmanned aerial vehicles (UAVs) are increasingly used in aerial photography, logistics, and other fields, but this has also brought security risks such as unauthorized flights disrupting air traffic and privacy theft. Anti-UAV tracking technology, as a core component of countermeasure systems, is responsible for continuously locking onto the target's location in complex and dynamic scenarios.
[0003] Existing anti-drone tracking methods face four main challenges. First, there is the limitation of single-modality processing: visible light images become ineffective at night or under smoke cover, and infrared images lack texture details. Although existing multimodal fusion methods attempt to combine the two, the significant modal heterogeneity between different modalities often leads to mutually exclusive features, or even a "1+1<1" effect. Second, there is the problem of losing high-frequency information during feature extraction. Traditional convolutional neural networks (CNNs) or Transformers are prone to losing edge and texture details of tiny drones during downsampling. While diffusion models possess powerful generation and feature representation capabilities, they have not yet been effectively applied to feature extraction for real-time tracking tasks.
[0004] Furthermore, there is the problem of small targets easily escaping the field of view. Existing local search trackers (such as FocusTrack) often allow targets to move out of the fixed search area when they undergo violent movement or camera shake. Although FocusTrack proposes adaptive search region adjustment (SRA), it relies solely on a single classification confidence score and does not consider the interference of environmental imaging quality (such as modal information imbalance) on the confidence score. Finally, there is the problem of low regression accuracy for small targets. UAV targets occupy a very small proportion of the image, and traditional IoU loss functions are not very sensitive to the loss value when the target shifts slightly. Moreover, existing improved versions (such as MPDIoU) often fail to apply the penalty term when dealing with extremely small targets because the denominator is usually the entire image size. Summary of the Invention
[0005] In view of this, the purpose of the present invention is to provide an anti-drone tracking method and apparatus based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling.
[0006] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides an anti-drone tracking method based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling, comprising the following steps: S1: Acquire the infrared and visible light synchronized video stream of the anti-drone system, and initialize the target template frame and the current search frame; S2: Construct a dual-stream feature extraction network based on a diffusion model, input infrared and visible light images into the diffusion denoising branch respectively, extract multi-scale latent feature representations, and perform feature decoupling and detail preservation through a dense reversible neural network module; S3: Utilize an entropy-guided adaptive search region adjustment strategy to calculate the conditional information entropy of dual-modal features to evaluate modal reliability, and dynamically adjust the expansion factor of the search region in conjunction with the target existence probability to ensure that the target is always within the field of view; S4: Construct a dense attention mask module, fuse multimodal features and generate a target segmentation mask, and guide feature focusing through the mask; S5: Utilize the prediction head to output the target classification score map and bounding box regression parameters, and use the coverage-type minimum point distance intersection-union loss function to optimize the network, outputting the final target location.
[0007] Furthermore, in step S2, the two-stream feature extraction network based on the diffusion model includes a forward diffusion process and a reverse feature extraction process: Forward process on input image Gaussian noise is added gradually at time step Noisy images Represented as:
[0008] in, Standard Gaussian noise, This is the cumulative noise variance coefficient; The reverse process utilizes a denoising network. Predict noise and extract latent features The feature extraction formula is defined as follows:
[0009] in, This represents the feature mapping of the intermediate layer in the denoising network. This is for channel splicing operations.
[0010] Furthermore, in step S2, the dense reversible neural network module adopts an affine coupling layer structure to input features. Split into two parts Output The calculation is as follows:
[0011]
[0012] in, is the transformation function for any convolutional neural network.
[0013] Furthermore, in step S3, the specific calculation process of the entropy-guided adaptive search region adjustment strategy is as follows: First, the conditional information entropy of each modality is estimated using the classification prediction head. Quantization mode Uncertainty, among which Represents an infrared image. Representing a visible light image:
[0014] in, For modality Features For category labels, The probability distribution output by the classification head. For modality The i 1 eigenvector N For modality The total number of eigenvectors; Next, the probability of the target existing after fusion is calculated. And combine modal uncertainty to dynamically calculate search factors :
[0015] in, As the current search factor, Step size, The average normalized information entropy for the two modes, For entropy weighting coefficients, The preset maximum search factor threshold, The confidence threshold. Used as the benchmark search factor.
[0016] Furthermore, in step S4, the dense attention masking module uses a cross-attention mechanism to fuse infrared and visible light features to generate fused features. :
[0017]
[0018]
[0019] in Indicates infrared characteristics, Representing visible light image features, Let Q represent a linear transformation, K represent the query vector, K represent the key vector, and V represent the value vector. The dimension of the key is represented by T, the matrix transpose is represented by Softmax, the normalized exponential function is represented by Concat, the concatenation operation is represented by Conv, and the convolution operation is represented by Conv. Then, a spatial mask is generated using Sigmoid activation. Used for weighted fusion features:
[0020]
[0021] in, This indicates element-wise multiplication.
[0022] Furthermore, in step S5, the definition of the coverage-type minimum point distance intersection-union ratio loss function is as follows: For small unmanned aerial vehicle (UAV) targets, a distance penalty term based on the minimum bounding rectangle is constructed, and the loss function is... The calculation formula is:
[0023] in, The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. and These are the Euclidean distances between the top-left and bottom-right corners of the predicted bounding box and the ground truth bounding box, respectively. and These are the width and height of the smallest rectangle that can tightly enclose the predicted bounding box and the ground truth bounding box, respectively. Total loss function It consists of classification loss, c-MPDIoU regression loss, and mask loss:
[0024] in To classify the focus loss, The minimum point distance intersection-union ratio regression loss is used for coverage. For mask loss, , , These are the weight coefficients for classification loss, c-MPDIoU regression loss, and mask loss, respectively.
[0025] In a second aspect, the present invention provides an apparatus comprising an image acquisition module, a processor, an input device, an output device, and a memory, wherein the image acquisition module, the processor, the input device, the output device, and the memory are interconnected, wherein the image acquisition module is used to acquire synchronized infrared and visible light video streams, the memory is used to store a computer program, the computer program comprising program instructions, and the processor is configured to invoke the program instructions to execute the step instructions as described in the first aspect of the present invention.
[0026] Thirdly, the present invention provides a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of the present invention.
[0027] Fourthly, the present invention provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of the present invention. The computer program product may be a software installation package.
[0028] The beneficial effects of this invention are as follows: By introducing a diffusion model combined with a reversible neural network as a feature extractor, this invention breaks through the bottleneck of small target feature extraction and improves the feature recognition of micro-UAVs; by adopting the E-SRA strategy to introduce information entropy to measure environmental quality, it achieves robust search for environmental perception and reduces the target loss rate; further, by adopting the c-MPDIoU loss function, it solves the problem of the extremely small proportion of small targets and improves the regression accuracy of micro-targets; at the same time, through information entropy evaluation and dense attention masking mechanism, it effectively solves the problem of multimodal information imbalance and avoids feature mutual exclusion.
[0029] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description
[0030] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the overall process of the method provided by the present invention; Figure 2 This is a structural diagram of the feature extraction module based on the diffusion model and dense reversible neural network in this invention; Figure 3This is a logical schematic diagram of the entropy-guided adaptive search region adjustment strategy in this invention. Detailed Implementation
[0031] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0032] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0033] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.
[0034] Example 1: like Figure 1 As shown, this embodiment provides an anti-drone tracking method and apparatus based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling, characterized by the following steps: S1: Acquire the synchronized infrared (IR) and visible light (VIS) video streams of the anti-drone system and initialize the target template frame. and the current search frame ; S2: Construct a dual-stream feature extraction network based on a diffusion model, input infrared and visible light images into the diffusion denoising branch respectively, extract multi-scale latent feature representations, and perform feature decoupling and detail preservation through a dense reversible neural network module (DIB); Optionally, the feature extraction process based on the diffusion model includes a forward diffusion process and a backward feature extraction process: Forward process on input image Gaussian noise is added gradually at time step Noisy images Represented as:
[0035] in, Standard Gaussian noise, This is the cumulative noise variance coefficient; The reverse process utilizes a denoising network. Predict noise and extract latent features The feature extraction formula is defined as follows:
[0036] in, This represents the feature mapping of the intermediate layer in the denoising network. This is for channel splicing operations. Figure 2 This is a structural diagram of the feature extraction module based on the diffusion model and dense reversible neural network.
[0037] Optionally, the Dense Reversible Neural Network (DIB) module employs an affine coupling layer structure to integrate input features. Split into two parts Output The calculation is as follows:
[0038]
[0039] in, For any convolutional neural network transformation function, this structure ensures lossless information transfer during feature extraction.
[0040] S3: Utilizing an entropy-guided adaptive search region adjustment (E-SRA) strategy, the conditional information entropy of the bimodal features is calculated to evaluate modal reliability, and the expansion factor of the search region is dynamically adjusted based on the target existence probability. To ensure that the target remains within the field of view at all times; Optionally, the specific calculation process of the entropy-guided adaptive search region adjustment (E-SRA) strategy is as follows: First, the conditional information entropy of each modality is estimated using the classification prediction head. Quantization mode Uncertainty:
[0041] in, For modality Features For category labels, The probability distribution output by the classification head. For modality The i 1 eigenvector NFor modality The total number of eigenvectors; Next, the probability of the target existing after fusion is calculated. And combine modal uncertainty to dynamically calculate search factors :
[0042] in, As the current search factor, Step size, The average normalized information entropy for the two modes, For entropy weighting coefficients, The confidence threshold. The preset maximum search factor threshold, Used as the benchmark search factor. For example... Figure 3 A logical diagram of the entropy-guided adaptive search region adjustment (E-SRA) strategy.
[0043] S4: Construct a Dense Attention Mask (DAM) module, fuse multimodal features and generate a target segmentation mask, and guide feature focusing through the mask; Optionally, the Dense Attention Mask (DAM) module employs a cross-attention mechanism to fuse infrared and visible light features to generate fused features. :
[0044]
[0045]
[0046] in Indicates infrared characteristics, Representing visible light image features, Let Q represent a linear transformation, K represent the query vector, K represent the key vector, and V represent the value vector. The dimension of the key is represented by T, the matrix transpose is represented by Softmax, the normalized exponential function is represented by Concat, the concatenation operation is represented by Conv, and the convolution operation is represented by Conv. Then, a spatial mask is generated using Sigmoid activation. Used for weighted fusion features:
[0047]
[0048] in, This indicates element-wise multiplication.
[0049] S5: Utilize the target classification score map and bounding box regression parameters output by the prediction head, optimize the network using the covering minimum point distance intersection-union ratio (c-MPDIoU) loss function, and output the final target location.
[0050] Optionally, in step S5, the definition of the coverage-type minimum point distance intersection-union ratio (c-MPDIoU) loss function is as follows: For small unmanned aerial vehicle (UAV) targets, a distance penalty term based on the minimum bounding rectangle is constructed, and the loss function is... The calculation formula is:
[0051] in, The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. and These are the Euclidean distances between the top-left and bottom-right corners of the predicted bounding box and the ground truth bounding box, respectively. and These are the width and height of the smallest rectangle that can tightly enclose the predicted bounding box and the ground truth bounding box, respectively. Total loss function It consists of classification loss, c-MPDIoU regression loss, and mask loss:
[0052] in To classify the focus loss, The minimum point distance intersection-union ratio regression loss is used for coverage. For mask loss, , , These are the weight coefficients for classification loss, c-MPDIoU regression loss, and mask loss, respectively.
[0053] Example 2: An electronic device, including a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method described in Embodiment 1 when executing the computer program.
[0054] Example 3: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in Embodiment 1.
[0055] Example 4: A computer program product includes a computer program that, when executed by a processor, implements the method described in Example 1.
[0056] In the above embodiments, the reference to "this embodiment" in the specification indicates that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple appearances of "this embodiment" do not necessarily refer to the same embodiment.
[0057] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.
[0058] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0059] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.
[0060] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.
[0061] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0062] This invention can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.
[0063] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A method for anti-UAV tracking based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling, characterized in that: Includes the following steps: S1: Acquire the infrared and visible light synchronized video stream of the anti-drone system, and initialize the target template frame and the current search frame; S2: Construct a dual-stream feature extraction network based on a diffusion model, input infrared and visible light images into the diffusion denoising branch respectively, extract multi-scale latent feature representations, and perform feature decoupling and detail preservation through a dense reversible neural network module; S3: Utilize an entropy-guided adaptive search region adjustment strategy to calculate the conditional information entropy of dual-modal features to evaluate modal reliability, and dynamically adjust the expansion factor of the search region in conjunction with the target existence probability to ensure that the target is always within the field of view; S4: Construct a dense attention mask module, fuse multimodal features and generate a target segmentation mask, and guide feature focusing through the mask; S5: Utilize the prediction head to output the target classification score map and bounding box regression parameters, and use the coverage-type minimum point distance intersection-union loss function to optimize the network, outputting the final target location.
2. The anti-UAV tracking method based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling according to claim 1, characterized in that: In step S2, the two-stream feature extraction network based on the diffusion model includes a forward diffusion process and a backward feature extraction process: Forward process on input image Gaussian noise is added gradually at time step Noisy images Represented as: in, Standard Gaussian noise, This is the cumulative noise variance coefficient; The reverse process utilizes a denoising network. Predict noise and extract latent features The feature extraction formula is defined as follows: in, This represents the feature mapping of the intermediate layer in the denoising network. This is for channel splicing operations.
3. The anti-UAV tracking method based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling according to claim 1, characterized in that: In step S2, the dense reversible neural network module adopts an affine coupling layer structure to process the input features. Split into two parts Output The calculation is as follows: in, is the transformation function for any convolutional neural network.
4. The anti-UAV tracking method based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling according to claim 1, characterized in that: In step S3, the specific calculation process of the entropy-guided adaptive search region adjustment strategy is as follows: First, the conditional information entropy of each modality is estimated using the classification prediction head. Quantization mode Uncertainty, among which Represents an infrared image. Representing a visible light image: in, For modality Features For category labels, The probability distribution output by the classification head. For modality The i 1 eigenvector N For modality The total number of eigenvectors; Next, the probability of the target existing after fusion is calculated. And combine modal uncertainty to dynamically calculate search factors : in, As the current search factor, Step size, The average normalized information entropy for the two modes, For entropy weighting coefficients, The preset maximum search factor threshold, The confidence threshold. Used as the benchmark search factor.
5. The anti-UAV tracking method based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling according to claim 1, characterized in that: In step S4, the dense attention masking module uses a cross-attention mechanism to fuse infrared and visible light features to generate fused features. : in Indicates infrared characteristics, Representing visible light image features, Let Q represent a linear transformation, K represent the query vector, K represent the key vector, and V represent the value vector. The dimension of the key is represented by T, the matrix transpose is represented by Softmax, the normalized exponential function is represented by Concat, the concatenation operation is represented by Conv, and the convolution operation is represented by Conv. Then, a spatial mask is generated through Sigmoid activation. Used for weighted fusion features: in, This indicates element-wise multiplication.
6. The anti-UAV tracking method based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling according to claim 1, characterized in that: In step S5, the definition of the coverage-type minimum point distance intersection-union ratio loss function is as follows: For small unmanned aerial vehicle (UAV) targets, a distance penalty term based on the minimum bounding rectangle is constructed, and the loss function is... The calculation formula is: in, The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. and These are the Euclidean distances between the top-left and bottom-right corners of the predicted bounding box and the ground truth bounding box, respectively. and These are the width and height of the smallest rectangle that can tightly enclose the predicted bounding box and the ground truth bounding box, respectively. Total loss function It consists of classification loss, c-MPDIoU regression loss, and mask loss: in To classify the focus loss, The minimum point distance intersection-union ratio regression loss is used for coverage. For mask loss, , , These are the weight coefficients for classification loss, c-MPDIoU regression loss, and mask loss, respectively.
7. An apparatus, characterized in that: The system includes an image acquisition module, a processor, an input device, an output device, and a memory, which are interconnected. The image acquisition module is used to acquire synchronized infrared and visible light video streams. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions to execute the anti-drone tracking method based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling as described in any one of claims 1-6.
8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the anti-drone tracking method based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling as described in any one of claims 1-6.
9. A computer program product, characterized in that: The method includes a computer program that, when executed by a processor, implements the anti-drone tracking method based on multimodal diffusion feature enhancement and entropy-guided adaptive sampling as described in any one of claims 1-6.