An unmanned aerial vehicle infrared small target detection method and system based on key information enhancement and cross-level feature progressive refinement

Abstract

The application discloses an unmanned aerial vehicle infrared small target detection method and system based on key information enhancement and cross-layer feature progressive refinement, and the method comprises the following steps: step one, collecting a training image dataset; step two, inputting the training image dataset into a preset infrared small target detection model to obtain a detection result; the infrared small target detection model enhancement comprises a backbone, an encoder and a decoder; the backbone comprises a plurality of sequentially connected multi-path dynamic weighted attention modules; step three, training and optimizing the infrared small target detection model enhancement according to a loss function to obtain an optimal infrared small target detection model enhancement; and step four, inputting a target image into the optimal infrared small target detection model enhancement to obtain a final detection result. The infrared small target detection model enhancement in the application enhances target key information, and effectively improves the detection performance of the infrared small target.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a method and system for detecting small infrared targets on unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features. Background Technology

[0002] Infrared small target detection (ISTD) refers to the process of identifying targets, typically only a few to a dozen pixels in size, within an infrared image against a complex background. Thanks to infrared imaging technology's ability to penetrate atmospheric disturbances such as fog and smoke, and to ignore the effects of day and night variations, it has wide and significant applications in fields such as security monitoring, remote sensing, military reconnaissance, traffic control, agricultural detection, underwater target identification, and infrared search and track systems. However, limited by infrared imaging hardware and the characteristics of long-range imaging, infrared images generally suffer from low resolution, small temperature differences between the target and background, and weak contrast. These inherent characteristics make the development of ISTD algorithms that simultaneously achieve high detection rates and low computational complexity particularly difficult, making it a highly challenging research direction in the field of computer vision.

[0003] Traditional methods rely on manually designed features and complex hyperparameter tuning. While effective in some simple scenarios, they struggle with complex background variations, suffer from high computational complexity, and exhibit poor versatility and robustness. Existing deep learning methods have become a research hotspot due to their powerful feature learning and generalization capabilities. Although CNN-based methods have made significant progress in ISTD, their inherent structural limitations still restrict further improvements in detection accuracy. Summary of the Invention

[0004] To address the low accuracy of existing models and algorithms in detecting small targets, this invention provides a method and system for detecting small infrared targets on UAVs based on key information enhancement and progressive cross-level feature refinement. This invention significantly enhances the key information of shallow features through a key information enhancement and fusion module; it constructs a progressive cross-level feature refinement module, which effectively improves the contour preservation ability and detection robustness of small targets by iteratively fusing shallow edge features and deep semantic features, and implements a hierarchical feature refinement mechanism: mid-level features are guided by an attention-based refinement module to focus on the target region, while deep features are refined by a context-aware module to capture remote contextual information; both retain the original features through residual connections. The backbone multi-path dynamic weighted attention module uses a multi-path structure to provide richer feature representations and introduces a dimensionality reduction-up hierarchical feature transformation mechanism to further enhance the model's feature expression capabilities. The infrared small target detection model in this invention enhances the key information of the target, effectively improving the detection performance of infrared small targets.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] The first aspect of this invention proposes a method for detecting small infrared targets on unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features, comprising:

[0007] Step 1: Collect the training image dataset;

[0008] Step 2: Input the training image dataset into the preset infrared small target detection model to obtain the detection results; the infrared small target detection model enhancement includes a backbone, encoder, and decoder; the backbone includes multiple sequentially connected multi-path dynamic weighted attention modules, each including a first path, a second path, a third path, and a fusion layer, with the second path including two parallel branches; the multi-path dynamic weighted attention module can provide richer feature representations;

[0009] Step 3: Train and optimize the infrared small target detection model enhancement based on the loss function to obtain the optimal infrared small target detection model enhancement, which facilitates the improvement of detection accuracy;

[0010] Step 4: Input the target image into the optimal infrared small target detection model enhancement to obtain the final detection result.

[0011] Furthermore, the multi-path dynamic weighted attention module is represented by the following formula:

[0012]

[0013] In the formula,

[0014]

[0015]

[0016]

[0017]

[0018]

[0019]

[0020] in, This is the output of the multi-path dynamic weighted attention module. This is the output of the first path. This is the output of the second path. For the output of the third path, For standard normalized convolution, The features are obtained by applying an activation function to a standard normalized convolution. Let X be the activation function and X be the input feature. and These are the outputs of the two parallel branches, where SA and CA represent the spatial attention mechanism and the channel attention mechanism, respectively. This is a depthwise separable convolution, AP is average pooling, and BN is batch normalization. For convolutional features, To add element by element, This is for element-wise multiplication.

[0021] Furthermore, the encoder includes a key information enhancement and fusion module, a cross-level feature progressive refinement module, and a hierarchical feature refinement module;

[0022] The key information enhancement and fusion module is used to iteratively fuse shallow edge features and deep semantic features;

[0023] The cross-level feature progressive refinement module is used to improve the ability to preserve the contours of small targets and the detection robustness in complex backgrounds;

[0024] The hierarchical feature refinement module is used to enhance key information.

[0025] Furthermore, the key information enhancement and fusion module includes a small target key information enhancement submodule and a key information perception feature fusion submodule, wherein the small target key information enhancement submodule is represented by the following formula:

[0026]

[0027] In the formula,

[0028]

[0029]

[0030]

[0031]

[0032]

[0033] in, As a feature of fusion, For splicing, , and For learnable convolutional kernel weights, For convolution operations, , and For different convolution outputs, For activation function, The output of the submodule for enhancing key information of small targets is improved. This is the output of the first multi-path dynamic weighted attention module. To add element by element, For channel-by-channel multiplication, A is the output of the channel attention. and For ReLU activation function and Sigmoid activation function, and For learnable convolutional kernel weights, AAP stands for Adaptive Average Pooling;

[0034] The key information perception feature fusion submodule is represented by the following formula:

[0035]

[0036] In the formula,

[0037]

[0038]

[0039]

[0040] in, The output of the key information perception feature fusion submodule is G, which represents the gating generated branch feature. For feature transformation branch features, It is a 3×3 convolution. For splicing features, R is the set of real numbers, B is the training batch size, C is the number of channels, H is the height, and W is the width.

[0041] Furthermore, the hierarchical feature refinement module includes an attention-guided refinement submodule and a context-aware refinement submodule;

[0042] The attention-guided refining submodule is represented by the following formula:

[0043]

[0044] In the formula,

[0045]

[0046] in, To guide the output of the attention-driven refining submodule, WC is used for dynamic weighted fusion. This is the output of the second multi-path dynamic weighted attention module. For splicing, This represents the output of the convolutional block attention module, CA represents the channel attention module, and SA represents the spatial attention module. For element-wise multiplication;

[0047] The context-aware refining submodule is represented by the following formula:

[0048]

[0049] in, For the context-aware refining submodule output, To create a pyramid-shaped pool for empty spaces, This is the output of the third multi-path dynamic weighted attention module.

[0050] Furthermore, the cross-level feature progressive refinement module is represented by the following formula:

[0051]

[0052] In the formula,

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059] in, This is the output of the cross-level feature progressive refinement module. The C3 module is reparameterized and executed 3 times in a loop. and For different The output, and All are outputs of the hierarchical feature refinement module. For the output of attention-based intra-scale feature interactions, For attention-based intra-scale feature interactions, This is the output of the fourth multi-path dynamic weighted attention module. For the output of the self-attention module, For self-attention modules, The network uses windmill-type convolutions, FFN (convolutional feedforward network), LN (layer normalization), and F1 (intermediate features). For window attention, To add element by element, For convolutional features, This is the output of the key information perception feature fusion submodule.

[0060] Furthermore, the decoder is represented by the following formula:

[0061]

[0062] In the formula,

[0063]

[0064]

[0065]

[0066]

[0067]

[0068] Where M is the output of the real-time detection Transformer decoder. To detect the Transformer decoder in real time, and The input features are the processed features, and UP represents upsampling. , and Different The output of .

[0069] Furthermore, the loss function is expressed by the following formula:

[0070]

[0071] In the formula,

[0072]

[0073]

[0074] in, For loss function, The main focus is on detecting loss. To mitigate losses, For noise reduction loss, and These are the weighting coefficients. For bounding box loss, For classifying losses, For bounding box loss, For L1 loss, For GIoU loss, and For weights.

[0075] A second aspect of this invention proposes a UAV infrared small target detection system based on key information enhancement and progressive refinement of cross-level features, comprising:

[0076] The collection unit is used to collect the training image dataset;

[0077] The first detection unit is used to input the training image dataset into a preset infrared small target detection model to obtain the detection result. The infrared small target detection model enhancement includes a backbone, an encoder, and a decoder. The backbone includes multiple sequentially connected multi-path dynamic weighted attention modules. Each multi-path dynamic weighted attention module includes a first path, a second path, a third path, and a fusion layer. The second path includes two parallel branches. The multi-path dynamic weighted attention module can provide richer feature representations.

[0078] The training unit is used to train and optimize the infrared small target detection model enhancement based on the loss function, so as to obtain the optimal infrared small target detection model enhancement and improve detection accuracy.

[0079] The second detection unit is used to input the target image into the optimal infrared small target detection model enhancement to obtain the final detection result.

[0080] The beneficial effects of this invention are:

[0081] The multi-path dynamic weighted attention module in the backbone of this invention collaboratively preserves original information and mines deep patterns. Combined with a dimensionality reduction-upgrading hierarchical transformation mechanism, it improves feature discrimination power while maintaining computational efficiency, significantly enhancing the model's ability to represent weak infrared targets. The encoder captures structural cues of different granularities through multi-scale convolution, then adaptively highlights feature channels related to edges and details using channel attention, and fuses the enhanced information back into the original features in a residual manner, thereby improving the network's sensitivity to small targets and fine structures. Simultaneously, the cross-layer feature refinement module significantly improves the ability to preserve the contours of small targets and the robustness of detection in complex backgrounds. In the hierarchical feature refinement module, the middle layer uses channel-spatial attention to dynamically focus on the target region and suppress background interference; the deep features utilize multi-scale dilated convolution to effectively capture long-range contextual dependencies, improving semantic understanding. Therefore, this invention enhances the infrared small target detection model through design, improving the accuracy of small target detection in complex environments. Attached Figure Description

[0082] Figure 1 The flowchart illustrates a method for detecting small infrared targets from unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features, as provided in this embodiment of the invention.

[0083] Figure 2 This is a schematic diagram of the infrared small target detection model provided in an embodiment of the present invention.

[0084] Figure 3 This is a schematic diagram of a multi-path dynamic weighted attention module provided in an embodiment of the present invention.

[0085] Figure 4 This is a schematic diagram of a small target key information enhancement submodule provided in an embodiment of the present invention.

[0086] Figure 5 A schematic diagram of the key information perception feature fusion submodule provided in the embodiments of the present invention.

[0087] Figure 6 This is a schematic diagram of a cross-level feature progressive refinement module provided in an embodiment of the present invention.

[0088] Figure 7 This is a schematic diagram of the layered feature refining module provided in an embodiment of the present invention.

[0089] Figure 8 This is an architecture diagram of an infrared small target detection system for unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features, provided for an embodiment of the present invention. Detailed Implementation

[0090] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0091] Example 1

[0092] like Figure 1 As shown, this invention proposes a method for detecting small infrared targets on unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features, including:

[0093] S101: Collect the training image dataset.

[0094] S102: Input the training image dataset into the preset infrared small target detection model to obtain the detection result; the infrared small target detection model enhancement includes a backbone, an encoder and a decoder; the backbone includes multiple sequentially connected multi-path dynamic weighted attention modules, the multi-path dynamic weighted attention module includes a first path, a second path, a third path and a fusion layer, the second path includes two parallel branches.

[0095] S103: Train and optimize the infrared small target detection model enhancement based on the loss function to obtain the optimal infrared small target detection model enhancement.

[0096] S104: Input the target image into the optimal infrared small target detection model enhancement to obtain the final detection result.

[0097] This invention first collects a training image dataset, then uses this dataset to train an enhanced infrared small target detection model, obtaining the optimal enhanced model. Finally, the enhanced model is used to detect targets in the image. The backbone of the enhanced infrared small target detection model employs a multi-path structure to provide richer feature representations and introduces a hierarchical feature transformation mechanism of dimensionality reduction and upgrading to further enhance the model's feature representation capabilities. This invention enhances key target information in the infrared small target detection model, effectively improving the detection performance of infrared small targets.

[0098] Example 2

[0099] Based on the above embodiments, such as Figure 2 As shown, this invention proposes a specific process for a UAV infrared small target detection method based on key information enhancement and progressive refinement of cross-level features, including:

[0100] S201: Collect the training image dataset.

[0101] Specifically, to facilitate model training, this embodiment selects three publicly available infrared image datasets. These datasets consist of existing publicly available infrared images, including three types of datasets: IRSTD-1K, NUDT_SIRST, and NUAA_SIRST. It should be noted that each dataset is divided into a training set, a validation set, and a test set in an 8:1:1 ratio. The specific ratio and test content can be flexibly determined by the implementer according to actual needs and are not subject to rigid requirements.

[0102] S202: Input the training image dataset into the preset infrared small target detection model to obtain the detection result; the infrared small target detection model enhancement includes a backbone, an encoder and a decoder; the backbone includes multiple sequentially connected multi-path dynamic weighted attention modules, the multi-path dynamic weighted attention module includes a first path, a second path, a third path and a fusion layer, the second path includes two parallel branches.

[0103] Specifically, in this embodiment, three publicly available infrared image datasets are used as the training data for the model, that is, as the input datasets. Before input, the dataset format needs to be modified to meet the model training format requirements. Before model training, key training parameters need to be configured, including: neuron activation function, fully connected layer activation function, optimization function, loss function, number of network iterations, batch scale, and learning rate. The specific configurations of each training parameter are shown in Table 1.

[0104]

[0105] In Table 1, the model input features have three dimensions: height, width, and number of channels. The neuron activation function is the ReLU activation function. The network training optimization function is the Adam function, and the loss function is binary cross-entropy to solve the regression problem. The model iteration count is 300 epochs, and the weights are updated for 48 samples per batch, i.e., the batch_size is 4. The model learning rate is 0.0001.

[0106] It should be noted that the specific training process of the infrared small target detection model is existing technology and is not within the scope of protection of this invention, and will not be described in detail here.

[0107] Thus, this implementation has obtained a detection model for detecting small infrared targets.

[0108] The infrared target detection model includes a backbone, encoder, and decoder.

[0109] like Figure 3 As shown, the backbone is used to enhance the extraction of small target features from the input infrared image, obtaining rich feature representations and effectively improving the small target detection capability. Here, the core residual block uses simple residual connections, which is insufficient in representing small targets during the feature extraction stage, making it difficult to effectively capture the detailed features and contextual information of infrared small targets. To address this, this invention designs a multi-path dynamic weighted attention (DPAR-Block) module based on BasicBlock, which enhances the discriminativeness and robustness of feature extraction by introducing multi-path fusion, attention mechanisms, and depthwise separable convolution.

[0110] A multi-path structure was designed in the second path to provide richer feature representations. The left side of the second path is a newly added path, which introduces depthwise separable convolutions to reduce computation while maintaining feature extraction capabilities. Then, channel attention modules and spatial attention modules are added, enabling the network to adaptively adjust the importance of feature map channels and spatial positions, improving sensitivity to small infrared targets. Finally, it is multiplied and added to the original path on the right, which is equivalent to adding an attention-weighted feature modulation to the original path, helping to preserve the detailed information of small targets. This process can be expressed by the following formula:

[0111]

[0112]

[0113]

[0114] in, This is the output of the second path. and These are the outputs of the two parallel branches, and SA represents the spatial attention mechanism. It is a depthwise separable convolution, AP is average pooling, and X is the input feature. The features are convolutional features, and BN is batch normalization.

[0115] In the third path, a hierarchical feature transformation mechanism involving dimensionality reduction and upscaling significantly enhances the model's feature representation capability. First, a 1×1 convolutional kernel is used for channel dimensionality reduction compression, forcing the network to learn a more discriminative and compact feature representation. Then, spatially aware feature reconstruction is performed using 3×3 convolutions, restoring the original dimensionality while incorporating local contextual information. Finally, a channel attention mechanism is coupled to adaptively calibrate the importance weights of feature channels. This cascaded design forms an efficient feature refinement pipeline: dimensionality reduction suppresses redundant background information, upscaling strengthens the saliency of target features, and the attention mechanism dynamically improves the response of channels relevant to small targets. This process can be expressed as the following formula:

[0116]

[0117]

[0118] in, For the output of the third path, The features are obtained by applying an activation function to a standard normalized convolution. For activation function, For standard normalized convolution, CA represents channel attention mechanism.

[0119] In summary, the DPAR-Block module is represented by the following formula:

[0120]

[0121] In the formula,

[0122]

[0123] in, For the output of the DPAR-Block module, This is the output of the first path. To add element by element, This is for element-wise multiplication.

[0124] The encoder includes a key information enhancement and fusion module, a cross-level feature progressive refinement (CFPR) module, and a hierarchical feature refinement (ACRM) module. The key information enhancement and fusion module and the cross-level feature progressive refinement module enhance the robustness and accuracy of the detection model.

[0125] In the input key information enhancement and fusion module, multi-scale convolution captures structural cues of different granularities. Channel attention then adaptively highlights feature channels related to edges and details, and the enhanced information is fused back into the original features using a residual approach. This improves the network's sensitivity to small targets and fine structures without introducing explicit edge supervision. Subsequently, the feature map passes through the channel attention module to dynamically enhance task-relevant channels and suppress noise interference. Specifically, global average pooling is first performed to compress the spatial dimension, followed by a bottleneck structure to enhance feature discriminativity, and then Sigmoid activation is applied to generate channel weights, achieving channel-level feature selection.

[0126] The key information enhancement and fusion module includes a small target key information enhancement (STKE) submodule and a key information perception feature fusion (KAFF) submodule, such as... Figure 4 As shown, the submodule for enhancing key information of small targets can be implemented through the following steps:

[0127] First, convolutional kernels with different receptive fields are used to capture multi-scale edge features. For the input feature X, the module performs three convolutional operations in parallel: 1x1 convolution focuses on pixel-level local features, enhancing the point-like edge response of small targets and capturing local gradient changes; 3x3 convolution establishes neighborhood context associations, strengthening the contour continuity of small targets; and 3x3 dilated convolution, with an dilation rate of 2, expands the receptive field, connects discrete edge segments, and solves the breakage problem. This can be expressed as the formula:

[0128]

[0129]

[0130]

[0131] in, , For learnable convolutional kernel weights, , , C is the number of channels. For convolution operations, , and For different convolution outputs, This is the activation function.

[0132] Multi-scale features are concatenated and then fused through a fusion layer to adaptively integrate multi-scale edge responses, thereby reducing the semantic gap between feature maps. This can be represented as:

[0133]

[0134] in, As a feature of fusion, For splicing, For learnable convolutional kernel weights, .

[0135] First, global average pooling is performed to compress the spatial dimensionality. Then, a bottleneck structure is used to enhance feature discriminative power. Finally, sigmoid activation is applied to generate channel weights, achieving channel-level feature selection. This process can be described by the following formula:

[0136]

[0137] in, For feature fusion, A is the output of channel attention. and For ReLU activation function and Sigmoid activation function, and For learnable convolutional kernel weights, , r is the shrinkage ratio, with a default value of 16. AAP indicates adaptive average pooling.

[0138] The optimized multi-scale features are transformed into edge enhancement features compatible with the original feature space, and then residually linked with the original features to obtain the final feature. This preserves the original semantic information and avoids edge enhancement from destroying the existing feature structure. The specific formula is as follows:

[0139]

[0140] in, The output of the submodule for enhancing key information of small targets is improved. This is the output of the first multi-path dynamic weighted attention module. To add element by element, This involves multiplying each channel sequentially.

[0141] STKE module output Focusing on target edge details can easily lead to the loss of global semantic information; the original S1 feature X retains complete global semantics, but its edge response is weak. Directly concatenating or adding the two element-wise can easily lead to feature redundancy and semantic conflicts, thus reducing model performance. Therefore, a Kernel-Adaptive Fusion Module (KAFF) is designed. Through dual-branch processing and an adaptive gating mechanism, it achieves spatial adaptive fusion, taking into account both edge details and global semantics, thereby improving the completeness and discriminativeness of feature representation. Figure 5 As shown, the KAFF module enhances the features using the original S1 features and the key information output by STKE. For input, the specific process is as follows:

[0142] First, feature concatenation is performed, and By concatenating along the channel dimension, we obtain a dimension of splicing features The mathematical expression is:

[0143]

[0144] in, For splicing features, R is the set of real numbers, B is the training batch size, C is the number of channels, H is the height, and W is the width.

[0145] Subsequently, a dual-parallel branching approach was used, designing two independent branch pairs. The processing involves independent learning of branch weights, ensuring they do not interfere with each other. The gating-generated branches are then processed through 3×3 convolutions to capture local spatial context, and then... The activation function generates a spatially gated map G with dimensions C×H×W, quantifying the dependence of each pixel position on edge features—regions with G≈1 are the target regions, enhancing edge feature contributions; regions with G≈0 are the background regions, preserving original semantic features. The calculation method for the gating generation branch is shown in the formula:

[0146]

[0147] Where G is the output of the gated generation branch. It is a 3×3 convolution.

[0148] The feature transformation branch learns the optimal combination of original and edge features through 3×3 convolution, and then enhances the nonlinear representation capability through the ReLU activation function to output transformed features. As shown in the formula:

[0149]

[0150] in, This is the output of the feature transformation branch.

[0151] Finally, based on the gated graph G, the fusion ratio of edge features and transform features is dynamically adjusted to achieve spatial adaptive feature mixing, while ensuring that at least one feature representation is retained at all locations to avoid feature information loss. This can be expressed mathematically as follows:

[0152]

[0153] in, This is the output of the key information perception feature fusion submodule.

[0154] The key information perception feature fusion submodule can enhance edge details in the target area and retain semantic information in the background area, effectively alleviating the problems of feature redundancy and conflict.

[0155] In this embodiment, the Cross-Level Feature Progressive Refinement (CFPR) module is as follows: Figure 6 As shown, shallow features are enhanced and continuously combined with deep features to form a new encoder input head. The specific process is as follows:

[0156] Features from the upper layer are convolved in a windmill pattern to enhance feature extraction capabilities and expand the receptive field. This process introduces only a small number of parameter increments.

[0157] L is obtained by using overlapping patches via LRSA to enhance the interaction between local region features. This feature L undergoes dual fusion: on the one hand, it is fused with mid-level features to enrich high-level semantics; on the other hand, L is fused with features from different levels in CFPR. , , The data is then stitched together (a convolution operation is also required during the stitching process, but it is not shown in the figure) to improve the model's understanding of multi-scale information.

[0158] Reusing convolution operations and the RepC3 module, the 1x1 convolution operation makes the feature map and { , , The features are aligned separately to facilitate splicing. The RepC3 module enhances and refines the feature maps through multiple convolutional operations and skip connections, making it easier to capture local details and global context in small object detection.

[0159] In summary, the cross-level feature progressive refinement module is represented by the following formula:

[0160]

[0161] In the formula,

[0162]

[0163]

[0164]

[0165]

[0166]

[0167]

[0168] in, This is the output of the cross-level feature progressive refinement module. The C3 module is reparameterized and executed 3 times in a loop. and For different The output, and All are outputs of the hierarchical feature refinement module. For the output of attention-based intra-scale feature interactions, For attention-based intra-scale feature interactions, This is the output of the fourth multi-path dynamic weighted attention module. For the output of the self-attention module, For self-attention modules, The network uses windmill-type convolutions, FFN (convolutional feedforward network), LN (layer normalization), and F1 (intermediate features). For window attention, To add element by element, These are convolutional features.

[0169] The Hierarchical Feature Refinement (ACRM) module refines features at the intermediate and deep layers respectively, improving the feature quality of the input network. For example... Figure 7 As shown, the hierarchical feature refinement (ACRM) module includes the attention-guided refinement (AGRM) submodule and the context-aware refinement (CARM) submodule.

[0170] The attention-guided refining submodule is represented by the following steps:

[0171] The input features undergo a two-stage refinement process, following channel attention and spatial attention modules. This design allows the model to first focus on "which channels are important," and then on "which spatial locations are important," thereby capturing the key information in the features more comprehensively. This process can be represented as:

[0172]

[0173] This is the output of the second multi-path dynamic weighted attention module. For splicing, This represents the output of the convolutional block attention module, CA represents the channel attention module, and SA represents the spatial attention module. This is for element-wise multiplication.

[0174] s' is then concatenated with s2, and the resulting output is then dynamically weighted and fused with s2 to finally obtain the desired output. While refining features, it also retains as much original information as possible, as expressed by the following formula:

[0175]

[0176] in, WC is the output of the attention-guided refining module submodule, and WC is the dynamic weighted fusion.

[0177] A CARM module can be represented by the following steps:

[0178] The input feature map is fed into multiple dilated convolutional layers, each using a different dilation rate to extract features from different scales. The output feature maps of each dilated convolutional layer are concatenated along the channel dimension to form a multi-scale feature map. The concatenated feature map is then fused and mapped through a 1×1 convolutional layer to obtain the final output feature map.

[0179] The overall process of the CARM module can be represented as follows:

[0180]

[0181] in, For the output of the context-aware refining module submodule, To create a pyramid-shaped pool for empty spaces, This is the output of the third multi-path dynamic weighted attention module.

[0182] The decoder is represented by the following formula:

[0183]

[0184] In the formula,

[0185]

[0186]

[0187]

[0188]

[0189]

[0190] Where M is the output of the real-time detection Transformer decoder. To detect the Transformer decoder in real time, and The input features are the processed features, and UP represents upsampling. , and Different The output of .

[0191] S203: Train and optimize the infrared small target detection model enhancement based on the loss function to obtain the optimal infrared small target detection model enhancement.

[0192] Specifically, in infrared small target detection tasks, classification accuracy, localization precision, and anti-interference capability must be considered simultaneously. Traditional single loss functions are insufficient to balance these multi-dimensional requirements. Therefore, DEGNet designs a composite loss function with a total loss... Loss detected by the main detector Auxiliary losses and denoising loss It consists of three parts, achieving multi-objective optimization through weight adjustment, balancing accuracy, convergence speed, and anti-interference capability, mathematically expressed as:

[0193]

[0194] in, For loss function, The main focus is on detecting loss. To mitigate losses, For noise reduction loss, and These are the weighting coefficients (all set to 1.0).

[0195] Main detection loss The loss consists of two parts: bounding box regression loss and classification loss, which optimize target localization accuracy and class discrimination accuracy, respectively. The bounding box regression loss uses a combination of L1 loss and GIoU loss, jointly optimizing coordinate distance and geometric similarity to improve target localization accuracy while addressing the gradient vanishing problem in non-overlapping bounding boxes. The bounding box loss is calculated as follows:

[0196]

[0197] in, For bounding box loss, and We set the weights to 5.0 and 2.0 respectively, based on experimental verification, to balance the contribution ratios of the two losses. The L1 loss, a coordinate regression loss, optimizes the coordinate distance between the predicted bounding box and the ground truth bounding box. It can be expressed by the following formula:

[0198]

[0199] in, This represents the number of true bounding boxes (positive samples) matched in the current batch. and These represent the coordinate vectors of the i-th predicted bounding box and the ground truth bounding box, respectively. This represents the L1 norm (Manhattan distance). This is a loss for GIoU.

[0200] GIoU loss is the generalized intersection-union loss. It optimizes the geometric similarity of the bounding boxes by introducing the minimum closure rectangle between the predicted and ground truth bounding boxes, thus solving the gradient vanishing problem when there is no overlap. The calculation method is as follows:

[0201]

[0202]

[0203] in, For intersection, union, and comparison, For the prediction box, For the true frame, for and The minimum closure rectangle, Indicates the area of ​​the region. For generalized intersection and comparison, Let be the generalized intersection-union ratio (CUIR) corresponding to the i-th successfully matched positive sample.

[0204] The classification loss uses a binary cross-entropy loss function, introducing a 1:3 positive-to-negative sample weight ratio to alleviate the imbalance problem of positive and negative samples in infrared small target detection and improve the accuracy of class discrimination. The calculation method is as follows:

[0205]

[0206] in, For classifying losses, The total number of target categories, Original, authentic labels This is the label smoothing coefficient, with a default value of 0.1. The larger the value, the smoother the surface and the stronger the generalization ability, but the training convergence speed may be slower. Predicting categories for the model The probability is obtained by normalizing the output of the model's classification head using the Softmax function. The final calculation of the main detection loss is:

[0207]

[0208] Auxiliary loss With main detection loss The structures are completely identical, the difference lies in The loss from the 2nd to 5th intermediate decoding layers of RT-DETR is averaged as the final auxiliary loss. This design alleviates the vanishing gradient problem in deep networks, accelerates model convergence, improves the stability of the training process, and prevents the model from getting trapped in local optima.

[0209] Denoising loss To address the issue of strong noise interference and high false alarm rates in infrared images, a negative sample mining strategy is employed. This strategy penalizes background regions with a confidence level greater than 0.3, suppressing false detections caused by clutter interference and reducing the false alarm rate. The denoising loss is calculated as follows:

[0210]

[0211] in, The prediction confidence of the background region is used to penalize the background region with high confidence, which guides the model to distinguish between background clutter and target features and improves the anti-interference ability.

[0212] S204: Input the target image into the optimal infrared small target detection model enhancement to obtain the final detection result.

[0213] Example 3

[0214] Based on the above embodiments, this invention proposes a detection performance verification process for a UAV infrared small target detection method based on key information enhancement and progressive refinement of cross-level features, including:

[0215] (1) Determine the evaluation indicators.

[0216] Here, we use precision, recall, mAP50, mAP50-95, number of parameters, and F1 (determined by recall and precision).

[0217] (2) Prepare the dataset.

[0218] Here, the datasets are derived from three publicly released infrared image small target datasets: NUAA_SIRST dataset, IRSTD-1K dataset, and NUDT-SIRST dataset.

[0219] The NUAA_SIRST dataset holds a significant position among commonly used public datasets for single-frame infrared small target detection. This dataset contains 480 instances from different scenes, with 427 instances derived from real-world video, making it highly representative. Furthermore, most small targets in the NUAA_SIRST dataset are extremely low in brightness, not only concealed within highly complex backgrounds but also subject to clutter interference.

[0220] The IRSTD-1K dataset consists of 1000 real infrared images, all with pixel-level annotations. These images cover a variety of scenes and backgrounds, including urban, rural, and marine environments, effectively improving the representativeness of the dataset and the challenge of the detection task. Furthermore, the targets in the images vary in shape and size, and the backgrounds contain a large amount of clutter from different scenes.

[0221] NUDT-SIRST is a newly developed synthetic infrared small target dataset containing 1327 images. Its advantages lie in the diverse sizes and categories of targets within the images, the rich and varied backgrounds, and the highly accurate annotations. Each image is synthesized from a real-world background, incorporating various target types, and captured using different shooting devices and angles. Furthermore, the NUDT-SIRST dataset includes five main background scene categories: urban, field, highlight, ocean, and cloud, further constructing more challenging scenes such as multi-target, point target, and dimly lit targets.

[0222] (3) Set up the experiment.

[0223] Here, the experiment was conducted on a computer with 60GB of memory on the PyTorch platform. The programming language and version was Python 3.10, the framework and corresponding version was PyTorch 2.2.0, the graphics card used was an RTX 3090, the operating system was Ubuntu 22.04, and the development tool was Visual Studio Code.

[0224] (4) Comparative experiment.

[0225] Here, YOLOv5s, YOLOv7n, YOLOv8n, YOLOv9-c, YOLOv10n, Enhanced Feature Learning Network for Infrared Small Target Detection (EFLNet), SwinTransformer-based Multi-Scale Spatial Pyramid Pooling Network for Infrared Small Target Detection (STASPPNet), Deformable DET, Detrs with Hybrid Matching (H-DETR), and RT-DETR are used as comparison methods on the NUAA_SIRST dataset. Adversarial learning methods for small target segmentation in infrared images (MDvsFA), Attention-Guided Pyramid Context Network for Infrared Small Target Detection (AGPCNet), Attention-Guided Pyramid Context Method (AGPCNet), Asymmetric Context Modulation (ACM), ISNet, Attention-Based Local Contrast Method (ALCNet), Densely Nested Attention Network (DNANet), EFLNet, and STASPPNet are used as comparison methods on the IRSTD-1k and NUDT-SIRST datasets. All of these methods are deep learning methods.

[0226] Table 2 shows the evaluation results of the comparative experimental models on the NUAA_SIRST dataset:

[0227]

[0228] As shown in Table 2, the DETR series methods have higher parameters, weights, and GFLOPs than the YOLO series methods and STASPPNet. YOLOv10n has the fewest parameters. However, except for YOLOv5s, the DETR series has slightly higher mAP50 and mAP50-95 than the YOLO series. The infrared small target detection model (DEGNet) has parameter counts and weights similar to the DETR series, but higher than the YOLO series. Our method has the best mAP50 performance, 3%-13% higher than other comparative methods, and mAP50-95 is also 3%-10% higher than other comparative methods.

[0229] The evaluation results of the comparative experiments on the IRSTD-1k and NUDT-SIRST datasets are shown in Table 3:

[0230]

[0231] Currently, most deep learning-based solutions treat infrared small target detection as a pixel-level segmentation task. These methods require generating pixel-level segmentation results, but even small deviations can lead to false alarms or missed detections, resulting in a decline in target-level detection performance. Furthermore, existing methods do not adequately address the class imbalance and bounding box sensitivity issues inherent in infrared small target detection, causing them to perform poorly in target-level detection tasks, resulting in low precision, recall, and F1 scores. Table 3 provides a quantitative comparison of the results of different methods. It can be seen that, compared to the current state-of-the-art (SOTA) method, our proposed method leads in all evaluation metrics on the NUDT-SIRST and IRSTD1k datasets, fully demonstrating the effectiveness of our proposed method.

[0232] The DEGNet model provided in this embodiment achieves optimal values ​​for parameter count, FLOPs, and inference time, which are 0.04M, 3.41G, and 0.17s, respectively. Compared to other models, it has been minimized, fully demonstrating the lightweight nature of this method.

[0233] Therefore, Tables 2 and 3 show that the UAV infrared small target detection method based on key information enhancement and cross-level feature progressive refinement can achieve better accuracy and improve the robustness of infrared small target detection.

[0234] (5) Ablation experiment.

[0235] Here, to explore the contribution of each component of the DEGNet model to the infrared small target detection performance, ablation experiments were conducted. The DEGNet model consists of a backbone, encoder, hierarchical refinement mechanism, and decoder. These components will be added separately to evaluate their impact on model performance, specifically:

[0236] The core multipath feature extraction module is MARF. The edge enhancement modules (SKAF and KAFF) and cross-layer feature refinement module (CFPR) in the encoder both process shallow features; therefore, they are treated as a whole (SKC) for ablation. AGRM and CARM serve as hierarchical feature refinement mechanisms to refine mid-layer features; AGRM and CARM are treated as a whole (ACRM) for ablation experiments. Finally, performance metrics such as precision (P), recall (R), and mean precision (mAP50 and mAP50-95) are used to demonstrate the advantages and disadvantages of adding modules. The ablation experiments were uniformly set to 300 rounds and conducted on the NUAA-SIRST dataset.

[0237] The evaluation results of the ablation experimental model are shown in Table 4:

[0238]

[0239] Based on the results in Table 4, we can easily see the effectiveness of each module in DEGNet. Compared with the baseline model RT-DETR, after adding MARF, mAP50 improved by 6.5%, map50-95 improved by 2.1%, and recall improved by 1.8%, demonstrating the enhanced feature extraction capability and improved model localization ability for small targets. Compared with the baseline model, after adding ACRM, map50-95, precision, recall, and map50 improved by 1.9%, 4.9%, 2.9%, and 0.9%, respectively. This verifies that the model's detection capability is improved after feature refinement through ACRM. After adding the SKC module, precision improved by 0.5%, mAP50 improved by 5.7%, map50-95 improved by 3.1%, and recall improved by 3.8%, demonstrating the enhanced shallow feature extraction capability of the SKC module and the enhanced overall robustness of the model.

[0240] The DEGNet model, which combines MARF, ACRM, and SKC, has achieved a new level of target detection accuracy. This result fully demonstrates the complementarity and synergistic effect of MARF, ACRM, and SKC in lightweight infrared small target tasks, effectively improving the model's detection performance.

[0241] The ablation experiments demonstrate the effectiveness of each component of the DEGNet module in this embodiment, indicating that combining the multi-path feature extraction module, the hierarchical feature refinement mechanism, and the cross-layer feature refinement module is essential for improving the accuracy of infrared small target detection.

[0242] The VPX platform was tested using the Jetson AGX Orin 32GB series, and the specific parameters are shown in Table 5:

[0243]

[0244] The VPX platform has limited computing power and storage resources, thus requiring lightweight object detection algorithms. On the NUAA_SIRST dataset, the DEGNet method achieves high evaluation metrics (mAP50 of 95.2%, mAP50-95 of 49.1%, precision of 97.4%, and recall of 90.5%) while maintaining lightweight performance.

[0245] This embodiment was tested on a drone equipped with the VPX platform in an existing project. Through experimental verification on the VPX platform, the DEGNet method demonstrated excellent improvement in detection accuracy.

[0246] Example 4

[0247] Based on the above embodiments, such as Figure 8 As shown, this invention proposes a UAV infrared small target detection system based on key information enhancement and progressive refinement of cross-level features, characterized by comprising:

[0248] The collection unit is used to collect the training image dataset.

[0249] The first detection unit is used to input the training image dataset into a preset infrared small target detection model to obtain the detection result. The infrared small target detection model enhancement includes a backbone, an encoder, and a decoder. The backbone includes multiple sequentially connected multi-path dynamic weighted attention modules. Each multi-path dynamic weighted attention module includes a first path, a second path, a third path, and a fusion layer. The second path includes two parallel branches.

[0250] The training unit is used to train and optimize the infrared small target detection model enhancement based on the loss function, so as to obtain the optimal infrared small target detection model enhancement.

[0251] The second detection unit is used to input the target image into the optimal infrared small target detection model enhancement to obtain the final detection result.

[0252] It should be noted that the UAV infrared small target detection system based on key information enhancement and cross-level feature progressive refinement provided in this embodiment of the invention is to implement the above-mentioned UAV infrared small target detection method based on key information enhancement and cross-level feature progressive refinement. Its specific functions can be referred to the above-mentioned method embodiments, and will not be repeated here.

[0253] In summary, the multi-path dynamic weighted attention module in the core of this invention collaboratively preserves original information and mines deep patterns. Combined with a dimensionality reduction-upgrading hierarchical transformation mechanism, it improves feature discrimination while maintaining computational efficiency, significantly enhancing the model's ability to represent weak infrared targets. The encoder captures structural cues of different granularities through multi-scale convolution, then adaptively highlights feature channels related to edges and details using channel attention, and fuses the enhanced information back into the original features in a residual manner, thereby improving the network's sensitivity to small targets and fine structures. Simultaneously, the cross-layer feature refinement module significantly improves the ability to preserve the contours of small targets and the robustness of detection in complex backgrounds. In the hierarchical feature refinement module, the middle layer uses channel-spatial attention to dynamically focus on the target region and suppress background interference; the deep features utilize multi-scale dilated convolution to effectively capture long-range contextual dependencies, improving semantic understanding. Therefore, this invention enhances the infrared small target detection model through design, improving the accuracy of small target detection in complex environments.

[0254] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting small infrared targets on unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features, characterized in that, include: Step 1: Collect the training image dataset; Step 2: Input the training image dataset into the preset infrared small target detection model to obtain the detection results; The infrared small target detection model enhancement includes a backbone, an encoder, and a decoder; the backbone includes multiple sequentially connected multi-path dynamic weighted attention modules, each of which includes a first path, a second path, a third path, and a fusion layer, wherein the second path includes two parallel branches; Step 3: Train and optimize the infrared small target detection model enhancement based on the loss function to obtain the optimal infrared small target detection model enhancement; Step 4: Input the target image into the optimal infrared small target detection model enhancement to obtain the final detection result.

2. The method for detecting small infrared targets in unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features as described in claim 1, characterized in that, The multi-path dynamic weighted attention module is represented by the following formula: In the formula, in, This is the output of the multi-path dynamic weighted attention module. This is the output of the first path. This is the output of the second path. For the output of the third path, For standard normalized convolution, The features are obtained by applying an activation function to a standard normalized convolution. Let X be the activation function and X be the input feature. and These are the outputs of the two parallel branches, where SA and CA represent the spatial attention mechanism and the channel attention mechanism, respectively. This is a depthwise separable convolution, AP is average pooling, and BN is batch normalization. For convolutional features, To add element by element, This is for element-wise multiplication.

3. The method for detecting small infrared targets in unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features as described in claim 1, characterized in that, The encoder includes a key information enhancement and fusion module, a cross-level feature progressive refinement module, and a hierarchical feature refinement module; The key information enhancement and fusion module is used to iteratively fuse shallow edge features and deep semantic features; The cross-level feature progressive refinement module is used to improve the ability to preserve the contours of small targets and the detection robustness in complex backgrounds; The hierarchical feature refinement module is used to enhance key information.

4. The method for detecting small infrared targets in unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features according to claim 3, characterized in that, The key information enhancement and fusion module includes a small target key information enhancement submodule and a key information perception feature fusion submodule. The small target key information enhancement submodule is represented by the following formula: In the formula, in, As a feature of fusion, For splicing, , and For learnable convolutional kernel weights, For convolution operations, , and For different convolution outputs, For activation function, The output of the submodule for enhancing key information of small targets is improved. This is the output of the first multi-path dynamic weighted attention module. To add element by element, For channel-by-channel multiplication, A is the output of the channel attention. and For ReLU activation function and Sigmoid activation function, and For learnable convolutional kernel weights, AAP stands for Adaptive Average Pooling; The key information perception feature fusion submodule is represented by the following formula: In the formula, in, G is the output of the key information perception feature fusion submodule, and G is the gated generation branch feature. For feature transformation branch features, It is a 3×3 convolution. For splicing features, R is the set of real numbers, B is the training batch size, C is the number of channels, H is the height, and W is the width.

5. The method for detecting small infrared targets in unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement across hierarchical features according to claim 3, characterized in that, The hierarchical feature refinement module includes an attention-guided refinement submodule and a context-aware refinement submodule. The attention-guided refining submodule is represented by the following formula: In the formula, in, To guide the output of the attention-driven refining submodule, WC is used for dynamic weighted fusion. This is the output of the second multi-path dynamic weighted attention module. For splicing, This represents the output of the convolutional block attention module, CA represents the channel attention module, and SA represents the spatial attention module. For element-wise multiplication; The context-aware refining submodule is represented by the following formula: in, For the context-aware refining submodule output, To create a pyramid-shaped pool for empty spaces, This is the output of the third multi-path dynamic weighted attention module.

6. The method for detecting small infrared targets in unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features according to claim 5, characterized in that, The cross-level feature progressive refinement module is represented by the following formula: In the formula, in, This is the output of the cross-level feature progressive refinement module. The C3 module is reparameterized and executed 3 times in a loop. and For different The output, and All are outputs of the hierarchical feature refinement module. For the output of attention-based intra-scale feature interactions, For attention-based intra-scale feature interactions, This is the output of the fourth multi-path dynamic weighted attention module. For the output of the self-attention module, For self-attention modules, The network uses windmill-type convolutions, FFN (convolutional feedforward network), LN (layer normalization), and F1 (intermediate features). For window attention, To add element by element, For convolutional features, This is the output of the key information perception feature fusion submodule.

7. The method for detecting small infrared targets in unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features according to claim 6, characterized in that, The decoder is represented by the following formula: In the formula, Where M is the output of the real-time detection Transformer decoder. To detect the Transformer decoder in real time, and The input features are the processed features, and UP represents upsampling. , and Different The output of .

8. The method for detecting small infrared targets in unmanned aerial vehicles (UAVs) based on key information enhancement and progressive refinement of cross-level features according to claim 1, characterized in that, The loss function is expressed by the following formula: In the formula, in, For loss function, The main focus is on detecting loss. To mitigate losses, For noise reduction loss, and These are the weighting coefficients. For bounding box loss, For classifying losses, For bounding box loss, For L1 loss, For GIoU loss, and For weights.

9. A UAV infrared small target detection system based on key information enhancement and progressive refinement of cross-level features, characterized in that, include: The collection unit is used to collect the training image dataset; The first detection unit is used to input the training image dataset into a preset infrared small target detection model to obtain the detection results; The infrared small target detection model enhancement includes a backbone, an encoder, and a decoder; the backbone includes multiple sequentially connected multi-path dynamic weighted attention modules, each of which includes a first path, a second path, a third path, and a fusion layer, wherein the second path includes two parallel branches; The training unit is used to train and optimize the infrared small target detection model enhancement based on the loss function, so as to obtain the optimal infrared small target detection model enhancement. The second detection unit is used to input the target image into the optimal infrared small target detection model enhancement to obtain the final detection result.

Images