Infrared small target detection method based on multi-scale rotation deformable attention module

Abstract

The application discloses an infrared small target detection method based on a multi-scale rotating deformable attention module, and comprises the following steps: 1, collecting infrared images; 2, extracting and enhancing the high-frequency feature representation of the original infrared image; 3, performing deep feature extraction on the high-frequency feature representation, adopting a feature pyramid network to construct a multi-scale feature representation, and then performing feature enhancement to obtain multi-scale enhanced features with rich semantic information and spatial details; 4, adopting a multi-scale and multi-direction comprehensive module to extract the local spatial relationship between the multi-scale enhanced features, enhance the feature representation of the rotating target, and realize adaptive positioning and identification of the target in different directions and sizes; 5, dynamically adjusting the target feature weight, obtaining features fused with multi-scale, spatial and task adaptive information, and then adopting a detection head to output target categories and rotating frame parameters, and performing training and optimization of the detection head. The application improves the detection precision of the infrared rotating target.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer vision and relates to an infrared small target detection method based on a multi-scale rotating deformable attention module. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, target detection technology, as an important research direction in the field of computer vision and remote sensing image processing, has important application value in reconnaissance, security monitoring, industrial detection and other fields. In recent years, methods based on deep learning, such as YOLO and Faster R-CNN, have significantly improved the accuracy of target detection by automatically learning multi-level features. Nowadays, researchers enhance the feature expression of small targets by introducing attention mechanisms or improve computational efficiency by combining lightweight design. In addition, multi-modal fusion and spatio-temporal information modeling can further improve the detection robustness in complex scenes. To address the detection challenges posed by rotating targets, existing research has proposed a series of improvement methods, mainly including anchor box design, feature extraction optimization and loss function optimization. For anchor box design, researchers have added an angle variable to the traditional four-coordinate anchor box, using a rotating anchor box to improve the adaptability of the model to rotating targets. This method allows the anchor box to adaptively adjust according to the rotation angle of the target, thereby improving the detection of rotating targets. In addition, by dynamically adjusting the position of the sampling points through deformable convolution and weighting different sampling points through the adjustment mechanism, the characteristics of the rotating target can be better fitted, which significantly improves the feature representation ability of the target and makes the detection of rotating targets more accurate. In addition, to address the boundary positioning problem of rotating targets, one of the commonly used methods is to improve the regression loss function and focus on the geometric characteristics of rotating targets for optimization.

[0003] Although existing methods have made some progress in rotating target detection, there are still significant shortcomings. First, methods based on rotating anchor boxes rely on predefined angle intervals and geometric priors, making it difficult to adapt to the continuous distribution of target rotation angles. Second, although deformable convolution can fit the contour of rotating targets by dynamically shifting sampling points, its local receptive field limits the model's ability to model global rotation dependencies. In addition, existing rotating loss functions have the problems of ambiguous optimization direction and slow convergence speed, making it difficult to handle angle periodicity ambiguity, so it is particularly important to incorporate multi-scale rotating detail information features for infrared target detection. SUMMARY

[0004] To solve the above technical problems in the prior art, the present application proposes an infrared small target detection method based on a multi-scale rotating deformable attention module, and the specific technical solutions are as follows:

[0005] An infrared small target detection method based on a multi-scale rotation deformable attention module, comprising the following steps:

[0006] Step 1: Collecting an infrared image with small targets;

[0007] Step 2: Using a high-frequency direction injection module to extract and enhance the high-frequency feature representation of the original infrared image;

[0008] Step 3: Using a backbone network to perform deep feature extraction on the high-frequency feature representation, then using a feature pyramid network to construct a multi-scale feature representation, and introducing a path aggregation network for feature enhancement, outputting a multi-scale enhanced feature with rich semantic information and spatial details;

[0009] Step 4: Using a multi-scale multi-direction comprehensive module to extract the local spatial relationship between the multi-scale enhanced features, enhance the feature representation of the rotating target, and realize adaptive positioning and recognition of the target in different directions and sizes;

[0010] Step 5: Using a dynamic head to dynamically adjust the weight of the target feature through three attention networks of scale perception, spatial perception and task perception, obtain the fusion feature, and then output the target class and rotation box parameters through the detection head, and use the loss function to optimize and train the detection head.

[0011] Further, in step 2, the high-frequency direction injection module extracts the low-frequency background component of the input original infrared image through a Gaussian low-pass filter, and obtains the high-frequency part through the difference operation of the original image and the low-frequency component. Perform wavelet transform to enhance multi-scale perception, extract multiple scale high-frequency subbands, and use three groups of direction differential kernels to perform parallel convolution operation on the high-frequency part to obtain multi-direction gradient features. The operation includes horizontal kernel strengthening vertical edges, vertical kernel enhancing horizontal edges, and diagonal kernel capturing 45° direction features. Then, the multi-direction gradient features and the multiple scale high-frequency subbands are adaptively spliced through attention mechanism to obtain high-frequency fusion features. The fusion features are globally averaged and pooled, then MLP is used to generate weights. Finally, the weighted high-frequency fusion features and the low-frequency features are weighted and fused to obtain the enhanced high-frequency feature representation.

[0012] Further, the step 3 specifically comprises:

[0013] Step 3.1: The backbone network uses a ResNet34 improved structure. The enhanced high-frequency feature representation is passed through a 7×7 convolution with a stride of 2 and padding of 3 to obtain feature C1. Then, C1 is processed by three residual blocks and max pooling to obtain C2. C2 is processed by four residual blocks to obtain feature C3, where the first residual block has a stride of 2 and the remaining residual blocks have a stride of 1. Then, C3 is processed by six residual blocks to obtain feature C4. Finally, C4 is processed by three residual blocks to obtain feature C5. The residual blocks maintain a channel count of 64.

[0014] Step 3.2: Connect the features C3, C4, and C5 generated by the backbone network with the feature pyramid network, and use a 1×1 convolution to adjust the number of channels of feature C5 to 256 to generate feature T5;

[0015] Step 3.3: Upsample T5 to 32×32, adjust the number of channels of C4 to 256 using 1×1 convolution, and then perform element-wise addition. Add the upsampled T5 with the 1×1 convolutioned C4 to generate T4. Similarly, add the upsampled T4 with the 1×1 convolutioned C3 to generate T3.

[0016] Step 3.4: Next, feature refinement is performed on each layer T. i 3×3 convolution is used to eliminate upsampling aliasing, and a multi-scale feature representation with rich semantic information is constructed.

[0017] Step 3.5: Connect the features T3, T4, and T5 generated by the feature pyramid network with the path aggregation network. First, initialize the output feature L3 of the path aggregation network. Then, perform a 3×3 convolution and downsampling with a stride of 2 on L3 and add it element-wise with T4 to generate feature L4. Similarly, perform a 3×3 convolution and downsampling with a stride of 2 on L4 and add it element-wise with T5 to generate feature L5.

[0018] Furthermore, in step 1, the multi-scale, multi-directional integration module includes a multi-scale local relation learning module. This module performs four-branch calculations and concatenates the results of each branch to extract the local spatial relationships between the multi-scale enhanced features. The four branches include:

[0019] First branch: 1×1 convolutional branch;

[0020] The second branch is a lightweight 3×3 convolution branch, which divides the input feature channels into four groups for parallel processing. Each group performs a 3×3 convolution operation independently, followed by dilated convolution operations.

[0021] The third branch: 5×5 convolution branch, which uses 5×5 depthwise separable convolution and performs dilated convolution operation;

[0022] The fourth branch: First, a 3×3 convolution operation is performed, followed by a dilated convolution operation.

[0023] Furthermore, the multi-scale, multi-directional integrated module includes a multi-scale rotationally deformable attention module. This module constructs a rotation matrix by introducing a rotation angle θ, and uses the rotation matrix to perform rotational transformation on the target reference information to maintain the complete spatial information of the target. Subsequently, a multi-head attention architecture is used to extract multi-dimensional features of the target in parallel. Each attention head dynamically adjusts the distribution of sampling points in conjunction with the rotation matrix, and an adaptive weighted fusion mechanism is used to adapt the target features of different rotational forms. Finally, after Softmax normalization processing, an enhanced feature representation of the rotating target is output.

[0024] Furthermore, the multi-scale, multi-directional integration module includes a multi-directional feature perception module. This module employs four filters—horizontal, vertical, diagonal, and low-frequency—to obtain the feature representation of the rotating target in each direction. Next, the obtained components are processed using global average pooling to capture statistical features and global max pooling to retain salient features. After fusion along the channel dimension, fine-grained components are formed to locate the position of the rotating target. Then, the obtained fine-grained components are used to focus on the original feature map, thereby obtaining the feature map of the focused position in each direction. Finally, the feature maps obtained in the four directions are fused to output a multi-directional fused enhanced feature.

[0025] Furthermore, in step 5, the dynamic head first calculates the global average of all pyramid-level features through a scale-aware attention network, and generates weights through 1×1 convolution and a Hard Sigmoid function to dynamically fuse features from different levels, achieving dynamic feature fusion based on the importance of features at each level. Secondly, it uses a spatially aware attention network to fuse features from different levels at the same spatial location using deformable convolution, and enhances the spatial information of the target by learning offset and importance scalars. Then, it uses task-aware attention to dynamically adjust the switching state of channels through a hyperfunction to support the needs of different tasks. Finally, it outputs features that fuse multi-scale, spatial, and task-adaptive information.

[0026] Furthermore, in step 5, the detection head uses a dual-branch processing approach with a classification branch and a regression branch. First, feature alignment and channel unification are performed using a 3×3 convolution. The classification branch uses four-level convolutional blocks for feature refinement, with each level containing a 3×3 convolution + BN + ReLU, progressively refining the feature representation. Finally, a spatial class prediction map is output through a 1×1 convolution, and the class confidence map for each spatial location is obtained by activating it using the Sigmoid activation function. The regression branch outputs a regression parameter map containing center point offset and rotation angle through a parallel four-level convolutional block structure. Finally, the class confidence map output by the classification branch is combined with non-maximum suppression to filter low-confidence predictions, and the regression parameter map output by the regression branch is decoded to generate the final rotated bounding box parameters.

[0027] Furthermore, the loss function used in the classification branch is the adaptive threshold focus loss, which adaptively adjusts the loss value based on the predicted class confidence value.

[0028] Furthermore, the regression branch uses the KFIoU loss function. First, the rotated rectangle is converted into a Gaussian distribution. Then, the center loss forces the Gaussian distributions of the two rectangles to share the same mean. Subsequently, based on the Kalman filter principle, the two concentric Gaussian distributions are multiplied to obtain a new Gaussian distribution for the overlapping region. At the same time, by converting the calculation of the geometric intersection area into an algebraic operation of the covariance matrix, it is ensured that the calculation can still be performed effectively even when the targets do not actually overlap. Finally, by inverting the parameters of the three key Gaussian distributions: the predicted box, the ground truth box, and their intersection, the equivalent rotated rectangle is reconstructed, and the approximate SkewIoU value is further derived.

[0029] The advantages and beneficial effects of this invention are as follows:

[0030] (1) This invention has the advantage of rotating target recognition. By proposing a multi-scale rotating deformable attention module to integrate multi-scale input features, it enhances size adaptability. In the attention calculation, it integrates multi-dimensional information such as spatial position, rotation angle and deformation, and embeds the rotation matrix into the deformable attention module mechanism, which effectively improves the target detection accuracy.

[0031] (2) This invention achieves a breakthrough in small target detection by proposing a high-frequency directional injection network that significantly improves the feature retention rate. By employing a directional filtering mechanism in the initial stage, this network effectively suppresses background clutter interference in infrared images and precisely enhances the edge detail features of small targets.

[0032] (3) The multi-scale local relation learning module proposed in this invention significantly improves the network's ability to detect small targets through a multi-level feature interaction mechanism. By constructing a multi-scale receptive field fusion mechanism, the network can adaptively process small targets of different sizes, fully preserve the fine features of the targets, effectively improve the network's feature representation ability for small-scale targets, and provide a new technical path for small target detection tasks. Attached Figure Description

[0033] Figure 1 This is a flowchart of an infrared small target detection method based on a multi-scale rotating deformable attention module according to an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the high-frequency directional injection module in an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the structure of the multi-scale local relation learning module in an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of the structure of the multi-scale rotational deformable attention module in an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of the structure of the multi-directional feature perception module in an embodiment of the present invention;

[0038] Figure 6 These are experimental visualization comparison diagrams from embodiments of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0040] like Figure 1 As shown in this embodiment, an infrared small target detection method based on a multi-scale rotationally deformable attention module introduces additional angle information into the calculation of sampling point offset by using a multi-scale rotationally deformable attention module, enabling the model to have better feature capture capability for targets with arbitrary orientations. The method specifically includes the following steps:

[0041] Step 1: Collect raw infrared images with small targets.

[0042] In this embodiment of the invention, image I∈R(H×W×1), where H is the image height and W is the image width.

[0043] Step 2: Input the image to the high-frequency direction injection module. Based on the high-frequency direction prior feature injection mechanism, the high-frequency direction features of the injected image are used as prior knowledge to improve the network's ability to perceive detailed features such as the edge contour and morphological structure of infrared small targets. The high-frequency direction features of the original image are extracted and fused to output the enhanced high-frequency feature representation.

[0044] Among them, such as Figure 2 As shown, the implementation of the high-frequency directional injection module is as follows:

[0045] First, the input raw infrared image is smoothed using a Gaussian low-pass filter. This operation preserves the low-frequency background components of the image while suppressing high-frequency noise. Next, the high-frequency components are obtained through a difference operation between the raw image and the low-frequency components. The resulting residual image contains high-frequency features such as target edges and texture shapes. Parallel convolution operations are performed on the high-frequency components using three sets of directional differential kernels: a horizontal kernel enhances vertical edges, a vertical kernel enhances horizontal edges, and a diagonal kernel captures features in the 45° direction. A Haar wavelet transform is performed on the high-frequency residual, decomposing it into four sub-bands. Six types of features, including three directional gradients and three wavelet sub-bands, are adaptively fused using an attention mechanism. The fused features are then weighted using global average pooling followed by an MLP (Multi-Level Processing) to generate weights. Finally, the weighted features are linearly combined with the low-frequency components to output F. out .

[0046] The relevant calculation formulas for the high-frequency directional injection module are as follows:

[0047] ,

[0048] ,

[0049] ,

[0050] ,

[0051] ,

[0052] ,

[0053] ,

[0054] ,

[0055] ,

[0056] ,

[0057] ,

[0058] Among them, F oriLet G(x,y) represent the original infrared image, G(x,y) represent the Gaussian filtering operation, (x,y) represent the pixel coordinates, σ represent the standard deviation of the Gaussian kernel, z represent the kernel radius, and F represent the pixel coordinates. low F represents the low-frequency features extracted after Gaussian filtering. high K represents the high-frequency residual. dir This indicates a 3×3 directional core group, including horizontal, vertical, and diagonal directions. F dir This represents the features after convolution with directional kernel groups. `Approx()` indicates that the image's rows and columns are filtered separately using a low-pass filter to obtain low-frequency approximate components. `HighPass()` represents... h This indicates that the image is low-pass filtered by rows, and then filtered by columns. HighPass v This indicates that the image is filtered by rows, and then low-pass filtered by columns. (HighPass) d This indicates that both rows and columns of the image are filtered to extract high-frequency details along the diagonal. LL, LH, HL, and HH represent wavelet transform multi-scale decomposition, where LL is the low-frequency subband, LH is the horizontal detail, HL is the vertical detail, and HH is the diagonal detail. F c This indicates channel concatenation processing, where w represents the channel weights generated by global average pooling followed by MLP, and F represents the channel weights. out The weighted sum result is linearly combined with the low-frequency components, which is the enhanced high-frequency feature representation of the output.

[0059] Step 3: The enhanced high-frequency feature representation is used for deep feature extraction through a backbone network. Then, a feature pyramid network is employed to construct a multi-scale feature representation, fusing high-level semantic features with low-level detailed features. Based on this, a path aggregation network is further introduced to enhance information interaction between different levels of the feature pyramid. This hierarchical multi-scale feature fusion mechanism not only preserves the detailed information in high-frequency features but also enhances the network's ability to perceive targets at different scales, ultimately outputting multi-scale enhanced features with rich semantic information and spatial details. Specifically, this includes:

[0060] Step 3.1: The backbone network uses a ResNet34 improved structure. The features output by the high-frequency directional injection module are passed through the backbone network and subjected to a 7×7 convolution with a stride of 2 and padding of 3 to obtain feature C1. Then, C1 is processed by three residual blocks and max pooling to obtain C2. All residual blocks maintain the number of channels 64. C2 is processed by 4 residual blocks, with the first block having a stride of 2 and the rest having a stride of 1 to obtain C3. C3 is then processed by 6 residual blocks to obtain C4, and C4 is processed by 3 residual blocks to obtain C5.

[0061] The relevant calculation formulas for the backbone network are as follows:

[0062] ,

[0063] ,

[0064] ,

[0065] ,

[0066] ,

[0067] Among them, F out This indicates the output characteristics of the high-frequency directional injection module, Conv 7×7 () represents a 7×7 convolution operation, MaxPool() represents a max pooling operation, and ResBlock() represents a full pooling operation. 4 128 ( ) represents an operation with 4 residual blocks and 128 channels, ResBlock 6 256 With ResBlock 3 512 Similarly;

[0068] The specific calculation formula for the residual block operation is as follows:

[0069] ,

[0070] Where W1 represents the 1×1 convolution weights and W2 represents the 3×3 convolution weights. This indicates that ReLU is activated.

[0071] The operation of the feature pyramid network joint path aggregation network includes the following steps:

[0072] Step 3.2: Connect the C3, C4, and C5 features output by the backbone network to the feature pyramid, and uniformly adjust the channel dimension of C5 to 256 through 1×1 convolution to generate T5;

[0073] ,

[0074] Among them, Conv 7×7 ( ) represents a 7×7 convolution operation, C5 represents the output features of the backbone network, and T5 represents the output features of the feature pyramid network.

[0075] Step 3.3: Then, T5 is upsampled to 32×32, and C4 is adjusted to 256 channels by 1×1 convolution. Then, element-wise addition is performed. The upsampled T5 is added to the 1×1 convolutioned C4 to generate T4. Similarly, the upsampled T4 is added to the 1×1 convolutioned C3 to generate T3.

[0076] ,

[0077] Among them, Conv 1×1 () represents a 1×1 convolution operation, Upsample() represents an upsampling operation, T i+1 T represents the output features of the path aggregation network. i In the case of i, it is either 3 or 4, and C i This represents the output characteristics of the backbone network;

[0078] Step 3.4: To ensure feature quality, for each layer T i Using 3×3 convolution for feature refinement effectively eliminates the aliasing effect caused by the upsampling process, and finally constructs a multi-scale feature representation with rich semantic information;

[0079] Step 3.5: Connect T3, T4, and T5 generated by the feature pyramid network with the path aggregation network, starting with initializing L3;

[0080] L3=T3,

[0081] Wherein, T3 is the output feature of the feature pyramid network, and L3 is the output feature of the path aggregation network;

[0082] Then, L3 is generated by performing a 3×3 convolution with a stride of 2 and downsampling, and then added to T4 element by element. Similarly, L4 is generated by performing a 3×3 convolution with a stride of 2 and downsampling, and then added to T5 element by element.

[0083] ,

[0084] Among them, T i The feature pyramid network outputs features, and Downsample represents the downsampling operation. i represents the output features of the path aggregation network, where i is 3 or 4.

[0085] Step 4: Employ a multi-scale, multi-directional integrated module to extract local spatial relationships between multi-scale enhanced features, thereby enhancing the feature representation of the rotating target and enabling adaptive localization and recognition of the target under different orientations and sizes. The multi-scale, multi-directional integrated module includes a multi-scale local relationship learning module, a multi-scale rotationally deformable attention module, a multi-directional feature perception module, and an SE module.

[0086] Specifically, a multi-scale local relation learning module is used to deeply explore the local spatial correlations between the input multi-scale enhanced features. Building upon this, a multi-scale rotationally deformable attention module is employed, introducing a rotation matrix and using deformable convolution and rotation offset calculations to effectively enhance the network's feature representation ability for rotating targets, generating rotationally invariant deformable features. Subsequently, a multi-directional feature perception module is used to extract detailed information about the edges and shapes of small targets in multiple directions through a multi-branch feature extraction strategy, outputting multi-directional fused enhanced features, significantly improving the network's ability to recognize targets in different directions. Finally, SE dynamically adjusts the weight distribution of features in each channel, enabling the network to adaptively focus on features more discriminative to the current detection task, ultimately outputting the features and comprehensively improving the network's detection performance, achieving multi-level feature optimization from local details to global semantics.

[0087] Among them, such as Figure 3 As shown, the implementation of the multi-scale local relation learning module is as follows:

[0088] The multi-scale local relation learning module innovatively integrates the advantages of multi-scale convolution and extended convolution. This module performs four branches of computation: a 1×1 convolution branch; a lightweight 3×3 convolution branch, which divides the input feature channels into four groups for parallel processing, with each group independently performing a 3×3 convolution operation followed by dilated convolution; a 5×5 convolution branch, which uses 5×5 depthwise separable convolution followed by dilated convolution; and finally, a branch that first performs a 3×3 convolution operation followed by dilated convolution. This module effectively processes high-frequency directional features without compromising the ability to preserve small targets in deep features, providing a practical optimization path for deploying networks in resource-constrained environments.

[0089] The relevant calculation formulas for the multi-scale local relation learning module are as follows:

[0090] ,

[0091] ,

[0092] ,

[0093] ,

[0094] ,

[0095] Among them, F a F b、 F c、 F d GroupConv represents the feature mapping corresponding to each branch. 3×3，g=4This indicates a grouped convolution operation, where g=4 means the channels are divided into four groups. (DepthwiseSepConv) 5×5 DConv represents a 5×5 depthwise separable convolution. i This indicates a dilated convolution with a 3×3 kernel, where the subscript i indicates the unwinding rate. 1×1 Represents a 1×1 convolution operation, Conv 3×3 Represents a 3×3 convolution operation, Conv 5×5 This represents a 5×5 convolution operation. Indicates channel splicing operation, F o It represents the characteristics of local relationships at multiple scales.

[0096] like Figure 4 As shown, the implementation of the multi-scale rotationally deformable attention module is as follows:

[0097] The module takes four-dimensional parameter points (x, y, w, h) as input, representing the target's position and scale information, while also incorporating a rotation angle θ. This design allows the model to simultaneously process the target's spatial position, size, and rotation angle, thus better adapting to rotating target detection tasks. The module proposes a rotation-aware multi-head attention mechanism, which enhances the detection performance of rotating targets through the synergistic effect of geometric transformation and feature learning.

[0098] First, a rotation matrix is ​​constructed based on the rotation angle θ to perform a geometric transformation on the input coordinates, preserving the complete spatial information of the target. Then, a multi-head attention architecture is employed to extract multi-dimensional features in parallel. Each attention head dynamically adjusts the sampling point distribution in conjunction with the rotation matrix, and a weighted fusion mechanism adaptively represents the target under different rotational forms. Finally, after Softmax normalization, a rotation-robust detection result is output. This module integrates multi-scale input features to enhance size adaptability, combining spatial position, rotation angle, and deformation information in the attention calculation. This invention's design of embedding geometric transformation into the attention mechanism effectively solves the feature alignment problem in rotating target detection, significantly improving the model's adaptability to complex scenes.

[0099] The relevant calculation formulas for the multi-scale rotationally deformable attention module are as follows:

[0100] ,

[0101] ,

[0102] ,

[0103] ,

[0104] Where F iRepresents valid input features, Linear() represents a linear transformation operation, Q represents the query vector, and Offset... MLP This represents a multilayer perceptron operation. A represents the offset of the sampling point relative to the reference point. mqk For normalized attention weights, Let be a rotation matrix, where, Rdeformable MHA Representing a multi-scale rotationally deformable attention module mechanism, z q For the query, consider it as the query for calculating attention weights, where q is the corresponding index, k is the index corresponding to the key, m is the header index of the corresponding attention (i.e., which attention), and p... q To query the corresponding coordinates, which correspond to the sampled local reference point, W m This is a linear transformation performed after multiplying the attention value by the value. Value represents the interpolation point at the sampling point location.

[0105] The key feature of this multi-scale rotationally deformable attention module lies in its ability to not only add a rotation matrix to enhance the model's adaptability to rotating targets, but also combine a multi-head attention mechanism and dynamic sampling point adjustment to improve the model's feature representation capabilities when processing rotating targets. This structure significantly improves the detection accuracy of rotating targets and can effectively handle changes in targets at different rotation angles and scales, thereby providing more accurate target localization and recognition results.

[0106] like Figure 5 As shown, the implementation of the multi-directional feature perception module is as follows:

[0107] The multi-directional feature perception module passes the input features through four filters: horizontal, vertical, diagonal, and low-frequency, to obtain the feature map components for each direction. Next, after performing global average pooling and global max pooling on the obtained components, they are concatenated along the channel dimension to better locate small targets. Then, the obtained fine-grained components are used to focus on the original feature map, thus obtaining the feature map of the focused position in each direction. Finally, the feature maps obtained in the four directions are fused to obtain the final output F. output .

[0108] The relevant calculation formulas for executing the multi-directional feature perception module are as follows:

[0109] ,

[0110] ,

[0111] ,

[0112] ,

[0113] ,

[0114] Among them, F input F output Indicates the module's input and output, DefConv dir G represents a deformable convolution operation with orientation constraints. M This represents global max pooling, where C represents convolution and S represents sigmoid. This indicates the concatenation of channel dimensions. and These represent element-wise multiplication and element-wise addition, respectively.

[0115] Step 5: Utilize a feature enhancement architecture based on a triple attention mechanism, specifically a module called the dynamic head designed to improve the performance of multi-scale object detection. This module dynamically adjusts feature weights through three attention sub-networks: scale-aware, spatial-aware, and task-aware, achieving multi-level feature optimization. Finally, the optimized features are passed through the detection head, outputting the target's class probability and rotation bounding box parameters. End-to-end optimization training is then performed using a loss function, significantly improving detection accuracy and the localization accuracy of rotated targets.

[0116] The execution of the dynamic header includes the following steps:

[0117] Step 5.1: Given features, input the feature tensor F∈R(L×S×C);

[0118] Where L represents the number of pyramid layers, S represents the size of the feature, S = H×W, H and W represent the height and width of the feature, and C represents the number of channels.

[0119] A dynamic header can be represented as:

[0120] ,

[0121] in, , , Let L, S, and C represent the attention functions, respectively.

[0122] Step 5.2: Calculate the global average value of features at all pyramid levels through scale-aware attention, and generate weights through 1×1 convolution and Hard Sigmoid function to dynamically fuse features from different levels, thereby achieving dynamic feature fusion based on the importance of features at each level.

[0123] ,

[0124] Where f(·) is a 1 × 1 convolutional layer, It is a Hard activation function.

[0125] Step 5.3: Next, spatial awareness attention is used to fuse features from different levels at the same spatial location using deformable convolution, and the spatial information of the target is enhanced by learning offset and importance scalars;

[0126] ,

[0127] Where K is the number of sparse sampling locations, and It is learned from the input features. It is a self-learning spatial offset The position of the shift, It is position p k An important scalar for self-learning.

[0128] Step 5.4: Dynamically adjust the on / off state of the channel through a hyperfunction using task-aware attention;

[0129] ,

[0130] in It is a superfunction that controls the threshold. It is dimensionality-reduced in L×S dimensions by average pooling, then two fully connected layers and one normalization layer are used, and finally normalization is performed by the sigmoid activation function.

[0131] Step 5.5: The final output is a feature that integrates multi-scale, spatial, and task-adaptive information.

[0132] The detection head employs a dual-branch structure to process classification and regression tasks in parallel, as specifically implemented below:

[0133] First, multi-level feature alignment and channel unification are achieved through 3×3 convolutional layers. The classification branch employs a four-level cascaded convolutional module, with each level containing a combination of 3×3 convolution + BN + ReLU to progressively refine feature representation. Finally, a 1×1 convolution outputs a spatial class prediction map, and a sigmoid function activation is used to obtain the class confidence map for each spatial location. The regression branch uses a four-level convolutional structure to directly output a parameter map containing center point offset and rotation angle. These parameters will subsequently be decoded into the geometric representation of the rotated bounding box. In the output processing stage, the classification confidence map uses non-maximum suppression to filter high-confidence predictions, while the regression parameters, after geometric decoding, generate the final rotated target detection box. These two processes work together to accurately locate and classify the rotated target. Furthermore, an end-to-end optimization training is performed using a joint loss function of adaptive threshold focus loss and KFIoU loss, significantly improving detection accuracy and the accuracy of rotating target localization.

[0134] The loss function used in the classification branch is the adaptive threshold focus loss, and the calculation formula is as follows:

[0135] ,

[0136] Where p t p represents the predicted probability of the target class. c Let λ represent the smoothed probability, and λ represent the weight of the hard sample.

[0137] The regression branch uses the KFIoU loss function, a simpler and more efficient SkewIoU approximation loss. This method requires no additional hyperparameters and can be easily implemented using existing operators within a deep learning framework, thus improving target detection accuracy. First, the rotating rectangle... Convert to Gaussian distribution Then, by using a center loss to force the Gaussian distributions of the two bounding boxes to share the same mean, a product operation is performed on the two concentric Gaussian distributions based on the Kalman filter principle to obtain a new Gaussian distribution for the overlapping region. By transforming the calculation of the geometric intersection area into an algebraic operation of the covariance matrix, effective calculation is ensured even when the targets do not actually overlap. Finally, by inverting the parameters of three key Gaussian distributions—the predicted box, the ground truth box, and their intersection—an equivalent rotated bounding box is reconstructed, and the approximate SkewIoU value is further derived. The calculation formula is as follows:

[0138] ,

[0139] ,

[0140] ,

[0141] in, Let R represent the variance of the rotated rectangle after transforming it into a Gaussian distribution, and let R denote the rotation matrix. Let B1 be a diagonal matrix of eigenvalues, and B2 and B3 be different rotated rectangles in the figure. Calculate the area of ​​the rotating rectangle corresponding to the Gaussian distribution, which is the cumulative multiplication of the covariance eigenvalues.

[0142] To demonstrate the effectiveness of this method, the experiment used a publicly available dataset from IRSTD-1k, consisting of 1000 infrared images. Each image provides a pixel-level mask of the target. The dataset was divided into a training set (800 images) and a test set (200 images). The training set was used to train the model, and the model parameters were adjusted through an optimization algorithm to enable the model to learn features and patterns from the data. After the model training was completed, the test set was used to evaluate the quality metrics; the input dataset size was 512×512. The test results were then compared with publicly available network models in existing technologies. The proposed model framework was implemented in PyTorch.

[0143] Table 1. Test results of this invention compared to three other infrared small target detection methods.

[0144]

[0145] Of the five evaluation indicators, all are commonly used for infrared small target detection. The arrow pointing upwards indicates that the larger the value, the more accurate the detection, while the arrow pointing downwards indicates that the smaller the value, the more accurate the detection.

[0146] Figure 6 The diagram shows a visual comparison of the experimental results of the method of this invention and other methods. The results show that the infrared small target detection method proposed in this invention can obtain more complete and accurate detection results.

[0147] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting small infrared targets based on a multi-scale rotationally deformable attention module, characterized in that, Includes the following steps: Step 1: Collect infrared images with small targets; Step 2: Use a high-frequency directional injection module to extract and enhance the high-frequency feature representation of the original infrared image; Step 3: Using a backbone network, deep feature extraction is performed on the high-frequency feature representation. Then, a feature pyramid network is used to construct a multi-scale feature representation, and a path aggregation network is introduced for feature enhancement, outputting multi-scale enhanced features with rich semantic information and spatial details. Step 4: Employ a multi-scale, multi-directional integrated module to extract the local spatial relationships between multi-scale enhanced features, enhance the feature representation of the rotating target, and achieve adaptive localization and recognition of the target under different directions and sizes; The multi-scale, multi-directional integrated module includes a multi-scale rotationally deformable attention module and a multi-directional feature perception module. The multi-scale rotational deformable attention module constructs a rotation matrix by introducing a rotation angle θ. The rotation matrix is ​​used to rotate and transform the target reference information to maintain the complete spatial information of the target. Then, a multi-head attention architecture is used to extract multi-dimensional features of the target in parallel. Each attention head dynamically adjusts the distribution of sampling points in combination with the rotation matrix and adapts to the target features of different rotation forms through a weighted fusion mechanism. Finally, after Softmax normalization processing, the enhanced feature representation of the rotating target is output. The multi-directional feature perception module uses four filters—horizontal, vertical, diagonal, and low-frequency—to obtain the feature representation of the rotating target in each direction. Next, the obtained components are processed by global average pooling to capture statistical features and global max pooling to retain salient features. After fusion along the channel dimension, fine components are formed to locate the position of the rotating target. Then, the obtained fine components are used to focus on the original feature map, thereby obtaining the feature map of the focused position in each direction. Finally, the feature maps of the four directions are fused to output a multi-directional fused enhanced feature. Step 5: Employ a dynamic head and dynamically adjust the weights of target features through three attention networks: scale awareness, spatial awareness, and task awareness. Obtain fused features, then output the target category and bounding box parameters through the detection head using the fused features, and optimize the detection head using a loss function.

2. The infrared small target detection method according to claim 1, characterized in that, In step 2, the high-frequency directional injection module extracts low-frequency background components from the input raw infrared image through a Gaussian low-pass filter, and obtains the high-frequency part through the difference operation between the raw image and the low-frequency components. Wavelet transform is performed on the high-frequency part to enhance multi-scale perception, extracting high-frequency sub-bands at multiple scales. Parallel convolution operations are performed on the high-frequency part using three sets of directional differential kernels to obtain multi-directional gradient features. The operations include horizontal kernel to enhance vertical edges, vertical kernel to enhance horizontal edges, and diagonal kernel to capture 45° directional features. The multi-directional gradient features and high-frequency sub-bands at multiple scales are then adaptively concatenated through an attention mechanism to obtain high-frequency fusion features. The fusion features are then subjected to global average pooling and MLP to generate weights. Finally, the weighted high-frequency fusion features are weighted and fused with low-frequency features to obtain the enhanced high-frequency feature representation.

3. The infrared small target detection method according to claim 1, characterized in that, Step 3 specifically includes: Step 3.1: The backbone network uses a ResNet34 improved structure. The enhanced high-frequency feature representation is passed through a 7×7 convolution with a stride of 2 and padding of 3 to obtain feature C1. Then, C1 is processed by three residual blocks and max pooling to obtain C2. C2 is processed by four residual blocks to obtain feature C3, where the first residual block has a stride of 2 and the remaining residual blocks have a stride of 1. Then, C3 is processed by six residual blocks to obtain feature C4. Finally, C4 is processed by three residual blocks to obtain feature C5. The residual blocks maintain a channel count of 64. Step 3.2: Connect the features C3, C4, and C5 generated by the backbone network with the feature pyramid network, and use a 1×1 convolution to adjust the number of channels of feature C5 to 256 to generate feature T5; Step 3.3: Upsample T5 to 32×32, adjust the number of channels of C4 to 256 using 1×1 convolution, and then perform element-wise addition. Add the upsampled T5 with the 1×1 convolutioned C4 to generate T4. Similarly, add the upsampled T4 with the 1×1 convolutioned C3 to generate T3. Step 3.4: Next, feature refinement is performed on each layer T. i 3×3 convolution is used to eliminate upsampling aliasing, and a multi-scale feature representation with rich semantic information is constructed. Step 3.5: Connect the features T3, T4, and T5 generated by the feature pyramid network with the path aggregation network. First, initialize the output feature L3 of the path aggregation network. Then, perform a 3×3 convolution and downsampling with a stride of 2 on L3 and add it element-wise with T4 to generate feature L4. Similarly, perform a 3×3 convolution and downsampling with a stride of 2 on L4 and add it element-wise with T5 to generate feature L5.

4. The infrared small target detection method according to claim 1, characterized in that, In step 1, the multi-scale, multi-directional integration module includes a multi-scale local relation learning module. This module performs four-branch calculations and concatenates the results of each branch to extract the local spatial relationships between the multi-scale enhanced features. The four branches include: First branch: 1×1 convolutional branch; The second branch is a lightweight 3×3 convolution branch, which divides the input feature channels into four groups for parallel processing. Each group performs a 3×3 convolution operation independently, followed by dilated convolution operations. The third branch: 5×5 convolution branch, which uses 5×5 depthwise separable convolution and performs dilated convolution operation; The fourth branch: First, a 3×3 convolution operation is performed, followed by a dilated convolution operation.

5. The infrared small target detection method according to claim 1, characterized in that, In step 5, the dynamic head first calculates the global average of features across all pyramid levels using a scale-aware attention network, and generates weights through 1×1 convolutions and a Hard Sigmoid function to dynamically fuse features from different levels, achieving dynamic feature fusion based on the importance of features at each level. Secondly, a spatially-aware attention network uses deformable convolutions to fuse features from different levels at the same spatial location, enhancing the spatial information of the target by learning offsets and importance scalars. Then, a task-aware attention network dynamically adjusts the switching state of channels via a hyperfunction to support the needs of different tasks. Finally, it outputs features that fuse multi-scale, spatial, and task-adaptive information.

6. The infrared small target detection method according to claim 1, characterized in that, In step 5, the detection head uses a dual-branch processing method with a classification branch and a regression branch. First, feature alignment and channel unification are performed through 3×3 convolution. The classification branch uses four-level convolutional blocks to refine features, with each level containing 3×3 convolution + BN + ReLU, progressively refining the feature representation. Finally, a spatial class prediction map is output through 1×1 convolution, and the class confidence map for each spatial location is obtained by activating it with the Sigmoid activation function. The regression branch outputs a regression parameter map containing center point offset and rotation angle through a parallel four-level convolutional block structure. Finally, the category confidence map output by the classification branch is combined with non-maximum suppression to filter out low-confidence predictions, and the regression parameter map output by the regression branch is decoded to generate the final rotated bounding box parameters.

7. The infrared small target detection method according to claim 6, characterized in that, The loss function used in the classification branch is the adaptive threshold focus loss, which adaptively adjusts the loss value based on the predicted class confidence value.

8. The infrared small target detection method according to claim 6, characterized in that, The regression branch uses the KFIoU loss function. First, the rotated rectangle is converted into a Gaussian distribution. Then, the center loss forces the Gaussian distributions of the two rectangles to share the same mean. Subsequently, based on the Kalman filter principle, the two concentric Gaussian distributions are multiplied to obtain a new Gaussian distribution for the overlapping region. At the same time, by converting the calculation of the geometric intersection area into an algebraic operation of the covariance matrix, it is ensured that the calculation can still be performed effectively even when the targets do not actually overlap. Finally, by inverting the parameters of three key Gaussian distributions: the predicted box, the ground truth box, and their intersection, the equivalent rotated rectangle is reconstructed, and the approximate SkewIoU value is further derived.

Images