Infrared small target detection method based on enhanced dynamic deformable convolution

Abstract

The invention relates to the technical field of computer vision, and particularly discloses an infrared small target detection method based on enhanced dynamic deformable convolution, which comprises the following steps of: constructing an encoder-decoder reference network, replacing standard convolution with the enhanced dynamic deformable convolution in an encoder, fusing multi-scale features through a dynamic routing mechanism, and detecting a small target based on the enhanced dynamic deformable convolution. And a direction perception offset enhancement module is utilized to guide a convolution sampling point to accurately position a small target, and background noise is effectively suppressed. In addition, a semantic refinement module is introduced in an encoder-decoder transition stage, and deep semantics are enhanced through multi-scale deep convolution and channel shuffling and are focused on a small target. Experiments show that compared with the prior art, the method achieves higher detection precision and lower false alarm rate on a plurality of public data sets, and is especially suitable for infrared small target accurate detection under a complex background.

Description

Infrared Small Target Detection Method Based on Enhanced Dynamically Deformable Convolution Technical Field

[0001] This invention relates to the fields of computer vision and infrared image processing technology, and in particular to an infrared small target detection method based on enhanced dynamic deformable convolution. Background Technology

[0002] Infrared small target detection has wide applications in many fields such as military reconnaissance, security monitoring, and disaster early warning. However, due to factors such as long imaging distance, extremely small target size (usually only a few pixels), low signal-to-noise ratio, and complex background interference, accurate detection of infrared small targets has always been a highly challenging issue.

[0003] Traditional infrared small target detection methods, such as those based on filtering, human visual system modeling, or low-rank matrix recovery, often exhibit high false alarm or false negative rates when dealing with complex backgrounds and variable target shapes. In recent years, convolutional neural network (CNN)-based methods, particularly U-Net and its variants, have made significant progress in this field. However, existing CNN-based methods still suffer from the following major problems: 1. Limitations of fixed convolution kernels: Traditional convolution operations use kernels with fixed shapes (e.g., 3×3). When the target size is smaller than the kernel, the convolution operation covers a large area of ​​background, inevitably introducing severe background noise during feature extraction and drowning out weak target signals.

[0004] 2. Shortcomings of existing deformable convolutions: Deformable convolutions introduce offsets to make the sampling points no longer regularly distributed, which theoretically can better fit the target shape. However, for extremely small infrared targets, standard deformable convolutions have difficulty learning effective offsets, and their sampling points may be randomly distributed or even deviate from the target, leading to feature extraction failure.

[0005] 3. Loss of deep semantic information: In the encoder-decoder structure, as the number of downsampling times increases, the resolution of deep feature maps decreases, and the detailed information of small targets is almost lost, making it difficult for the network to recover and accurately locate small targets during the decoding stage.

[0006] Therefore, there is an urgent need to design a novel convolution operation that minimizes background noise during feature extraction and enhances the network's ability to extract and preserve features of small targets. This is a problem that urgently needs to be solved under the current circumstances. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides an infrared small target detection method based on enhanced dynamic deformable convolution to solve the problems mentioned in the background art.

[0008] The infrared small target detection method based on enhanced dynamic deformable convolution includes the following steps: Step S1: Construct a baseline network with an encoder-decoder structure, and replace the standard convolution in the encoder of the baseline network with enhanced dynamic deformable convolution to adaptively adjust the position of the convolution sampling points during the feature extraction stage to reduce the introduction of background noise; Step S2: In the enhanced dynamic deformable convolution, integrate a direction-aware offset enhancement module to locate the boundary of small targets through directional pooling operations to enhance the ability of the convolution kernel sampling points to offset towards the small target region; Step S3: In the transition stage between the encoder and decoder of the baseline network, introduce a semantic refinement module to enhance deep semantic information and focus it on small targets through multi-branch depthwise separable convolution and channel shuffling; Step S4: Train the enhanced dynamic deformable convolution network using a training dataset containing infrared small targets, and use the trained model to detect small targets in the input infrared image.

[0009] As a further optimization of the infrared small target detection method based on enhanced dynamic deformable convolution, the enhanced dynamic deformable convolution in step S1 specifically includes the following steps: S11: Generate dynamic weights corresponding to multi-scale convolution paths based on input features through a dynamic routing mechanism; S12: Use dynamic weights to perform weighted fusion of the outputs of 1×1, 3×3, 5×5 and Gaussian convolution paths, and introduce a modulation factor for each sampling point to suppress the influence of error offset.

[0010] As a further optimization of the infrared small target detection method based on enhanced dynamic deformable convolution, the dynamic routing mechanism in step S11 specifically involves performing global max pooling, convolution, and sigmoid activation function processing on the input feature map in sequence to generate the dynamic weights.

[0011] As a further optimization of the infrared small target detection method based on enhanced dynamic deformable convolution, the direction perception offset enhancement module in step S2 specifically performs the following operations: S21: Perform global max pooling on the input feature map in the X and Y directions and concatenate them; S22: Perform convolution operation on the concatenated features and generate an attention map through the Sigmoid function, multiply it with the original features to enhance the perception of the boundary position of the small target, thereby generating a more accurate sampling point offset.

[0012] As a further optimization of the infrared small target detection method based on enhanced dynamic deformable convolution, the semantic refinement module in step S3 specifically includes: S31: extracting multi-scale semantic features using depth-separable convolution branches with different dilation rates; S32: enhancing feature interaction and information flow between the encoder and decoder through channel shuffling.

[0013] Beneficial Effects Compared with the prior art, the present invention has obvious advantages and beneficial effects, can achieve considerable technical progress and practicality, and has a wide range of application value. It has at least the following advantages: (1) The EDDNet network proposed in this invention systematically solves the problems of background noise interference, difficulty in extracting small target features and loss of deep semantics in infrared small target detection by introducing enhanced dynamic deformable convolution, orientation-aware offset enhancement module and semantic refinement module.

[0014] (2) Experiments on three public datasets, NUAA-SIRST, NUDT-SIRST, and IRSTD-1K, show that the method of this invention outperforms existing state-of-the-art methods in the key metric of Intersection over Union (IoU) while maintaining a low false alarm rate. For example, on the NUAA-SIRST dataset, the IoU reaches 79.289%, which is a significant improvement over the baseline model. Attached Figure Description

[0015] Figure 1 is an example of an infrared small target image provided in an embodiment of the present invention; Figure 2 is an overall network structure diagram of EDDNet proposed in an embodiment of the present invention; Figure 3 is a structure diagram of enhanced dynamic deformable convolution in an embodiment of the present invention; Figure 4 is a structure diagram of the orientation-aware offset enhancement module in an embodiment of the present invention; Figure 5 is a structure diagram of the semantic refinement module in an embodiment of the present invention; Figure 6 is a comparison diagram of the detection performance of the present invention on the NUAA-SIRST dataset with the original model; Figure 7 is a comparison diagram of the detection performance of the present invention on the NUDT-SIRST dataset with the original model; Figure 8 is a comparison diagram of the detection performance of the present invention on the IRSTD-1K dataset with the original model; Figure 9 is a comparison diagram of the detection performance of the present invention on the NUAA-SIRST dataset. Figure 10 is a 3D comparison of the detection performance of the original model on the NUDT-SIRST dataset in this embodiment of the invention; Figure 11 is a 3D comparison of the detection performance of the original model on the IRSTD-1K dataset in this embodiment of the invention; Figure 12 is a comparison of the number of parameters and detection accuracy (IoU) of different network models on the NUDT-SIRST dataset in this embodiment of the invention; Figure 13 is a comparison of the number of parameters and detection accuracy (IoU) of different network models on the NUDT-SIRST dataset in this embodiment of the invention; Figure 14 is a comparison of the number of parameters and detection accuracy (IoU) of different network models on the IRSTD-1K dataset in this embodiment of the invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments and accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of protection. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art, without creative effort, including formal modifications to the technical solutions described in the following embodiments or equivalent substitutions of some technical features, based on the inspiration of the present invention, are within the scope of protection of the present invention.

[0017] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0018] Example 1 This embodiment of the invention provides an infrared small target detection method based on enhanced dynamic deformable convolution. The overall network structure (EDDNet) is shown in Figure 2. It uses U-Net as the backbone network and mainly includes three core innovative modules.

[0019] Step S1: Constructing the Network and Applying Enhanced Dynamically Deformable Convolutions. First, a U-Net network with an encoder-decoder structure is constructed. In the encoder stage, instead of using traditional standard convolutions, the enhanced dynamically deformable convolutions designed in this invention are used entirely for feature extraction.

[0020] Referring to Figure 3, the workflow of the enhanced dynamically deformable convolution is as follows: S11: The input feature map F_x first enters the dynamic routing module. This module performs global max pooling on F_x, then passes it through a convolutional layer and a sigmoid activation function, outputting a set of dynamic weights W_d. This process can be represented as: .

[0021] S12: The input feature map F_x is fed into an enhanced deformable convolution branch (EDConv2d_k) with kernel sizes of 1, 3, and 5, and a Gaussian convolution branch (Conv2d_gas), respectively. The enhanced deformable convolution is based on the standard deformable convolution formula. Based on this, a modulation factor M(b_n) is introduced for each sampling point, and the formula is updated to... The modulation factor M(b_n) and the offset Δb_n are learned together through the network and are used to enhance or suppress the contribution of the sampling point to the output.

[0022] S13: Concatenate the features F_x1, F_x3, F_x5, and F_g from the four branches, and then perform a weighted fusion using the weights W_d generated by dynamic routing to obtain the final output F_d. That is: .

[0023] Step S2: Integrating the orientation-aware offset enhancement module. To improve the offset learning capability of the enhanced deformable convolution in step S1, especially for the localization of very small targets, we introduced an orientation-aware offset enhancement module, the structure of which is shown in Figure 4.

[0024] S21: The input to this module is the feature map F_x. First, global max pooling (MaxPool_x, MaxPool_y) is performed on it in the X and Y directions respectively to obtain feature vectors in the two directions.

[0025] S22: Concatenate the feature vectors from these two directions to obtain F_gapxy.

[0026] S23: Perform a convolution operation on F_gapxy to obtain F_convxy.

[0027] S24: F_convxy is passed through a Sigmoid gating mechanism to generate an attention map, which is then multiplied with the original features to obtain the final output F_out. This output is used to guide and optimize the offset of sampling points in deformable convolutions, making them more inclined to point to the regions where small targets are located. This process can be described as follows: .

[0028] Step S3: Introducing a semantic refinement module. In the transition stage between the encoder output and the decoder input, we introduce a semantic refinement module, the structure of which is shown in Figure 5.

[0029] S31: The input to this module is the deep, low-resolution feature map output by the encoder.

[0030] S32: Use multiple parallel depthwise separable convolutional layers with different dilation rates (e.g., 1, 3, 5) to obtain multi-scale contextual information without significantly increasing the number of parameters.

[0031] S33: Fuse the features output from each branch and perform channel shuffling. Channel shuffling can shuffle the order of feature channels, promote information interaction between features from different branches, and enhance the model's representation ability and generalization performance.

[0032] S34: Output the semantically enhanced feature map to the decoder for subsequent upsampling and target localization.

[0033] Step S4: Model Training and Detection S41: The proposed EDDNet is trained using publicly available infrared small target datasets (such as NUAA-SIRST, NUDT-SIRST, IRSTD-1K). The SoftIoU loss function and Adagrad optimizer are used during training.

[0034] S42: Input the infrared image to be detected into the trained model, and the network output is the predicted segmentation map of the small target.

[0035] S43: As shown in Figure 6-14, by comparing with the original model and other advanced methods, it can be seen intuitively and quantitatively that the method proposed in this invention has significant advantages in both detection accuracy and suppression of false alarms.

[0036] The preferred embodiments and examples of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments and examples. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the concept of the present invention.

Claims

1. An infrared small target detection method based on enhanced dynamic deformable convolution, characterized in that: The steps include: Step S1: Constructing a baseline network with an encoder-decoder structure, and replacing the standard convolution in the encoder of the baseline network with an enhanced dynamically deformable convolution, which is used to adaptively adjust the position of the convolution sampling points during the feature extraction stage to reduce the introduction of background noise; Step S2: In the enhanced dynamic deformable convolution, an orientation-aware offset enhancement module is integrated to locate the boundary of small targets through directional pooling operations, thereby enhancing the ability of the convolution kernel sampling points to offset towards the small target region; Step S3: In the transition stage between the encoder and decoder of the baseline network, a semantic refinement module is introduced to enhance the deep semantic information and focus it on small targets through multi-branch depthwise separable convolution and channel shuffling; Step S4: The enhanced dynamic deformable convolutional network is trained using a training dataset containing infrared small targets, and the trained model is used to detect small targets in the input infrared image.

2. The infrared small target detection method based on enhanced dynamic deformable convolution as described in claim 1, characterized in that: The enhanced dynamic deformable convolution in step S1 specifically includes the following steps: S11: Generate dynamic weights corresponding to multi-scale convolution paths based on input features through a dynamic routing mechanism; S12: Use dynamic weights to perform weighted fusion of the outputs of 1×1, 3×3, 5×5 and Gaussian convolution paths, and introduce a modulation factor for each sampling point to suppress the influence of error offset.

3. The infrared small target detection method based on enhanced dynamic deformable convolution as described in claim 2, characterized in that: The dynamic routing mechanism in step S11 specifically involves sequentially performing global max pooling, convolution, and sigmoid activation on the input feature map to generate the dynamic weights.

4. The infrared small target detection method based on enhanced dynamic deformable convolution as described in claim 1, characterized in that: The direction-aware offset enhancement module in step S2 specifically performs the following operations: S21: Perform global max pooling on the input feature map in the X and Y directions and concatenate them; S22: Perform convolution operation on the concatenated features and generate an attention map through the Sigmoid function, which is then multiplied with the original features to enhance the perception of the boundary position of small targets, thereby generating a more accurate sampling point offset.

5. The infrared small target detection method based on enhanced dynamic deformable convolution as described in claim 1, characterized in that: The semantic refinement module in step S3 specifically includes: S31: extracting multi-scale semantic features using depthwise separable convolutional branches with different dilation rates; S32: enhancing feature interaction and information flow between the encoder and decoder through channel shuffling.

Images