Infrared weak and small target detection method based on hybrid perception state space model

Abstract

The invention discloses an infrared weak and small target detection method based on a hybrid perception state space model. The method is specifically implemented according to the following steps: step 1, constructing an infrared weak and small target detection data set; step 2, constructing an encoder, wherein the encoder comprises a cross-scale shallow layer embedding module and a mixed feature mining module; step 3, constructing a decoder, wherein the decoder comprises a focusing feature fusion module and a full connection layer; and step 4, training the constructed detection network by using the data set, and realizing infrared weak and small target detection. According to the invention, a cross-scale shallow embedding module, a mixed feature mining module and a focusing feature fusion module are utilized; potential and possible target positions are fully perceived through a high-resolution image at a network inlet, the capability of distinguishing background clutters and noise is improved in a down-sampling stage, semantic information of a target is enhanced, and meanwhile, focusing and calibration of cross-layer features are completed in an up-sampling stage, so that continuous response to a captured target is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of image target detection methods, and in particular relates to an infrared dim small target detection method based on a hybrid perception state space model. Background Art

[0002] Infrared small target detection is a long-range detection technology with all-weather operation and good hiding properties. It has been widely used in various fields such as night navigation, remote sensing, military monitoring and search and rescue. However, traditional visual detection methods rely on researchers' manually designed features and hyperparameter selection, which can cause large fluctuations in detection results in complex and changeable actual scenes. In contrast, deep learning technology has shown stronger feature extraction capabilities and generalization compared to model-driven methods by adaptively learning and optimizing features on a large number of infrared small target images.

[0003] Deep learning models, especially convolutional neural networks, have achieved remarkable results in image recognition and feature extraction. For example, DNANet [Dense nested attention network for infrared small target detection [J]. IEEE Transactions on Image Processing, 2022, 32: 1745-1758.] uses dense nested interactions of features extracted across layers to make full use of the semantic and position information in cross-layer features, improving the detection performance of small and weak targets; AGPCNet [Attention-guided pyramid context networks for detecting infrared small target under complex background [J]. IEEE Transactions on Aerospace and Electronic Systems, 2023.] helps the model reduce false detections by perceiving targets at local scales and suppressing noise and clutter information. However, in order to extract multi-scale semantic information of the target, the above methods use a deep network structure, which will weaken some target features during multiple downsampling and feature fusion processes, resulting in target loss. Summary of the invention

[0004] The purpose of the present invention is to provide an infrared dim small target detection method based on a hybrid perception state space model, which solves the problem in the prior art that some target features are weakened during multiple downsampling and feature fusion processes using a deep network structure.

[0005] The technical solution adopted by the present invention is an infrared dim small target detection method based on a hybrid perception state space model, which is specifically implemented according to the following steps: Step 1: Construct infrared small target detection dataset; Step 2: Construct an encoder, which includes a cross-scale shallow embedding module and a hybrid feature mining module; Step 3: Build a decoder, which includes a focus feature fusion module and a fully connected layer; Step 4: Use the detection network constructed by training the data set to detect infrared dim small targets.

[0006] The technical solution of the present invention is also characterized in that: Step 2 is as follows: Use a cross-scale shallow embedding module to fully perceive the potential target location on the high-resolution image at the network entrance and suppress background-irrelevant noise and clutter information. Then use a hybrid feature mining module to perceive the local surrounding contrast of the target and conduct a comprehensive analysis of the global background environment.

[0007] The encoder in step 2 consists of 1 cross-scale shallow embedding module and 4 hybrid feature mining modules. After the cross-scale shallow embedding module, the channels of the original input 3-channel image are adjusted to increase the channel dimension to 32; then 4 hybrid feature mining modules are used to generate encoding feature maps with feature sizes of 128x128, 64x64, 32x32 and 16x16 respectively.

[0008] The cross-scale shallow embedding module first implements coarse-grained noise filtering through a convolution with a kernel size of 3×3 and a stride of 1. Then, the MaxPooling, Indetity, and AvgPooling operations are used to form cross-scale spatial features and adjust the channel dimension through convolution. The spatial information is aggregated and the Query, Key, and Value projections are generated at the same time. Next, the Query and Key projections are reshaped to generate the channel-wise self-attention matrix and normalized using the Softmax activation function to obtain the Attention Weight. The Attention Weight is then applied to the Value projection to weaken complex irrelevant background information and perceive the spatial position of potential targets. Finally, a depth-wise separable convolution with a kernel size of 3×3 is used to smooth the target features while balancing the model performance overhead.

[0009] The hybrid feature mining module consists of two sub-modules: the large kernel reconstruction module and the visual state space module.

[0010] The large kernel reconstruction module captures more subtle contrast fluctuations in localized pixel information, which helps the model distinguish between noise and targets at a fine-grained level while strengthening the semantic features of the target; the visual state space module introduces global context modeling capabilities, which can establish associations between spatial and semantic information on a global scale, thereby improving the model's ability to detect clustered targets.

[0011] The large kernel reconstruction module is an integrated three-branch structure. The three branches capture different scale information through convolution kernels of different sizes to obtain scale features. Different scale features are further agglomerated through channel splicing and 1x1 convolution to form cross-scale aggregation features. The receptive field is expanded by using dilated convolution with a convolution kernel size of 3×3, a step size of 1, and dilation rates of 2, 4, and 2 respectively. Finally, the output features are input into the final depth-wise separable convolution with a convolution kernel size of 3×3 to restore channel dependency. The visual state space module is a parallel processing dual-branch structure. In the main branch, the input features are first expanded through a linear layer, and then through a stack of depth-wise separable convolution, SiLU activation function, selective scanning module, and layer normalization operations to capture global target features. In the second branch, a linear layer is used to expand the channel and then activated by SiLU. Finally, the outputs of the two branches are element-wise multiplied and the channels are adjusted to be the same as the input channels through a linear layer.

[0012] Step 3 is specifically as follows: using the focused feature fusion module, by focusing and calibrating the cross-layer features between the encoder and the decoder, the inconsistency problem of semantics and texture details between features is improved, and the model is guided to continuously focus on the target semantic information and reconstruct the target edge structure features; then the fully connected layer is used to adjust the number of channels of the feature map and output the final detection image; the decoder, which is composed of 4 focused feature fusion modules and 1 fully connected layer, will generate decoding feature maps with feature sizes of 32x32, 64x64, 128x128 and 256x256 respectively after the 4 focused feature fusion modules, and then the number of feature map channels is adjusted to 1 through the fully connected layer.

[0013] The focused feature fusion module consists of two parallel stages: focusing and calibration. In the focusing stage, the feature map Enc from the encoder and the feature map Dec from the decoder are first spliced ​​along the channel, and then the fused information descriptor is given importance through average pooling, maximum pooling, shared perceptron and Sigmod activation function. The cross-level fusion features are further focused by element-wise multiplication and residual connection is used to reduce information loss, which promotes the model to continuously emphasize the target's noticed area; in the calibration stage, the edge structure information is captured from the two independent feature maps of the encoder and decoder respectively through convolution with a convolution kernel size of 3×3 and a step size of 1, and then feature calibration is performed through element-level merging combined with depthwise separable convolution to realize the model's reconstruction of target texture details; finally, the features obtained in the focusing and calibration stages are combined, and the enhanced fusion features will improve the model's accurate detection performance for weak targets.

[0014] In the fully connected layer, the input features are convolved with a convolution kernel size of 1×1 and a step size of 1, and the number of channels is adjusted to 1 to obtain the final infrared weak target detection map.

[0015] The beneficial effects of the present invention are: The present invention is based on the infrared dim small target detection method of the hybrid perception state space model. Through the cross-scale shallow embedding module, the hybrid feature mining module and the focus feature fusion module, the high-resolution image at the network entrance fully perceives the potential target position, improves the ability to distinguish background clutter and noise in the downsampling stage, strengthens the semantic information of the target, and completes the focus and calibration of the cross-layer features in the upsampling stage to achieve continuous response to the captured target. At the same time, when constructing a neural network using this method, by adjusting the training parameters and loss function to optimize the network, a better infrared dim small target detection effect can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the present invention; Figure 2 It is the overall network structure diagram of the present invention; Figure 3 It is a network structure diagram of the cross-scale shallow embedding module in the present invention; Figure 4 It is a network structure diagram of the hybrid feature mining module in the present invention; Figure 5 is a network structure diagram of the large core reconstruction module in the present invention; Figure 6 is a network structure diagram of the visual state space module in the present invention; Figure 7 It is a network structure diagram focusing on feature fusion in the present invention; Figure 8is a network structure diagram of the fully connected layer of the present invention; Fig. 9 It is a rendering of the present invention compared with other methods. DETAILED DESCRIPTION

[0017] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] The present invention is based on the infrared dim small target detection method of the hybrid perception state space model. Figure 1-2 As shown, the specific implementation steps are as follows: Step 1: Construct infrared small target detection dataset; Step 2: Construct an encoder, which includes a cross-scale shallow embedding module and a hybrid feature mining module; Step 3: Build a decoder, which includes a focus feature fusion module and a fully connected layer; Step 4: Use the detection network constructed by training the data set to detect infrared dim small targets.

[0019] Example 1 Infrared dim small target detection method based on hybrid perception state space model, wherein the data set in step 1 can use public data sets such as IRSTD-1K, NUAA-SIRST, NUDT-SIRST or DenseSIRST, and randomly divide the data set into training set and test set in a ratio of 8:2, use random horizontal and vertical flipping and rotation operations to perform data enhancement, and crop the image size to 256×256.

[0020] Example 2 Infrared weak small target detection method based on hybrid perception state space model, where step 2 is specifically: using a cross-scale shallow embedding module to fully perceive the potential target position on the high-resolution image at the network entrance and suppress background irrelevant noise and clutter information, and then using a hybrid feature mining module to perceive the local surrounding contrast of the target and conduct a comprehensive analysis of the global background environment. The encoder consists of 1 cross-scale shallow embedding module and 4 hybrid feature mining modules. After the cross-scale shallow embedding module, the channels of the original input 3-channel image are adjusted to increase the channel dimension to 32; then 4 hybrid feature mining modules are used to generate encoding feature maps with feature sizes of 128x128, 64x64, 32x32 and 16x16 respectively.

[0021] Example 3 Infrared dim small target detection method based on hybrid perception state space model, in step 2, the input image is input into the first cross-scale shallow embedding module to perceive the potential target position on the high-resolution image and suppress background noise and clutter, and then by using 4 hybrid feature mining modules, a coding feature map that enhances the semantic features of the target is obtained.

[0022] Among them, Figure 3 As shown in the figure, the cross-scale shallow embedding module first implements coarse-grained noise filtering through a convolution with a kernel size of 3×3 and a stride of 1. Then, the MaxPooling, Indetity, and AvgPooling operations are used to form cross-scale spatial features and adjust the channel dimension through convolution. The spatial information is aggregated and the Query, Key, and Value projections are generated at the same time. Next, the Query and Key projections are reshaped to generate the channel-wise self-attention matrix and normalized using the Softmax activation function to obtain the Attention Weight. The Attention Weight is then applied to the Value projection to weaken the complex irrelevant background information and perceive the spatial position of the potential target. Finally, a depth-wise separable convolution with a kernel size of 3×3 is used to smooth the target features while balancing the model performance overhead.

[0023] like Figure 4 As shown in the figure, the hybrid feature mining module consists of two submodules: the large kernel reconstruction module and the visual state space module. The large kernel reconstruction module captures more subtle contrast fluctuations on localized pixel information, which helps the model distinguish noise from targets at a fine-grained level while strengthening the semantic features of the target; the visual state space module introduces global context modeling capabilities, which can establish associations between spatial and semantic information in a global range and improve the model's ability to detect clustered targets. In addition, the use of layer normalization and residual connections enhances the nonlinearity and stability of the model.

[0024] like Figure 5 As shown in the figure, the large kernel reconstruction module is an integrated three-branch structure. The three branches capture different scale information through convolution kernels of different sizes to obtain scale features. Different scale features are further feature-aggregated through channel splicing and 1x1 convolution to form cross-scale aggregation features. The receptive field is expanded by using dilated convolution with a convolution kernel size of 3×3, a step size of 1, and dilation rates of 2, 4, and 2 respectively. Finally, the output features are input into the final depth-wise separable convolution with a convolution kernel size of 3×3 to restore channel dependency.

[0025] like Figure 6As shown in the figure, the visual state space module is a parallel processing dual-branch structure. In the main branch, the input features first pass through a linear layer to expand the channel, and then pass through a stack of depth-wise separable convolution, SiLU activation function, selective scanning module and layer normalization operations to capture the global target features; in the second branch, a linear layer is used to expand the channel and then activated by SiLU; finally, the outputs of the two branches are element-wise multiplied and the channels are adjusted to be the same as the input channels through a linear layer.

[0026] Example 4 Infrared weak small target detection method based on hybrid perception state space model, where step 3 is specifically: using the focused feature fusion module, by focusing and calibrating the cross-layer features between the encoder and the decoder, improve the inconsistency problem between the semantics and texture details of the features, guide the model to continuously focus on the target semantic information and reconstruct the target edge structure features; then use the fully connected layer to adjust the number of channels of the feature map and output the final detection image. The decoder consists of 4 focused feature fusion modules and 1 fully connected layer. After the 4 focused feature fusion modules, decoding feature maps with feature sizes of 32x32, 64x64, 128x128 and 256x256 will be generated respectively, and then the number of feature map channels will be adjusted to 1 through the fully connected layer.

[0027] Example 5 In the infrared dim small target detection method based on the hybrid perception state space model, in step 3, the cross-layer features from the encoding stage and the decoding stage are focused and calibrated four times using the focused feature fusion module, and the final infrared dim small target detection map is output by the fully connected layer.

[0028] like Figure 7 As shown in the figure, the focus feature fusion module consists of two parallel stages: focusing and calibration. In the focusing stage, the feature map Enc from the encoder and the feature map Dec from the decoder are first spliced ​​along the channel, and then the fused information descriptor is given importance through average pooling, maximum pooling, shared perceptron and Sigmod activation function. The cross-level fusion features are further focused by element multiplication and residual connection is used to reduce information loss, which promotes the model to continuously emphasize the target's noticed area; in the calibration stage, the edge structure information is captured from the two independent feature maps of the encoder and decoder respectively through convolution with a convolution kernel size of 3×3 and a step size of 1, and then the feature calibration is performed through element-level merging combined with depthwise separable convolution to realize the model's reconstruction of the target texture details; finally, the features obtained in the focusing and calibration stages are combined, and the enhanced fusion features will improve the model's accurate detection performance for weak targets.

[0029] like Figure 8As shown in the figure, the fully connected layer processes the input features through a convolution with a convolution kernel size of 1×1 and a step size of 1, adjusts the number of channels to 1, and obtains the final infrared weak target detection map.

[0030] Example 6 Infrared dim small target detection method based on hybrid perception state space model, the specific implementation steps of step 4 are as follows: Step 4.1, determine the loss function of the target detection neural network, and the loss function adopts SoftIoULoss; In step 4.1, the loss function SoftIoULoss is: (1) In the formula, is the model for pixels The predicted probability of is the target mask at pixel The value of It is to prevent the denominator from being divided by zero and improve the stability of loss calculation.

[0031] Step 4.2: Use AdamW optimizer to optimize the loss function, with a batch size of 4, a learning rate of 0.0001, and train the network for 500 rounds. Step 4.3: Use the training set to train the network.

[0032] like Fig. 9 As shown in FIG. 1 , the visualization results of the present invention and other methods on the IRSTD-1K and DenseSIRST datasets are shown, where the rectangular boxes represent detected targets, missed targets, and misdetected targets, respectively. Fig. 9 It can be seen from the fourth row that only the present invention can maintain the existence of the target under extremely low contrast conditions while achieving accurate detection of the target edge, and different degrees of target loss also occur in other images to be detected. Therefore, the present invention significantly improves the detection performance of infrared weak targets.

[0033] The infrared dim small target detection method based on the hybrid perception state space model of the present invention utilizes the proposed cross-scale shallow embedding module, hybrid feature mining module and focused feature fusion module, fully perceives the potential target position in the high-resolution image at the network entrance, improves the ability to distinguish background clutter and noise and strengthens the semantic information of the target in the downsampling stage, and completes the focusing and calibration of cross-layer features in the upsampling stage to achieve continuous response to the captured target, which can effectively improve the detection accuracy of infrared dim small targets and reduce the false alarm rate.

Claims

1. An infrared small target detection method based on a hybrid perception state space model is characterized by: Follow the steps below to implement it: Step 1: Construct infrared small target detection dataset; Step 2: Construct an encoder, which includes a cross-scale shallow embedding module and a hybrid feature mining module; Step 3: Build a decoder, which includes a focus feature fusion module and a fully connected layer; Step 4: Use the detection network constructed by training the data set to detect infrared dim small targets.

2. The infrared dim small target detection method based on the hybrid perception state space model according to claim 1 is characterized in that: The step 2 is specifically as follows: using a cross-scale shallow embedding module to fully perceive the potential target position on the high-resolution image at the network entrance and suppress background irrelevant noise and clutter information, and then using a hybrid feature mining module to perceive the local surrounding contrast of the target and perform a comprehensive analysis of the global background environment.

3. The infrared dim small target detection method based on the hybrid perception state space model according to claim 1 is characterized in that: The encoder in step 2 is composed of 1 cross-scale shallow embedding module and 4 hybrid feature mining modules. After the cross-scale shallow embedding module, the channels of the original input 3-channel image are adjusted to increase the channel dimension to 32; then, 4 hybrid feature mining modules are used to generate encoding feature maps with feature sizes of 128x128, 64x64, 32x32 and 16x16 respectively.

4. The infrared dim small target detection method based on the hybrid perception state space model according to claim 3 is characterized in that: The cross-scale shallow embedding module first implements coarse-grained noise filtering through a convolution with a kernel size of 3×3 and a stride of 1, then concatenates the cross-scale spatial features through MaxPooling, Indetity, and AvgPooling operations, adjusts the channel dimension through convolution, aggregates spatial information, and generates Query, Key, and Value projections at the same time; next, the Query and Key projections are reshaped to generate the channel-wise self-attention matrix, and are normalized using the Softmax activation function to obtain the Attention Weight; Then, the Attention Weight is applied to the Value projection to weaken the complex irrelevant background information and perceive the spatial position of the potential target. Finally, a depth-wise separable convolution with a convolution kernel size of 3×3 is used to smooth the target features while balancing the model performance overhead.

5. The infrared dim small target detection method based on the hybrid perception state space model according to claim 3 is characterized in that: The hybrid feature mining module consists of two submodules: a large core reconstruction module and a visual state space module.

6. The infrared dim small target detection method based on the hybrid perception state space model according to claim 5 is characterized in that: The large kernel reconstruction module captures more subtle contrast fluctuations on localized pixel information, thereby helping the model to distinguish noise from targets at a fine granularity while strengthening the semantic features of the target. The visual state space module introduces global context modeling capabilities, which can establish associations between spatial and semantic information on a global scale and improve the model's ability to detect clustered targets.

7. The infrared dim small target detection method based on the hybrid perception state space model according to claim 6 is characterized in that: The large kernel reconstruction module is an integrated three-branch structure. The three branches capture different scale information through convolution kernels of different sizes to obtain scale features. Different scale features are further feature-aggregated through channel splicing and 1x1 convolution to form cross-scale aggregation features. The receptive field is expanded by using dilated convolution with a convolution kernel size of 3×3, a step size of 1, and dilation rates of 2, 4, and 2 respectively. Finally, the output features are input into the final depth-separable convolution with a convolution kernel size of 3×3 to restore channel dependency. The visual state space module is a dual-branch structure with parallel processing. In the main branch, the input features are first expanded through a linear layer, and then stacked through depthwise separable convolution, SiLU activation function, selective scanning module and layer normalization operations to capture the global target features. In the second branch, a linear layer is used to expand the channels and then activated by SiLU. Finally, the outputs of the two branches are element-wise multiplied and the channels are adjusted to be the same as the input channels through a linear layer.

8. The infrared dim small target detection method based on the hybrid perception state space model according to claim 1 is characterized in that: The step 3 specifically includes: using a focused feature fusion module to improve the inconsistency problem between semantics and texture details of features by focusing and calibrating cross-layer features between the encoder and the decoder, guiding the model to continuously focus on the target semantic information and reconstruct the target edge structure features; Then, the fully connected layer is used to adjust the number of channels of the feature map and output the final detection image; the decoder consists of 4 focused feature fusion modules and 1 fully connected layer. After passing through the 4 focused feature fusion modules, decoding feature maps with feature sizes of 32x32, 64x64, 128x128 and 256x256 are generated respectively, and then the number of feature map channels is adjusted to 1 through the fully connected layer.

9. The infrared dim small target detection method based on the hybrid perception state space model according to claim 8 is characterized in that: The focused feature fusion module consists of two parallel stages: focusing and calibration. In the focusing stage, the feature map Enc from the encoder and the feature map Dec from the decoder are first concatenated along the channel, and then the importance of the fused information descriptor is given through average pooling, maximum pooling, shared perceptron and Sigmod activation function. The cross-level fused features are further focused by element multiplication and residual connection is used to reduce information loss, so as to promote the model to continuously emphasize the target's noticed area. In the calibration stage, the edge structure information is captured from the two independent feature maps of the encoder and decoder through convolution with a kernel size of 3×3 and a step size of 1. Then, feature calibration is performed through element-level merging combined with depthwise separable convolution to enable the model to reconstruct the target texture details. Finally, the features obtained in the focusing and calibration stages are combined, and the enhanced fusion features will improve the model's accurate detection performance for weak targets.

10. The infrared dim small target detection method based on the hybrid perception state space model according to claim 8 is characterized in that: The fully connected layer convolves the input features with a convolution kernel size of 1×1 and a step size of 1, adjusts the number of channels to 1, and obtains the final infrared dim target detection map.

Images