Infrared weak and small target detection method based on wavelet guide state space model

Abstract

The invention discloses an infrared weak and small target detection method based on a wavelet guide state space model. The infrared weak and small target detection method is specifically implemented according to the following steps: step 1, constructing an infrared weak and small target detection data set; step 2, constructing an encoder, wherein the encoder comprises a convolutional layer and a dynamic frequency adaptive module; step 3, constructing a decoder, wherein the decoder comprises a mixed feature fusion module and a full connection layer; and step 4, training the constructed detection network by using the data set, and realizing infrared weak and small target detection. The method can effectively enhance the capability of capturing infrared weak and small target features under different scales, improves the perception of the target position under the condition of low signal-to-noise ratio, reduces the false alarm rate of target detection, and improves the detection rate.

Description

Technical Field

[0001] The invention belongs to the technical field of image target detection methods, and in particular relates to an infrared dim small target detection method based on a wavelet guided state space model. Background Art

[0002] In applications such as military surveillance, search and rescue, and remote sensing, accurate detection of small targets in infrared images is crucial. However, traditional visual detection methods rely on artificially designed features and prior knowledge, which are difficult to achieve ideal results in complex and changing real-world scenarios and face significant performance bottlenecks. In contrast, deep learning technology, with its powerful automatic feature extraction and representation capabilities, has significantly improved the performance of infrared small target detection, overcomes the limitations of traditional methods, and provides a more robust and adaptable solution for this field.

[0003] Deep learning models, especially convolutional neural networks, have achieved remarkable results in image recognition and feature extraction. However, existing methods still cannot effectively use frequency domain information to enhance target perception. At the same time, the use of interpolation upsampling in the feature fusion process will introduce interpolation artifacts, causing the weak features of the target to be submerged, resulting in low detection accuracy and high false alarm rate in existing methods. Summary of the invention

[0004] The purpose of the present invention is to provide an infrared dim small target detection method based on a wavelet guided state space model, which can improve the detection accuracy of infrared dim small targets.

[0005] The technical solution adopted by the present invention is an infrared dim small target detection method based on a wavelet guided state space model, which is specifically implemented according to the following steps:

[0006] Step 1: Construct infrared small target detection dataset;

[0007] Step 2: construct an encoder, which includes a convolutional layer and a dynamic frequency adaptation module;

[0008] Step 3: Construct a decoder, which includes a hybrid feature fusion module and a fully connected layer;

[0009] Step 4: Use the detection network constructed by training the data set to detect infrared dim small targets.

[0010] The technical solution of the present invention is also characterized in that:

[0011] Step 2 is as follows: Use the convolutional layer to obtain the shallow feature embedding of the target and perform preliminary noise suppression, and then use the dynamic frequency adaptation module to efficiently model the global and local information of the target in both the spatial domain and the frequency domain.

[0012] The dynamic frequency adaptation module in step 2 realizes the cross-domain extraction of global features through a set of parallel branch combinations: one set of branches first undergoes layer normalization to make the input distribution of the network more stable, and then uses the visual state space module to capture the global long-distance dependency interaction relationship in the spatial domain; the other set of branches uses the wavelet guidance module to introduce frequency domain cues to enhance the visual state space module's perception of global information; after the target global features are extracted by parallel branches, layer normalization will be further used in combination with the hybrid restoration gated network module for local information modeling.

[0013] In the visual state space module, the input features are processed by two parallel branches: in the main branch, the input features are first expanded by a linear layer, and then stacked by depthwise separable convolution, SiLU activation function, selective scanning module and layer normalization. In the second branch, the linear layer is used to implement channel expansion and then activated by SiLU. Finally, the inputs of the two branches are combined by element-wise multiplication, and the channels are adjusted to be the same as the input channels by a linear layer.

[0014] In the wavelet-guided module, the input features are first transformed by discrete wavelets to generate an approximate component LL and detail components H composed of horizontal HL, vertical LH and diagonal HH directions. After that, the L component is refined by deep separable convolution to obtain the low-frequency component L'. The H component is first subjected to 1x1 convolution to increase the dimension and deep separable convolution to capture more feature patterns in high dimensions and reduce the influence of noise. Then, the channel dimension is reduced by 1x1 convolution to filter out redundant information and obtain the high-frequency component H'. Finally, the L' component and the H' component are subjected to inverse discrete wavelet transform to restore the feature size, and the edge effect caused by the inverse discrete wavelet transform is filtered out in combination with deep separable convolution to smooth the frequency domain features.

[0015] Hybrid restoration gated network module, which uses two sets of information flows to work together. The first set of information flows uses dilated convolutions with a convolution kernel size of 3×3, a step size of 1, and dilation rates of 2, 4, and 2, respectively, and combines feature addition to expand the receptive field and enhance the perception of spatially neighboring pixels in local features. The second set of information flows uses depthwise separable convolution and GELU activation function to generate pixel-level gated masks. Finally, the two sets of information flows are element-wise multiplied to interact and highlight the features of faint targets. At the same time, the output features are refined through a convolution with a convolution kernel size of 3×3 and a step size of 1.

[0016] Step 3 is as follows: Use the hybrid feature fusion module to fuse and enhance the low-level and high-level features from the codec to promote feature sharing at different levels, and then use the fully connected layer to adjust the number of channels of the feature map and output the final detection image.

[0017] The hybrid feature fusion module in step 3 merges the two feature maps from the encoder and decoder that are upsampled by bilinear interpolation with a scale factor of 2; then, the feature relationship between channels is enhanced through Channel Mixing. First, average pooling and maximum pooling operations are used to generate two independent feature descriptors that aggregate channel information; next, the two descriptors are input into a shared multi-layer perceptron, and the outputs of the multi-layer perceptron are summed; finally, the features after the codec merge feature is summed with the multi-layer perceptron and activated with Sigmoid are element-wise multiplied.

[0018] The features enhanced by Channel Mixing will be corrected by Spatial Mixing for false alarms caused by spatial position distortion. First, the features obtained by Channel Mixing are refined by depth-wise convolution. Then, dilated convolution with a kernel size of 3×3, a step size of 1, and dilation rates of 2, 4, and 2 are used to capture spatial relationships under different receptive fields. Next, the features extracted by depth-wise separable convolution and different dilated convolution are summed and target information is shared through 1x1 convolution to promote selective aggregation of targets at each spatial position. Finally, the aggregated features are activated by Sigmoid and element-wise multiplied with the output features of Channel Mixing while combining residual connections.

[0019] The beneficial effects of the present invention are:

[0020] The infrared small target detection method based on the wavelet guided state space model of the present invention can effectively extract target features in the frequency domain, space domain and dimension domain through a dynamic frequency adaptive module and a hybrid feature fusion module, improve the phenomenon of insufficient extraction of details and texture details under the existing technical conditions, and suppress the interpolation artifact problem introduced by the use of interpolation upsampling in the feature fusion process. At the same time, when constructing a neural network using this method, by adjusting the training parameters and loss function to optimize the network, a better infrared small target detection effect can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of the present invention;

[0022] Figure 2 It is the overall network structure diagram of the present invention;

[0023] Figure 3 It is a convolutional layer network structure diagram of the present invention;

[0024] Figure 4 It is a network structure diagram of the dynamic frequency adaptation module in the present invention;

[0025] Figure 5 is a network structure diagram of the wavelet guided module in the present invention;

[0026] Figure 6 is a network structure diagram of the visual state space module in the present invention;

[0027] Figure 7 is a network structure diagram of the hybrid repair gating network module in the present invention;

[0028] Figure 8 It is a network structure diagram of the hybrid feature fusion module in the present invention;

[0029] Fig. 9 is a network structure diagram of the fully connected layer of the present invention;

[0030] Fig.10 It is a rendering of the present invention compared with other methods. DETAILED DESCRIPTION

[0031] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] The present invention is based on the infrared weak small target detection method of the wavelet guided state space model, such as Figure 1-2 As shown, the specific implementation steps are as follows:

[0033] Step 1: Construct infrared small target detection dataset;

[0034] Step 2: construct an encoder, which includes a convolutional layer and a dynamic frequency adaptation module;

[0035] Step 3: Construct a decoder, which includes a hybrid feature fusion module and a fully connected layer;

[0036] Step 4: Use the detection network constructed by training the data set to detect infrared dim small targets.

[0037] Example 1

[0038] Infrared dim small target detection method based on wavelet guided state space model, wherein the data set in step 1 uses public data sets such as IRSTD-1K, NUAA-SIRST, NUDT-SIRST or DenseSIRST, and randomly divides the data set into training set and test set in a ratio of 8:2. Random horizontal and vertical flipping and rotation operations are used for data enhancement and the image size is cropped to 256×256.

[0039] Example 2

[0040] Infrared weak small target detection method based on wavelet guided state space model, in which step 2 uses convolution layer to obtain shallow feature embedding of the target and preliminarily suppresses noise, and then uses dynamic frequency adaptive module to efficiently model the global and local information of the target in both spatial and frequency domains. The encoder consists of 1 convolution layer and 4 dynamic frequency adaptive modules. After the convolution layer, the channels of the original input 3-channel image will be adjusted to increase the channel dimension to 32. Then, 4 dynamic frequency adaptive modules will be used to generate encoding feature maps with feature sizes of 128x128, 64x64, 32x32 and 16x16 respectively.

[0041] The specific steps of building the encoder are as follows: the input image is fed into the first convolutional layer to initially suppress the background noise and clutter information and extract the shallow feature embedding information of the target. Then, by using four dynamic frequency adaptation modules, a coding feature map with obvious semantic features of the target is obtained;

[0042] like Figure 3 As shown in the figure, it is a network structure diagram of the convolution layer in this embodiment. This layer uses a convolution group consisting of a convolution with a convolution kernel size of 3×3 and a step size of 1, batch normalization, and ReLU activation function to adjust the number of channels to 64, so as to achieve preliminary extraction of shallow features of the target. Then, a convolution group consisting of a convolution with a convolution kernel size of 3×3 and a step size of 1, batch normalization, and ReLU activation function is used again to refine the features obtained by the previous convolution group and adjust the number of channels to 32 required by the dynamic frequency adaptation module.

[0043] like Figure 4 As shown, it is the network structure diagram of the dynamic frequency adaptation module in this embodiment. The module first realizes the cross-domain extraction of global features through a set of parallel branch combinations: one set of branches first undergoes layer normalization to make the input distribution of the network more stable, and then uses the visual state space module (VSSM) to capture the global long-distance dependent interaction relationship in the spatial domain; another set of branches uses the wavelet guided module (WSM) to introduce frequency domain clues to enhance the visual state space module's perception of global information. After the parallel branches extract the global features of the target, they will further use layer normalization and combine the hybrid repair gating network (HRGN) to perform local information modeling to improve the target loss that may be caused by the fixed receptive field in the global information flow and reduce the missed detection rate.

[0044] like Figure 5As shown, it is a network structure diagram of the wavelet guided module in this embodiment. The input features of this module are first transformed by discrete wavelets to generate an approximate (low-frequency) component LL(L) and detail (high-frequency) components H composed of horizontal HL, vertical LH and diagonal HH directions. After that, the L component is refined by deep separable convolution to obtain the low-frequency component L'. The H component is first subjected to 1x1 convolution dimension increase and deep separable convolution to capture more feature patterns in high dimensions and reduce the influence of noise. Then, the channel dimension is reduced by 1x1 convolution to filter redundant information and obtain the high-frequency component H'. Finally, the L' component and the H' component are subjected to inverse discrete wavelet transform to restore the feature size, and the edge effect caused by the inverse discrete wavelet change is filtered out in combination with deep separable convolution to smooth the frequency domain features.

[0045] like Figure 6 As shown in the figure, it is a network structure diagram of the visual state space module in this embodiment, which is processed by two parallel branches. In the main branch, the input features are first expanded by a linear layer, and then stacked by operations such as depth-separable convolution, SiLU activation function, selection scanning module and layer normalization, which effectively integrates the target features from the global. In the second branch, the linear layer is used to achieve channel expansion and then activated by SiLU. Finally, the inputs of the two branches are combined by element-wise multiplication, and the channels are adjusted to be the same as the input channels through a linear layer.

[0046] like Figure 7 As shown in the figure, it is the network structure diagram of the hybrid restoration gated network module in this embodiment. The module uses two sets of information flows to work together. The first set of information flows uses a dilated convolution with a convolution kernel size of 3×3, a step size of 1, and dilation rates of 2, 4, and 2, respectively, and combines feature addition to expand the receptive field to enhance the perception of spatially adjacent pixels in local features, while the second set of information flows uses a depth-separable convolution and a GELU activation function to generate a pixel-level gated mask. Finally, the two sets of information flows are interactively multiplied by elements to highlight the features of weak targets, and the output features are refined by a convolution with a convolution kernel size of 3×3 and a step size of 1.

[0047] Example 3

[0048] Infrared weak small target detection method based on wavelet guided state space model. In step 3, a hybrid feature fusion module is used to fuse and enhance low-level and high-level features from the codec, promote feature sharing at different levels, and reduce the loss of target information. Then the fully connected layer is used to adjust the number of channels of the feature map and output the final detection image. The decoder consists of 4 hybrid feature fusion modules and 1 fully connected layer. After passing through the 4 hybrid feature fusion modules, decoding feature maps with feature sizes of 32x32, 64x64, 128x128 and 256x256 are generated respectively, and then the number of feature map channels is adjusted to 1 through the fully connected layer.

[0049] The low-dimensional features from the encoding stage and the high- and low-frequency information generated by wavelet transformation are aggregated using a hybrid feature fusion module. After four recursive aggregations, the final infrared dim target detection map is output by the fully connected layer.

[0050] like Figure 7 As shown, it is a network structure diagram of the hybrid feature fusion module in this embodiment. The module first merges the two feature maps from the encoder and decoder that are upsampled by bilinear interpolation with a scale factor of 2. After that, the feature relationship between channels is enhanced by Channel Mixing. First, average pooling and maximum pooling operations are used to generate feature descriptors of two independent aggregated channel information. Next, the two descriptors are input into a shared multi-layer perceptron, and the outputs of the multi-layer perceptron are summed. Finally, by element-wise multiplying the features after the codec merged features are summed with the multi-layer perceptron and activated by Sigmoid, channel-by-channel target information enhancement is achieved while improving the problem of feature loss through residual connections. Further, the features enhanced by Channel Mixing will be corrected by SpatialMixing for false alarms caused by spatial position distortion. First, the features obtained by Channel Mixing are refined using depth-wise convolution. Then, dilated convolutions with a kernel size of 3×3, a step size of 1, and dilation rates of 2, 4, and 2 are used to capture spatial relationships under different receptive fields. Next, the features extracted by depth-wise separable convolutions and different dilated convolutions are summed and target information is shared through 1x1 convolutions to promote selective aggregation of targets at various spatial positions. Finally, the aggregated features are activated by Sigmoid and element-wise multiplied with the output features of Channel Mixing, and residual connections are combined to enhance spatial target information.

[0051] Example 4

[0052] like Fig. 9As shown, this is the network structure diagram of the fully connected layer in this embodiment. This layer convolves the input features with a convolution kernel size of 1×1 and a step size of 1, adjusts the number of channels to 1, and obtains the final infrared weak target detection map.

[0053] Infrared weak small target detection method based on wavelet guided state space model, the specific implementation steps of step 4 are as follows:

[0054] Step 4.1, determine the loss function of the target detection neural network, and the loss function adopts SoftIoULoss;

[0055] In step 4.1, the loss function SoftIoULoss is:

[0056]

[0057] Where P i is the model’s predicted probability for pixel i, T i is the value of the target mask at pixel i. Smooth prevents the denominator from being divided by zero and improves the stability of the calculated loss.

[0058] Step 4.2: Use AdamW optimizer to optimize the loss function, with a batch size of 4, a learning rate of 0.0001, and train the network for 500 rounds.

[0059] Step 4.3: Use the training set to train the network.

[0060] like Fig.10 The figure shows the effect of the present invention compared with other methods. Fig.10 The visualization results of the method of the present invention and other methods on the IRSTD-1K and DenseSIRST datasets are shown, where the red rectangles represent detected targets, missed targets, and falsely detected targets. Fig.10 It can be seen from the first row that both DNANet and AGPCNet missed the target consisting of only one pixel in the upper left corner, and different degrees of false detection and missed detection also occurred in other images to be detected. At the same time, it can be seen from other images to be detected that this method can effectively improve the missed detection and false detection problems by extracting information from target features in three domains and suppressing the interpolation artifact problem in upsampling. Therefore, the present invention significantly improves the detection performance of infrared weak small targets.

[0061] The infrared small target detection method based on the wavelet guided state space model of the present invention can effectively extract target features in the frequency domain, space domain and dimension domain through a dynamic frequency adaptive module and a hybrid feature fusion module, improve the phenomenon of insufficient extraction of details and texture details under the existing technical conditions, and suppress the interpolation artifact problem introduced by the use of interpolation upsampling in the feature fusion process. At the same time, when constructing a neural network using this method, by adjusting the training parameters and loss function to optimize the network, a better infrared small target detection effect can be achieved.

Claims

1. Infrared small target detection method based on wavelet guided state space model, characterized in that: Follow the steps below to implement it: Step 1: Construct infrared small target detection dataset; Step 2: construct an encoder, which includes a convolutional layer and a dynamic frequency adaptation module; Step 3: Construct a decoder, which includes a hybrid feature fusion module and a fully connected layer; Step 4: Use the detection network constructed by training the data set to detect infrared dim small targets.

2. The infrared small target detection method based on wavelet guided state space model according to claim 1 is characterized in that: The step 2 is specifically as follows: using a convolutional layer to obtain shallow feature embedding of the target and preliminarily suppress noise, and then using a dynamic frequency adaptation module to efficiently model the global and local information of the target in both the spatial domain and the frequency domain.

3. The infrared small target detection method based on wavelet guided state space model according to claim 2 is characterized in that: The dynamic frequency adaptation module in step 2 realizes the cross-domain extraction of global features through a group of parallel branch combinations: one group of branches first undergoes layer normalization to make the input distribution of the network more stable, and then uses the visual state space module to capture the global long-distance dependency interaction relationship in the spatial domain; the other group of branches uses the wavelet guidance module to introduce frequency domain clues to enhance the visual state space module's perception of global information; after the target global features are extracted by the parallel branches, layer normalization will be further used in combination with the hybrid restoration gated network module to perform local information modeling.

4. The infrared dim small target detection method based on wavelet guided state space model according to claim 3 is characterized in that: In the visual state space module, the input features are processed by two parallel branches: in the main branch, the input features are first processed by a linear layer to expand the channels, and then processed by a stack of depthwise separable convolution, SiLU activation function, selective scanning module and layer normalization operation; in the second branch, the linear layer is used to implement channel expansion and then activated by SiLU, and finally, the inputs of the two branches are combined by element-wise multiplication, and the channels are adjusted to be the same as the input channels by a linear layer; In the wavelet-guided module, the input features are first transformed by discrete wavelets to generate an approximate component LL and detail components H composed of horizontal HL, vertical LH and diagonal HH directions. After that, the L component is refined by deep separable convolution to obtain the low-frequency component L'. The H component is first subjected to 1x1 convolution to increase the dimension and deep separable convolution to capture more feature patterns in high dimensions and reduce the influence of noise. Then, the channel dimension is reduced by 1x1 convolution to filter out redundant information and obtain the high-frequency component H'. Finally, the L' component and the H' component are subjected to inverse discrete wavelet transform to restore the feature size, and the edge effect caused by the inverse discrete wavelet transform is filtered out in combination with deep separable convolution to smooth the frequency domain features. Hybrid restoration gated network module, which uses two sets of information flows to work together. The first set of information flows uses dilated convolutions with a convolution kernel size of 3×3, a step size of 1, and dilation rates of 2, 4, and 2, respectively, and combines feature addition to expand the receptive field and enhance the perception of spatially neighboring pixels in local features. The second set of information flows uses depthwise separable convolution and GELU activation function to generate pixel-level gated masks. Finally, the two sets of information flows are element-wise multiplied to interact and highlight the features of faint targets. At the same time, the output features are refined through a convolution with a convolution kernel size of 3×3 and a step size of 1.

5. The infrared small target detection method based on wavelet guided state space model according to claim 1 is characterized in that: The step 3 is specifically as follows: using a hybrid feature fusion module to fuse and enhance low-level and high-level features from the codec, promoting feature sharing at different levels, and then using a fully connected layer to adjust the number of channels of the feature map and output the final detection image.

6. The infrared dim small target detection method based on wavelet guided state space model according to claim 5 is characterized in that: The hybrid feature fusion module in step 3 merges the two feature maps from the encoder and decoder that are upsampled by bilinear interpolation with a scale factor of 2; then, the feature relationship between channels is enhanced through Channel Mixing, and first, average pooling and maximum pooling operations are used to generate two feature descriptors that independently aggregate channel information; next, the two descriptors are input into a shared multi-layer perceptron, and the outputs of the multi-layer perceptron are summed; finally, the features after the codec merge feature is summed with the multi-layer perceptron and activated by Sigmoid are element-wise multiplied.

7. The infrared small target detection method based on wavelet guided state space model according to claim 6 is characterized in that: The features enhanced by Channel Mixing will be corrected by Spatial Mixing for false alarms caused by spatial position distortion. First, the features obtained by Channel Mixing are refined by depth-wise convolution. Then, dilated convolution with a kernel size of 3×3, a step size of 1, and dilation rates of 2, 4, and 2 are used to capture spatial relationships under different receptive fields. Next, the features extracted by depth-wise separable convolution and different dilated convolution are summed and target information is shared through 1x1 convolution to promote selective aggregation of targets at each spatial position. Finally, the aggregated features are activated by Sigmoid and element-wise multiplied with the output features of Channel Mixing while combining residual connections.

Images