A robust infrared small target detection method combining multi-information and noise prediction

Abstract

The application provides a robust infrared small target detection method combining multiple information and noise prediction. The detection method comprises the following steps: firstly, a gradient and context information extraction module is used to extract more target features at multiple scales, so that the model can still better maintain the deep target information; then, a noise prediction sub-network is constructed to better distinguish the infrared weak small target and image noise; further, a region positioning branch is used to reduce the false detection rate of the infrared weak small target; finally, all results are sent to a decoder, and the decoder generates a segmentation image with only the target. The detection method can effectively improve the detection accuracy of the infrared weak small target and reduce the false detection rate.

Description

TECHNICAL FIELD

[0001] The present application relates to a robust infrared small target detection method combining multi-information and noise prediction. BACKGROUND

[0002] Infrared weak small target detection technology is one of the important application technologies in the field of military target detection. The infrared radiation of the target in the infrared image has strong penetration, and the infrared radiation is not easily absorbed by the cloud layer and is not easily disturbed by the weather and environment. The infrared detection system has the advantages of strong anti-interference ability, small model, easy installation, all-weather work in harsh environment, etc. Therefore, the infrared weak small target detection technology is widely used in the field of military target detection. Such as applied to air defense early warning, missile interception, missile guidance, camouflage search, satellite remote sensing and other important tasks.

[0003] At present, the traditional method and deep learning method of infrared weak small target detection have high accuracy and low missed detection rate for most infrared image scenes. However, the infrared weak small target lacks color, contour, texture and other features, and in the case of complex background and noise interference, the missed detection rate and false detection rate of the existing infrared weak small target detection method still need to be further reduced. SUMMARY

[0004] Based on this, the present application provides a robust infrared small target detection method combining multi-information and noise prediction, which can realize accurate infrared small target detection and achieve a good balance between detection probability and false detection rate. First, a gradient and context information extraction module for extracting multi-scale features from the input infrared image is designed. This module captures multiple information to identify infrared small targets. Then, a noise prediction network is proposed to model the background noise, which helps to distinguish targets and noise. Finally, a region positioning branch is planned to provide additional location information for target detection, which can further improve the detection probability and reduce the false detection rate.

[0005] The present application provides a robust infrared small target detection method combining multi-information and noise prediction, comprising the following steps:

[0006] Step S1: inputting an infrared image;

[0007] Step S2: defining a gradient and context information extraction module;

[0008] Step S3: defining a noise prediction network, the noise prediction network is used for predicting the noise distribution probability of the input infrared image;

[0009] Step 4: constructing a region positioning branch;

[0010] Step 5: input all information into the decoder: input the feature maps in step 2, step 3 and step 4 into the decoder, and the decoder decodes all the feature maps to generate the final result;

[0011] Step 6: the decoder outputs the final segmented Mask map.

[0012] Further, the step S2 passes through a gradient and context information extraction module, the module includes two center difference convolution layers and a cross attention module, first, the feature map passes through the first center difference convolution layer; then passes through the second center difference convolution layer, which is a dilated convolution with a dilation rate of 2, which is used to increase the receptive field of the model; finally, pass through the cross attention module, which is used to extract context information; the center difference convolution is a better convolution for gradient extraction, in the convolution process, the pixels in the entire convolution region are subtracted from the center pixel value, and then the standard convolution is performed, the output y of the center difference convolution can be calculated by the following formula:

[0013]

[0014] In formula (1), p0 represents the current position on the input and output feature map, p n Enumerate the positions in R0, and w(p n ) represents the weight at p n , x(p n ) is the input value at p n , and θ represents the weight coefficient. For each pixel, the cross attention module is used to collect the context information of all pixels on the pixel cross path, and through further loop operation, each pixel can finally obtain the complete image correlation, the cross attention module can be calculated by the following formula:

[0015]

[0016] In formula (2), c u ' represents the feature vector at u in the output feature map, the symbol A i，u represents the scalar value at channel i and position u, and Ф i,u represents the vector set at channel i position and u.

[0017] Further, the noise prediction network of the step S3 first passes through the BayarConv layer to enhance the noise inconsistency between the manipulation area and the real area in the input infrared image, and the BayarConv is realized by the following formula:

[0018]

[0019] In formula (3), W k represents W kThe parameterized convolution filter with the center element (0, 0) is BayarConv, and the symbol k represents the kth layer of the convolution filter, then the output of the BayarConv layer is sent to the CovNext network to generate noise feature maps with different resolutions of 1 / 2, 1 / 4, 1 / 8, 1 / 16, which are represented as follows:

[0020] N i (I R )=CovNext i (BayarConv(I R )) (4)

[0021] Where I R represents the input infrared image, the symbol N i (I R ) is the noise feature map output by the i th module of the CowNext network, BayarConv(I R ) represents the output feature map of the BayarConv layer, and CovNext(…) represents the feature map output by the backbone network.

[0022] Further, the region positioning branch of step 4 is first constructed by receiving the minimum resolution feature map output by the backbone network, the proposed region positioning network uses a 3x3 convolution layer to generate a single-channel feature map, then generates a target position probability feature map through an S-shaped function, and finally uses a segmentation threshold to predict which receptive field of the input infrared image may include an object, and realizes the rough positioning mask of the infrared target according to the following judgment standard:

[0023]

[0024] Where P(i) represents the probability that the i th receptive field may contain a target, M(i) is the segmentation result of the i th receptive field in the position mask M, and the symbol ε represents a segmentation threshold set as ε = 0.6 in this paper.

[0025] Compared with the prior art, the advantages of the present application are as follows:

[0026] The robust infrared small target detection method combining multiple information and noise prediction of the present application first uses a gradient and context information extraction module to capture multiple different scale gradient and context information to identify infrared small targets, then uses a noise prediction network to model the background noise, which helps to distinguish targets and noise, and finally we use a region positioning branch to provide additional position information for target detection, which can further improve the detection probability and reduce the false detection rate. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 An image in an infrared weak small target data set;

[0028] Figure 2 Here is a structural diagram of the gradient and context information extraction module;

[0029] Figure 3 This is a diagram of the noise prediction network structure.

[0030] Figure 4 Branch structure diagram for regional positioning;

[0031] Figure 5 This is a structural diagram of a robust infrared small target detection method that combines multi-information and noise prediction.

[0032] Figure 6 This is a graph showing the results of the model's predictions. Specific implementation methods

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Please see Figures 1-6 This invention provides a robust infrared small target detection method that combines multi-information and noise prediction, comprising the following steps:

[0035] Step 1: Input an image from the infrared weak target dataset (e.g., Figure 1 (as shown);

[0036] Step 2: Define the gradient and context information extraction module:

[0037] This patent proposes a gradient and context information extraction module (such as...). Figure 2 As shown in the diagram, this module is designed to enable our model to better understand infrared small targets and the infrared background through gradient and contextual information. The module consists of two center-difference convolutional layers and a cross-attention module. First, the feature map passes through the first center-difference convolutional layer, then through the second center-difference convolutional layer, which is a dilated convolution with a dilation rate of 2, designed to increase the model's receptive field. Finally, it passes through the cross-attention module, designed to extract contextual information. The center-difference convolution is a better convolution for gradient extraction than the standard convolution. During convolution, the pixel value of the center pixel is subtracted from the pixel value of the entire convolution region, and then a standard convolution is performed. The output y of the center-difference convolution can be calculated using the following formula:

[0038]

[0039] In equation (1), p0 represents the current position on the input and output feature maps, pn enumerate the positions in R0, while w(p n ) denotes the weight at p n , x(p n ) is the input value at p n , and Θ denotes the weight coefficient. The cross-attention module utilizes the correlation of features on the cross path to obtain the context information, which greatly reduces the computational complexity. Specifically, for each pixel, the cross-attention module is used to collect the context information of all pixels on the cross path of the pixel. Through further loop operations, each pixel can finally obtain the complete image correlation. The cross-attention module can be calculated by the following formula:

[0040]

[0041] In formula (2), c u ' denotes the feature vector at u in the output feature map, the symbol A i，u denotes the scalar value at channel i and position u, and Ф i,u denotes the vector set at channel i and position u.

[0042] Step 3, define the noise prediction network:

[0043] In addition to the inherent limitations of infrared small targets (extremely small size and lack of structural features), background noise is also a major challenge for current infrared weak small target detection methods. Although many studies have been devoted to developing various noise suppression strategies to overcome the negative interference caused by background noise, most existing methods are designed for specific scenarios. This severely limits the performance of existing infrared weak small target detection methods in different scenarios.

[0044] We designed a noise prediction network to predict the noise distribution probability of the input infrared image (as shown in Figure 3 ). The noise prediction network first uses the BayarConv layer to enhance the noise inconsistency between the manipulation area and the real area in the input infrared image. Specifically, BayarConv is implemented by the following formula:

[0045]

[0046] In formula (3), W k denotes the parameterized convolution filter with W k (0,0) as the center element, and the symbol k denotes the kth layer of the convolution filter. Then, the output of the BayarConv layer is sent to the CovNext network to generate noise feature maps with different resolutions of 1 / 2, 1 / 4, 1 / 8, 1 / 16, which can be represented as follows:

[0047] N i(I R )=CovNext i (BayarConv(I R )) (4)

[0048] where I R represents the input infrared image, the symbol N i (I R ) is the noise feature map output by the i-th module of the CovNext network, BayarConv(I R ) represents the output feature map of the BayarConv layer, and CovNext(…) represents the feature map output by the backbone network. Since low-resolution feature maps contain more semantic information than high-resolution feature maps, we only send noise feature maps with resolutions of 1 / 8 and 1 / 16 to help the decoder better distinguish targets and noise.

[0049] Step 4, build the region localization branch:

[0050] In general, richer feature information is beneficial to improve the detection probability of infrared targets. However, since infrared small targets usually only occupy a very small area of the input image, redundant information may increase the false detection rate. In order to suppress background interference and achieve a good balance between detection probability and false detection rate, we designed a region localization branch to predict the location of the infrared target Mask (as shown in Figure 4 By receiving the minimum resolution feature map output by the backbone network, the proposed region localization network uses a 3x3 convolution layer to generate a single-channel feature map. Then, a target location probability feature map is generated through the S-shaped function. Finally, we use a segmentation threshold to predict which receptive field of the input infrared image may include the object, and realize the rough localization mask of the infrared target according to the following judgment criteria:

[0051]

[0052] where P(i) represents the probability that the i-th receptive field may contain the target, and M(i) is the segmentation result of the i-th receptive field in the location mask M. The symbol ε represents the segmentation threshold set to ε=0.6 in this paper. We directly use the output feature map of the backbone network to predict the probability feature map of the target, and then use the segmentation threshold to obtain the final rough location mask. The proposed region localization branch can better overcome background interference and achieve a good balance between detection probability and false detection rate.

[0053] Step 5, input all information into the decoder:

[0054] The feature maps output in steps 2, 3, and 4 are input into the decoder, which decodes all the feature maps to generate the final result (as shown in Figure 5 ).

[0055] Step 6, the decoder outputs the final segmented Mask map (as shown in Figure 6 The robust infrared small target detection method of the present application combined with multi-information and noise prediction first uses a gradient and context information extraction module to capture gradient and context information of various scales to identify infrared small targets, then uses a noise prediction network to model background noise, which helps to distinguish targets and noise, and finally, we use a region positioning branch to provide additional location information for target detection, which can further improve the detection probability and reduce the false detection rate.

[0056] Finally, it should be noted that: the above only for the preferred embodiments of the present application, not limited to the present application, although the foregoing detailed description of the present application is made with reference to the foregoing examples, for those skilled in the art, it still can be modified, or part of the technical features of the equivalent replacement, within the spirit and principles of the present application, any modification, equivalent replacement, improvement, etc., should be included within the scope of the present application.

Claims

1. A robust infrared small target detection method combining multi-information and noise prediction, comprising the following steps: Step S1: inputting an infrared image; the infrared image is input to an encoder; Step S2: defining a gradient and context information extraction module; the feature map output by step S1 is input to the gradient and context information extraction module; Step S3: defining a noise prediction network, and inputting the infrared image to the noise prediction network; The noise prediction network is used to predict the noise distribution probability of the input infrared image; Step S4: constructing a region positioning branch; the feature map output by step S1 is input to the region positioning branch; Step S5: inputting all information to a decoder; the feature maps output in steps S2, S3 and S4 are input to the decoder, and the decoder decodes all the feature maps to generate a final result; Step S6: the decoder outputs a final segmented Mask map; The step S2 passes through the gradient and context information extraction module, which includes two center difference convolution layers and a cross-attention module. First, the feature map passes through the first center difference convolution layer; then passes through the second center difference convolution layer, which is a dilated convolution with a dilation rate of 2, which is used to increase the receptive field of the model; finally, it passes through the cross-attention module, which is used to extract context information; the center difference convolution is a better convolution for gradient extraction. In the convolution process, the pixels in the entire convolution region are subtracted from the center pixel value, and then the standard convolution is performed. The output y of the center difference convolution can be calculated by the following formula: wherein, p 0 denotes the current position on the input and output feature map, p n enumerate R 0 the position in w ( p n ) denotes p n the weight at x ( p n ) is p n the input value at denotes the weight coefficient; for each pixel, the context information of all pixels on the pixel cross path is collected using the cross attention module, and through further loop operation, each pixel can finally obtain complete image correlation, and the cross attention module can be calculated by the following formula: wherein, c u ’ denotes a feature vector at position u in the output feature map, the symbol A i，u denotes a scalar value at position i and position u , while i,u denotes a set of vectors at position i position and u .

2. The robust infrared dim target detection method combining multiple information and noise prediction according to claim 1, characterized in that: The noise prediction network of step S3 first enhances the noise inconsistency between the manipulation region and the real region in the input infrared image through a BayarConv layer, which is implemented by the following formula: wherein, W k denotes a parametric convolution filter centered at W k a parametric convolution filter centered at (0, 0), the notation k denotes the k-th layer of the convolution filter, then the output of the BayarConv layer is sent to the CovNext network for generating noise feature maps with different resolutions of 1 / 2, 1 / 4, 1 / 8, 1 / 16, which are denoted as follows: where I R represents the input infrared image, and the symbol N i represents the number of channels of the input infrared image. R is the noise feature map output by the i-th module of the CowNext network, BayarConv(I R represents the output feature map of the BayarConv layer, and CovNext(…) represents the feature map output by the backbone network.

3. The robust infrared dim target detection method combining multiple information and noise prediction according to claim 1, characterized in that: The region positioning branch of step S4 first receives the minimum resolution feature map output by the backbone network. The proposed region positioning network uses a 3x3 convolution layer to generate a single-channel feature map, then generates a target position probability feature map through an S-shaped function, and finally uses a segmentation threshold to predict which receptive field of the input infrared image may include an object, and realizes the rough positioning mask of the infrared target according to the following judgment standard: Where P(i) represents the probability that the ith receptive field may contain a target, M(i) is the segmentation result of the ith receptive field in the position mask M, and the symbol ε represents a segmentation threshold set to ε=0.6 in this paper.

Images