An infrared small target detection method based on learnable guided filtering

Abstract

The present application relates to a kind of infrared small target detection method based on learnable guided filtering, belong to computer vision and image processing field.The method is first preprocessed to infrared image, after image size is unified, input model.Then using central difference convolution module (Central Difference Convolution, CDC) to image feature is encoded, and shallow feature is obtained.Using fast fourier convolution module (Fast Fourier Convolution, FFC) in frequency domain on feature is handled to obtain global deep feature.Then using the designed learnable guided filtering module (Learnable Guided Filtering, LGF) to up-sampling feature and jump connection feature is reinforced after edge, and then up-sampling operation is carried out.Last using the convolution of last layer, small target is extracted.The present application makes full use of the edge distribution characteristics of small target and the edge-preserving characteristics of guided filtering, realizes the edge-enhanced multiscale target detection, increases the details and edge information of target, and improves the detection performance of infrared small target.

Description

Technical Field

[0001] This invention relates to an infrared small target detection method based on learnable guided filtering, belonging to the fields of computer vision and image processing. This method effectively overcomes the common defects in infrared images, such as small pixel counts, insufficient texture information, and blurred edges. Based on the excellent edge-preserving properties of guided filtering, a learnable guided filtering module is constructed by using a box-means filter and guided filter parameter calculation module defined by a series of convolutional layers in a neural network, and by processing features differently. By widely applying this module in the model, the edge features of candidate targets and the connectivity of target regions are greatly enhanced, making the targets more prominent and clearer in the image. It also exhibits stronger robustness against multi-scale targets and clutter of different shapes. Background Technology

[0002] Infrared small target detection is an important research topic in the field of computer vision, widely used in military reconnaissance, security monitoring, environmental monitoring, and other fields. However, due to the long imaging distance, weak target density, and lack of details such as texture, structure, and edges, small targets often exhibit weak infrared radiation signals, making them susceptible to interference from noise and complex backgrounds. Therefore, infrared small target detection remains a challenging task. By analyzing the characteristics of small targets in infrared images, significant differences in grayscale features exist between the target and background regions. These differences are mainly reflected in brightness intensity or grayscale distribution. Typically, the target region exhibits higher reflectivity or emission characteristics than its surroundings, thus appearing as a bright area in the infrared image, while the background appears as a relatively dark area. This reflectivity or emission characteristic leads to a significant grayscale gradient between the target region and the background, resulting in a noticeable change in brightness. Therefore, traditional methods typically utilize this difference as the basis for target detection and recognition, distinguishing the target from the background by analyzing the grayscale distribution in the image. Mathematical modeling is used to leverage differences in target distribution characteristics, and filters are used to separate targets from the background. A saliency measurement method is designed to segment small objects. Targets are identified using the contrast between the target and its surroundings. Patch similarity is used to suppress false alarms and acquire targets. However, these methods are sensitive to hyperparameter settings, lack generalization, and perform poorly in complex real-world scenarios.

[0003] With deep learning becoming mainstream in many computer vision tasks, many researchers have chosen to use deep neural networks for infrared target detection and have made significant progress. Methods employ classic encoder-decoder structures, embedding fully connected layers to adjust the weights of the target and background and enhance their differences; adversarial learning schemes are used to learn features that reduce false positives and false negatives; and modules are proposed to fuse low-level and high-level features. However, these methods suffer from significant biases in representing shape and edge information, failing to preserve fine details such as target shape and edges effectively. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned shortcomings of the prior art and provide an infrared small target detection method based on learnable guided filtering. To this end, this invention adopts the following technical solution.

[0005] The infrared small target detection method includes the following steps:

[0006] Step 1: Input the preprocessed data into the model, use the central difference convolution module to extract shallow features, and use the fast Fourier convolution module to extract deep features;

[0007] Step 2: Design a learnable guided filter module, use this module to process shallow features and deep features respectively, fuse the processed output features to obtain edge-enhanced fused features.

[0008] 1. In step 1, shallow and deep features are extracted using the central difference convolution module and the fast Fourier convolution module, respectively. The specific steps are as follows:

[0009] 1) Preprocess the data by adjusting the infrared image size to a uniform size before inputting it into the model. Use ordinary convolution and central difference convolution in the central difference convolution module to process the features and obtain different outputs. The process is as follows:

[0010]

[0011] Where y1 and y2 are the output features of ordinary convolution and central difference convolution, respectively, and Conv(·), Cdc(·), and x represent ordinary convolution operation, central difference convolution operation, and input feature, respectively.

[0012] 2) Perform convolution and fusion operations on the two output features y1 and y2 to obtain the output of the central difference convolution module. The process is as follows:

[0013]

[0014] Where L and This represents the operation of fusing and adding the output and features of the central difference convolutional module. L is used as the input feature of the next central difference convolutional module, and multiple central difference convolutional modules are used to obtain the shallow output feature L of the last layer. f ;

[0015] 3) Input this feature into the Fast Fourier Transform (FFT) module. After processing the input feature using local and global branches respectively, the two parts are fused and added together to obtain the deep feature output by the FFT module. The process is as follows:

[0016]

[0017] Where F, h(·), and l(·) represent the output, global branch operation, and local branch operation of the Fast Fourier Transform (FFT) module, respectively. F serves as the input to the next FFT module. By using multiple FFT modules, the output feature of the last module can be obtained as F. f .

[0018] 2. In step 2, a learnable guided filter module is designed to obtain the fusion features for edge enhancement. The specific steps are as follows:

[0019] 1) The correlation coefficient of the filter is calculated based on the principle of traditional guided filtering, and its expression is as follows:

[0020]

[0021] Where a and b represent the filter correlation coefficients, and I, p, ... Let represent the guide image, input image, mean of the guide image, and mean of the input image, respectively; let cov(I, p) represent the covariance of I and p; let var(I) represent the variance of I; and let ε represent the L2 norm regularization coefficient.

[0022] 2) Based on the above coefficients, the input image, and the guide image, the filtered output can be obtained, and its expression is as follows:

[0023] q=a·I+b (5)

[0024] Where q represents the output of the input image after filtering;

[0025] 3) Based on the operations in steps 1) and 2), the skip connection feature L f Filtering is performed, where both the input and guide graphs use L. f The filtered result can be obtained, and its expression is as follows:

[0026]

[0027] Where q1 represents the filtered output of the skip connection feature;

[0028] 4) Based on the operations in steps 1) and 2), the upsampled feature F f Filtering is performed, where the input image is F. f Guided diagram using L f The filtered result can be obtained, and its expression is as follows:

[0029]

[0030] Where q2 represents the filtered output of the upsampled feature;

[0031] 5) By fusing the filtered outputs of the obtained skip connection features and the filtered outputs of the upsampled features, the output of the learnable guided filter module can be obtained, and its expression is as follows:

[0032]

[0033] in Represents the concatenation operation, ReLU represents the activation operation, BN represents the batch normalization operation, and M represents the output feature of the first learnable guided filter module. M is used as the upsampled feature of subsequent learnable guided filter modules. By using multiple learnable guided filter modules, the output feature M of the last module can be obtained. f .

[0034] The beneficial effects of this invention are:

[0035] 1. Edge Enhancement Detection: This invention is based on a learnable guided filter model, central difference convolution, and fast Fourier convolution, which fully ensures the integrity and segmentation accuracy of targets of different sizes, and can achieve multi-scale edge enhancement segmentation without manually setting parameters.

[0036] 2. Excellent detection performance: This invention not only ensures the integrity of candidate targets but also exhibits stronger robustness to images under different noise environments. Furthermore, it enhances edge information for targets of different scales during segmentation, significantly improving the detection performance of small targets. Attached Figure Description

[0037] Figure 1 : Flowchart of the infrared small target image detection method based on learnable guided filtering of the present invention.

[0038] Figure 2 Flowchart of an infrared small target image detection network based on learnable guided filtering.

[0039] Figure 3 : Structure diagram of the encoder section CDC module and FFC module in the detection network.

[0040] Figure 4 : Schematic diagram of the learnable guided filter (LGF) module in the decoder part of the detection network. Detailed Implementation

[0041] The process of this invention is as follows Figure 1As shown, this method first preprocesses the infrared image data, unifying the image size before inputting it into the network model. Then, it encodes the image features using Central Difference Convolution (CDC) and a CDC module to obtain shallow downsampled features that better preserve shape and edge information. Next, it uses a Fast Fourier Convolution (FFC) module to transform the features from the spatial domain to the frequency domain, then processes the features in the frequency domain and uses an inverse transform to transform them back to the spatial domain, thereby obtaining high-level semantic features of the global receptive field. Then, it uses the proposed Learnable Guided Filtering (LGF) module to enhance the edge features by applying different processing strategies to the upsampled features and skip connection features. The enhanced edge features are then fused and upsampled again. Finally, a 1x1 convolutional operation is used in the last layer to obtain a feature map with one channel, extracting small targets. The specific implementation process of the technical solution of this invention is described below with reference to the accompanying drawings.

[0042] 1. Data preprocessing and extraction of shallow and deep features.

[0043] The infrared images were standardized to 512×512 pixels, and then the preprocessed images were input into the network model. The specific operation process for obtaining shallow and deep features is as follows: Figure 2 As shown in the left and lower sections, the specific steps are as follows:

[0044] 1) Use two differential center convolution operations to obtain feature representations with contrast information; the differential convolution operation process is shown in formula (1), and we can obtain the following result after two center difference convolutions:

[0045]

[0046] In the above formula, L0 represents Figure 2 The output of CDC_1 is the result after two central difference convolutions. C(x) represents the combined operation of central difference and ordinary convolution, as defined in formula (1).

[0047] 2) Four central difference convolutional modules are used to obtain shallow features of different resolutions that can better preserve shape edge information, and these features are then skipped and connected to the decoder. After obtaining the above results, L0 is used as input to obtain shallow features using the central difference convolutional modules. The specific operation process is as follows: Figure 3The left side is shown. The operation process in the solid box corresponds to C(x) in formula (9). Then, the obtained center difference convolution result and the subtracted result are subjected to ordinary convolution and then fused and added to obtain the final output L of the center difference convolution module. f The specific operation is shown in formula (2). Use the corresponding... Figure 2 The four-layer central difference convolutional module of CDC_2 in the code sequentially produces outputs L1, L2, L3, L4, L5, L6, L7, L8, L9, L1, L1, L2, L1, L2, L3, L9, L1, L1, L2, L1, L1, L2, L3, L1, L1, L2 ... f The previous output is the input of the next module. The output after the last central difference convolution module is L. f , means as follows:

[0048]

[0049] The outputs L1, L2, and L3 correspond to the input x values ​​L0, L1, and L2, respectively.

[0050] 3) Convert the final output L of the central difference convolution module f As input to the Fast Fourier Convolutional (FFC) module, the specific process is as follows: Figure 2 As shown in the lower section, N FFC modules are used to extract high-level semantic features with a global receptive field. In practice, N is set to 7. The specific structure of the FFC module is as follows: Figure 3 The box shows the FFC section. The left half of the box indicates that the FFC module is divided into two branches, L and G, representing the local branch and the global branch, respectively. The local branch processes features using ordinary convolution, ReLU activation, and BN batch normalization. The global branch uses the ordinary convolution, ReLU activation, and BN batch normalization operations (as shown in the right half of the FFC box) followed by a Fast Fourier Transform (FFT) to transform the input features from the spatial domain to the frequency domain. Then, ordinary convolution, ReLU activation, and BN batch normalization are performed in the frequency domain, followed by an Inverse Fast Fourier Transform (IFT) to transform the features back to the spatial domain. The residuals are then connected to the features before the transformation, and ordinary convolution is used to obtain the output features of the global branch. Finally, the features from the two branches are fused and added to obtain the final module output features. The process is as follows:

[0051]

[0052] In the above formula, x represents the input feature of the FFC module, U(x) represents the sequential operation of ordinary convolution, ReLU activation, and BN batch normalization, L(x), G(x), and F(x) represent the local branch output, global branch output, and final output feature of the FFC module, respectively, and InvFFT(·) and FFT(·) represent the inverse fast Fourier transform and fast Fourier transform, respectively. We use 7 FFC modules to obtain 7 outputs F1, F2, F3, F4, F5, F6, and F7 respectively.f The input to the first module is L. f Since the output of the previous module is the input of the next module, the final output of the FFC module is F. f .

[0053] 2. Obtain edge enhancement fusion features

[0054] After obtaining shallow features L1, L2, L3, L f and deep features F f Subsequently, the shallow features are used as skip connection features in the decoder. We then use our designed learnable guided filter (LGF) module to enhance and fuse these features for edge information, thereby obtaining more refined edge information. The specific process is as follows: Figure 2 The right half is shown. Specific details of our designed LGF module are as follows: Figure 4 As shown. Figure 4 The box LGFU (Learnable Guided Filtering Unit) is our designed learnable guided filtering unit. The input contains x and y, which represent the input image and the guide image, respectively. By using ordinary convolution with a kernel of 3×3, their variance and covariance can be calculated, corresponding to the parameters in formula (4). Then, ordinary convolution, ReLU activation, and BN batch normalization are used to obtain the final enhanced features. Figure 4 The boxed LGF section represents our designed learnable guided filter module. We designed different methods to process different input feature categories using the guided filter. Specifically, when processing skip connection features with the learnable guided filter, the filter treats the skip connection feature itself as the guided graph; when processing upsampled features, the filter treats the corresponding skip connection feature as the guided graph. The corresponding skip connection features are as follows: Figure 2 As shown, this refers to the shallow features of the skip connections that share the same LGF module as the upsampled features. Then, the output features of both are concatenated after being processed by a learnable guided filter to obtain new features. Finally, ordinary convolution, ReLU activation, and BN batch normalization are used to upsample these new features to obtain the final output of the LGF module. The specific process is as follows:

[0055]

[0056] In the above formula, Var_x represents the variance of the input image x, and Cov_xy represents the covariance between the input image and the guide image. This indicates a concatenation operation, and L_xy represents the output of the learnable guided filter unit. The output of the LGF module is as follows:

[0057]

[0058] In the above equation, L_x, L_xy, and M represent the outputs of the guided filter for the skip connection feature x, the guided filter for the upsampled feature y, and the learnable guided filter module, respectively. We use four LGF modules to enhance the edges of the features. The input of the first module is the skip connection feature L_xy. f and upsampling features F f The outputs obtained in sequence are M4, M3, M2, and M. f The previous output serves as the upsampled feature input for the next module; therefore, the final output feature of the LGF module can be obtained as M. f .

[0059] 3. Small Target Extraction

[0060] To extract small targets, we will use the final output feature M of the LGF module. f The input is fed into the last convolutional layer, and the process is as follows: Figure 2 As shown, by using a convolution operation with a kernel size of 1x1 and a stride of 1, a feature map with 1 channel is finally obtained, thus capturing the small target. The specific process is as follows:

[0061] T = Conv_L(M f (14)

[0062] T and Conv_L(·) are the final output results, the last convolution operation.

[0063] This invention proposes an infrared small target detection method based on learnable guided filtering. This method realizes multi-scale target detection with edge enhancement based on the edge distribution characteristics of small targets and the edge preservation characteristics of guided filtering, thereby increasing the details and edge information of the target and improving the detection performance of infrared small targets.

Claims

1. A method for detecting small infrared targets based on learnable guided filtering, the method comprising the following steps: Step 1: After preprocessing the data, input it into the model, use the center difference convolution module to extract shallow features, and use the fast Fourier convolution module to extract deep features. The specific steps are as follows: 1) Preprocess the data by adjusting the infrared image size to a uniform size before inputting it into the model. Use ordinary convolution and central difference convolution in the central difference convolution module to process the features and obtain different outputs. The process is as follows: Where y1 and y2 are the output features of ordinary convolution and central difference convolution, respectively, and Conv(·), Cdc(·), and x represent ordinary convolution operation, central difference convolution operation, and input feature, respectively. 2) Perform convolution and fusion operations on the two output features y1 and y2 to obtain the output of the central difference convolution module. The process is as follows: Where L and This represents the operation of fusing and adding the output and features of the central difference convolutional module. L is used as the input feature of the next central difference convolutional module, and multiple central difference convolutional modules are used to obtain the shallow output feature L of the last layer. f ; 3) Input this feature into the Fast Fourier Transform (FFT) module. After processing the input feature using local and global branches respectively, the two parts are fused and added together to obtain the deep feature output by the FFT module. The process is as follows: Where F, h(·), and l(·) represent the output, global branch operation, and local branch operation of the Fast Fourier Transform (FFT) module, respectively. F serves as the input to the next FFT module. By using multiple FFT modules, the output feature of the last module can be obtained as F. f ; Step 2: Design a learnable guided filter module. Use this module to process shallow and deep features separately, and then fuse the processed output features to obtain edge-enhanced fused features. The specific steps are as follows: 1) The correlation coefficient of the filter is calculated based on the principle of traditional guided filtering, and its expression is as follows: Where a and b represent the filter correlation coefficients, and I, p, ... Let I represent the guide image, input image, mean of the guide image, and mean of the input image, respectively. Let cov(I, p) represent the covariance of I and p, var(I) represent the variance of I, and ε represent the L2 norm regularization coefficient. 2) Based on the above coefficients, the input image, and the guide image, the filtered output can be obtained, and its expression is as follows: q=a·I+b (5) Where q represents the output of the input image after filtering; 3) Based on the operations in steps 1) and 2), the skip connection feature L f Filtering is performed, where both the input and guide graphs use L. f The filtered result can be obtained, and its expression is as follows: Where q1 represents the filtered output of the skip connection feature; 4) Based on the operations in steps 1) and 2), the upsampled feature F f Filtering is performed, where the input image is F. f Guided diagram using L f The filtered result can be obtained, and its expression is as follows: Where q2 represents the filtered output of the upsampled feature; 5) By fusing the filtered outputs of the obtained skip connection features and the filtered outputs of the upsampled features, the output of the learnable guided filter module can be obtained, and its expression is as follows: in Represents the concatenation operation, ReLU represents the activation operation, BN represents the batch normalization operation, and M represents the output feature of the first learnable guided filter module. M is used as the upsampled feature of subsequent learnable guided filter modules. Using multiple learnable guided filter modules yields the output feature M of the last module. f .

Images