Infrared small target detection method

Abstract

This invention proposes an infrared small target detection method, the implementation steps of which are: acquiring training sample sets and test sample sets; constructing an infrared small target detection network model and iteratively training it; and obtaining infrared small target detection results. In the process of training the infrared small target detection model and obtaining the infrared small target detection results, the following features are achieved: the spatial depth-based downsampling module in the Encoder retains all feature information in the channel dimension while performing downsampling, and the branch-fusion-based feature extraction module effectively extracts multi-level feature information for each sample without introducing new computational load; the attention-based feature fusion module in the Decoder can enhance high-level and low-level feature information respectively, fusing feature representations at different levels, optimizing the feature fusion effect, and effectively improving the accuracy and detection efficiency of infrared small target detection.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to an infrared small target detection method, which can be used in fields such as geographic target detection in aviation and intelligent traffic monitoring. Background Technology

[0002] Infrared small targets typically refer to targets no larger than 7×7 pixels in infrared images. These targets usually have few pixels and lack obvious shape and texture information, and can only be identified through features such as grayscale and position. Given these characteristics, the core of infrared small target detection lies in identifying tiny targets with low signal-to-noise ratios from complex infrared images. Therefore, how to perform target enhancement, background suppression, and feature extraction in infrared small target detection are key factors affecting detection accuracy. Infrared small target detection mainly employs two approaches: detection-based and segmentation-based. Segmentation-based approaches generally perform better at the pixel level.

[0003] For example, Xi'an Zhongke Lide Infrared Technology Co., Ltd. disclosed an infrared small target detection method in its patent document "An Infrared Small Target Detection Method Based on Multi-Scale Feature Fusion" (patent application number: CN202410894900.9, publication number: CN118918308A). This invention utilizes a path aggregation network framework to collect feature information at different levels into a collection module, and uses average pooling and bilinear interpolation operations to unify the feature map scale. A fusion module fuses features at various levels to generate global information, and an SE module adaptively adjusts feature weights to enhance important features and suppress secondary features. An injection module combines global information with local features to enhance the detection capability for small targets. However, its shortcomings lie in the fact that the feature fusion scheme used in this invention performs unified processing on high-level and low-level feature information before splicing, resulting in a relatively simple fusion method. Furthermore, it introduces a large amount of noise during fusion and lacks fine-grained modeling of local target information, thus leading to a still low accuracy rate in small target detection. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of the prior art by proposing an infrared small target detection method to solve the technical problem of low detection accuracy caused by weak feature extraction and feature fusion capabilities in complex backgrounds.

[0005] To achieve the above objectives, the technical solution adopted by the present invention includes the following steps:

[0006] (1) Obtain the training sample set and the test sample set:

[0007] M infrared small target images, encompassing multiple scene categories, were preprocessed, and small targets in each preprocessed infrared image were labeled. K preprocessed infrared images and their labels were then used to form a training sample set, and the remaining MK preprocessed infrared images and their labels were used to form a test sample set.

[0008] (2) Construct an infrared small target detection network model O:

[0009] An infrared small target detection network model O is constructed, consisting of a cascaded input layer, an encoder, a decoder, and a head. The encoder comprises three cascaded feature extraction modules based on branch fusion, with a spatial depth-based downsampling module loaded between adjacent feature extraction modules. The decoder comprises two attention-based feature fusion modules with a deconvolution-based upsampling module loaded at its input. The output of the last feature extraction module is cascaded with the input of the first upsampling module, and the outputs of the first two feature extraction modules are also cascaded with the input of the feature fusion module at the same level.

[0010] (3) Iteratively train the infrared small target detection network model O:

[0011] The infrared small target detection network model O is iteratively trained using the training sample set to obtain the trained infrared small target detection network model O. * ;

[0012] (4) Obtain infrared small target detection results:

[0013] The test sample set is used as the trained infrared small target detection network model O. * The input is propagated forward to obtain the infrared small target detection results corresponding to MK test samples.

[0014] Compared with the prior art, the present invention has the following advantages:

[0015] In the process of training the infrared small target detection model and obtaining the infrared small target detection results, the following are the key features of this invention: the spatial depth-based downsampling module in the Encoder retains all feature information in the channel dimension while performing downsampling, and the branch fusion-based feature extraction module can effectively extract multi-level feature information of each sample without introducing new computational load; the attention-based feature fusion module in the Decoder can enhance high-level and low-level feature information respectively, integrate feature representations of different levels, optimize feature fusion effect, and effectively improve the accuracy and detection efficiency of infrared small target detection. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the implementation of the present invention;

[0017] Figure 2 This is a schematic diagram of the infrared small target detection network in this invention;

[0018] Figure 3 This is a schematic diagram of the feature extraction module in this invention;

[0019] Figure 4 This is a schematic diagram of the feature fusion module in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0021] Reference Figure 1 The present invention includes the following steps:

[0022] Step 1) Obtain the training sample set and the test sample set:

[0023] M infrared small target images from the NUAA-SIRST and IRSTD-1K datasets, including multiple scene categories, are preprocessed and labeled. Then, K preprocessed infrared images and their labels are used to form a training sample set, and the remaining MK preprocessed infrared images and their labels are used to form a test sample set. In this embodiment, M=1427 and K=1141.

[0024] The preprocessing of the M infrared small target images acquired, which include multiple scene categories, involves the following steps:

[0025] (1a) Data augmentation techniques are used on the acquired dataset. Each original sample image is randomly horizontally flipped with a 50% probability, and then randomly scaled and blurred using bilinear interpolation. This increases the diversity of the training data, improves the generalization ability of the model, and reduces the risk of model overfitting.

[0026] (1b) In order to meet the model input requirements, the data-augmented image is randomly cropped to obtain an image with a size of 512×512;

[0027] (1c) Using normalization techniques, the pixel value distribution of the cropped image is transformed into a distribution with a mean of 0 and a standard deviation of 1, so that the various features of the image are unified to the same scale, which accelerates the convergence speed of the model. The mean values ​​of the three RGB channels are 0.485, 0.456 and 0.406, and the standard deviations are 0.229, 0.224 and 0.225, respectively.

[0028] Step 2) Construct an infrared small target detection network model O, the structure of which is as follows: Figure 2 As shown:

[0029] An infrared small target detection network model O is constructed, comprising a cascaded input layer, an encoder, a decoder, and a head. The encoder includes three cascaded feature extraction modules based on branch fusion, with a spatial depth-based downsampling module loaded between adjacent feature extraction modules. The decoder includes two attention-based feature fusion modules with a deconvolution-based upsampling module loaded at its input. The output of the last feature extraction module is cascaded with the input of the first upsampling module, and the outputs of the first two feature extraction modules are also cascaded with the input of the feature fusion module at the same level.

[0030] The input layer consists of multiple cascaded CBR modules, each of which includes stacked convolutional layers, batch normalization layers, and ReLU function layers.

[0031] The encoder, whose feature extraction module has the following structure: Figure 3 As shown, it includes three parallel batch normalization layers and a ReLU function layer cascaded with them, wherein the first and second batch normalization layers are respectively stacked with a 3×3 convolutional layer and a 1×1 convolutional layer; the downsampling module includes stacked spatial-depth layers and non-strut convolutional layers.

[0032] The decoder uses a transposed convolutional layer structure for its upsampling module; the feature fusion module structure is as follows: Figure 4 As shown, it includes channel attention blocks and spatial attention blocks, as well as computational units loaded between the two modules. The channel attention block includes parallel, stacked global average pooling layers, ReLU function layers, and Sigmoid function layers, as well as fully connected layers loaded between the global average pooling layer and the ReLU function layer, and between the ReLU function layer and the Sigmoid function layer. The spatial attention block includes parallel average pooling layers and max pooling layers, as well as convolutional layers and batch normalization layers cascaded with them.

[0033] Step 3) Iteratively train the infrared small target detection network model O:

[0034] The infrared small target detection network model O is iteratively trained using the training sample set to obtain the trained infrared small target detection network model O. * The specific steps are as follows:

[0035] (3a) Initialize the number of iterations to a, the maximum number of iterations to A, A≥1000, and the infrared small target detection network model O in the a-th iteration. a The weights and bias parameters in the equation are w.a b a Let a = 0, and in this example A = 1000;

[0036] (3b) P training samples randomly selected from the training sample set are used as inputs to the infrared small target detection network model O for forward propagation to obtain the detection results of P infrared small targets. In this example, P = 16.

[0037] (3b1) The input size of the input layer is 3×512×512. The number of channels for each training sample is increased to 8 to obtain the initial feature map F. 0p Each channel can extract different local features, enabling the network to learn richer and more complex feature representations;

[0038] (3b2) The encoder performs first-level feature extraction on each initial feature map and extracts features from the feature map F containing the first-level features. 1p After downsampling, secondary feature extraction is performed, and then the feature map F containing the secondary features is processed. 2p After downsampling, three-level feature extraction is performed to obtain a feature map F containing the three-level features. 3p ;

[0039] The feature extraction module employs a three-parallel branch structure. The first two branches contain 3×3 and 1×1 convolutional layers, respectively, while the third branch has no convolutional kernel. The size of the convolutional kernel is positively correlated with the size of the receptive field. By setting different convolutional kernel sizes, the feature extraction module enables the network to extract features of different fine-grained sizes under different receptive fields, thereby enhancing its feature extraction capabilities.

[0040] When reasoning about the infrared small target detection model, the three branches in the feature extraction module can be merged into a single branch consisting of stacked 3×3 convolutional layers, batch normalization layers, and ReLU function layers, based on the structural reparameterization technique, thereby reducing the amount of computation and improving detection efficiency.

[0041] Furthermore, for the feature map F containing first-level features 1p Perform downsampling and analyze the feature map F containing secondary features. 2p The downsampling operation is the same, as follows:

[0042] (3b21) Number the pixels sequentially along the height and width dimensions of the feature map;

[0043] (3b22) Concatenate pixels with odd-numbered labels in both height and width dimensions to form vector c1;

[0044] (3b23) Concatenate pixels with odd height numbers and even width numbers to form vector c2;

[0045] (3b24) Concatenate pixels with even-numbered height and odd-numbered width to form vector c3;

[0046] (3b25) Concatenate pixels with even-numbered labels in both height and width dimensions to form vector c4;

[0047] (3b26) Concatenate vectors c1, c2, c3 and c4 along the channel dimension and perform a non-stride convolution operation to obtain the downsampling result.

[0048] The downsampling operation of this invention differs from traditional pooling layers or stride convolutional layers. It retains all feature information in the feature map in the channel dimension, which can effectively avoid the defect of image detail loss in traditional downsampling process. It is sufficiently friendly to small targets in infrared images that contain only weak detail features.

[0049] (3b3) The decoder processes the feature map F. 3p After upsampling, it is compared with the feature map F 2p First-level feature fusion is performed to obtain a feature map Y containing second- and third-level features. Then, feature map Y is upsampled and compared with feature map F containing first-level features. 1p Perform two-level feature fusion to obtain a feature map T containing features from each level. p ;

[0050] For feature map F 3p Both upsampling and upsampling of the feature map Y use transposed convolution;

[0051] When performing feature fusion, considering that high-level features contain more semantic information and low-level features contain more spatial location information, this invention uses a channel attention mechanism to enhance the semantic information in high-level features and a spatial attention mechanism to enhance the spatial location information in low-level features, thereby enhancing the feature fusion capability of the model.

[0052] In the feature map F 3p After upsampling, it is compared with the feature map F 2p During the process of obtaining feature map Y through first-level feature fusion:

[0053] Y = L × L1 + F 2p ×H1+F 2p ×H1×L2

[0054] Where L is the feature map F 3p The upsampling results are: L1 is the low-level feature map output of the channel attention block, L2 is the output of the spatial attention block, and H1 is the high-level feature map output of the channel attention block.

[0055] After upsampling feature map Y, it is compared with feature map F1p The secondary feature fusion calculation is performed in the same way as described above. The high-level feature map input in the feature fusion module is feature map F. 2p The low-level feature map input is feature map F. 3p The upsampling result yields the feature map T. p ;

[0056] (3b4) The detection head uses a convolution operation to convert the feature map T... p The channel number is changed to 1, and the detection result is obtained;

[0057] (3c) The IoU loss function is adopted, and the loss value of the network model is calculated using the detection results of each infrared small target and its corresponding real label. Then, the gradient descent method is used, through... For weights and bias parameters w a b a The network model O for infrared small target detection in this iteration is updated. a ;

[0058] The network model loss value The calculation formula is:

[0059]

[0060] Among them, X p Y represents the region of the model's predicted target corresponding to the p-th sample in the training dataset. p Let represent the region of the real target corresponding to the p-th sample in the training dataset, |·| represents the absolute value operation, ∩ and ∪ represent the intersection and union operations respectively, and ∑ represents the summation operation.

[0061] Calculate using the chain rule For the weight parameter w a and bias parameter b a partial derivatives and And according to For w a b a Update:

[0062]

[0063] Among them, w a ′ represents the bias parameter w a The update result, b a ′ represents b a The update result, where α represents the learning rate. express For w a Take the partial derivative, express For b a Take the partial derivative;

[0064] (3d) Determine whether a≥A holds true. If so, obtain the trained infrared small target detection network model O. * Otherwise, let a = a + 1, O a =O, and execute step (3b).

[0065] Step 4) Obtain the infrared small target detection results:

[0066] The test sample set is used as the trained infrared small target detection network model O. * The input is propagated forward to obtain the infrared small target detection results corresponding to MK test samples.

[0067] The technical effects of this invention will be explained below with reference to simulation experiments:

[0068] 1. Experimental conditions and contents:

[0069] The hardware platform for the simulation experiment is as follows: Intel(R) Core i5-8400 CPU with a clock speed of 2.8GHz, 32GB of memory, and an NVIDIA GeForce RTX 4060Ti graphics card. The software platform for the simulation experiment is: Ubuntu 20.04 operating system, Python version 3.7, and PyTorch version 1.7.1.

[0070] This invention and two existing infrared small target detection methods differ in their performance measured by the normalized intersection-union ratio (nIoU) and detection rate P. d And false alarm rate F a The simulation results were compared across three metrics and are shown in Table 1.

[0071] Normalized Intersection over Union (nIoU) describes the degree of overlap between the predicted and true values ​​output by the algorithm at the pixel level. Its calculation formula is as follows:

[0072]

[0073] Where TP represents the number of positive samples predicted as positive, T represents the number of samples predicted correctly, and P represents the total number of positive samples.

[0074] Detection rate P d This represents the proportion of correctly classified positive and negative samples, and its calculation formula is:

[0075]

[0076] Where TP represents the number of positive samples predicted as positive, FP represents the number of negative samples predicted as positive, TN represents the number of negative samples predicted as negative, and FN represents the number of negative samples predicted as positive.

[0077] False alarm rate F a This represents the proportion of samples that were predicted to be positive but actually were negative. The formula for calculating this proportion is:

[0078]

[0079] Where TP represents the number of positive samples predicted as positive, and FP represents the number of negative samples predicted as positive.

[0080] 2. Analysis of experimental results:

[0081] Table 1 Simulation Comparison Results

[0082]

[0083] Referring to Table 1, compared with the prior art, the present invention achieves higher normalized intersection-union ratio (nIoU) and detection rate P on the NUAA-SIRST and IRSTD-1K datasets. d All indicators have improved, with the false alarm rate F... a The decrease in the indicator indicates that the detection accuracy has been effectively improved.

Claims

1. A method for detecting small infrared targets, characterized in that, Includes the following steps: (1) Obtain the training sample set and the test sample set: For the acquired data including multiple scene categories The infrared small target images are preprocessed, and the small targets in each preprocessed infrared image are labeled. Then... The preprocessed infrared images and their labels form the training sample set, and the remaining... The test sample set consists of preprocessed infrared images and their labels, among which ; (2) Construct an infrared small target detection network model : The construction includes cascaded input layers and encoders. decoder and detection head Infrared small target detection network model ;in It includes three cascaded feature extraction modules based on branch fusion, with spatial depth-based downsampling modules loaded between adjacent feature extraction modules; It includes two attention-based feature fusion modules with cascaded inputs loaded with deconvolution-based upsampling modules; The output of the last feature extraction module and The input of the second upsampling module is cascaded. The outputs of the first two feature extraction modules are also connected to The input ends of the feature fusion modules at the same level are cascaded; (3) Infrared small target detection network model Perform iterative training: The infrared small target detection network model was trained using the sample set. Iterative training is performed to obtain a well-trained infrared small target detection network model. ; (4) Obtain infrared small target detection results: The test sample set is used as the trained infrared small target detection network model. The input is propagated forward to obtain The infrared small target detection results corresponding to each test sample.

2. The method according to claim 1, characterized in that, Step (1) describes the acquisition of multiple scene categories. The preprocessing steps for infrared small target images are as follows: Data augmentation is performed on each infrared small target image, and each augmented infrared small target image is randomly cropped. Then, each cropped infrared small target image is normalized to obtain the preprocessed image. A small infrared target image.

3. The method according to claim 1, characterized in that, The infrared small target detection network model described in step (2) ,in: Input layer, including multiple cascaded layers Module, each The module includes stacked convolutional layers, batch normalization layers, and... Function layer; encoder The feature extraction module includes three parallel batch normalization layers and cascaded layers. The function layer consists of a 3×3 convolutional layer and a 1×1 convolutional layer before the first and second batches of normalization layers, respectively; the downsampling module includes stacked spatial-depth layers and non-strut convolutional layers. decoder The upsampling module employs a transposed convolutional layer structure; the feature fusion module includes channel attention blocks and spatial attention blocks, as well as computational units loaded between the two modules. The channel attention blocks consist of two parallel branches, each including stacked global average pooling layers. Function layer and Function layer, and loaded in the global average pooling layer and Between function layers and Function layer and Fully connected layers between function layers.

4. The method according to claim 3, characterized in that, The infrared small target detection network model described in step (3) The iterative training process involves the following steps: (3a) Initialize the number of iterations to be The maximum number of iterations is , , No. The next iteration of the infrared small target detection network model The weights and bias parameters in the data are respectively , and order ; (3b) Randomly select samples from the training sample set The training samples are for the infrared small target detection network model. The input is propagated forward to obtain Detection results of small infrared targets; (3c) Adopt The loss function is used to calculate the network model loss value based on the detection results of each infrared small target and its corresponding ground truth label. Then, the gradient descent method is used to... For weights and bias parameters , The network model for infrared small target detection in this iteration is updated to obtain the model for this iteration. ; (3d) Judgment If true, then the trained infrared small target detection network model is obtained. Otherwise, let , Then proceed with step (3b).

5. The method according to claim 4, characterized in that, The steps described in step (3b) The detection results of several small infrared targets, including the first one. The method for obtaining the detection results of small infrared targets is as follows: (3b1) The input layer adds channels to each training sample and performs preliminary feature extraction to obtain the first... Initial feature map of each training sample ; (3b2) Encoder For each initial feature map Perform first-level feature extraction and analyze the feature map containing the first-level features. After downsampling, secondary feature extraction is performed, and then the feature map containing the secondary features is processed. After downsampling, three-level feature extraction is performed to obtain a feature map containing three levels of features. ; (3b3) Decoder For feature maps After upsampling and feature map Perform first-level feature fusion to obtain a feature map containing second- and third-level features. and the feature map After upsampling and feature map Perform two-level feature fusion to obtain a feature map containing three levels of features. ; (3b4) Detection head For feature maps The test was conducted, and the result was obtained. Detection results of small infrared targets.

6. The method according to claim 5, characterized in that, Step (3b2) describes the processing of feature maps containing first-level features. The steps to perform downsampling are as follows: Feature maps containing first-level features are analyzed along both the height and width dimensions. The vectors are divided into equal intervals, and then concatenated sequentially along the channel dimension and non-stride convolution is performed to obtain the downsampled feature map.

7. The method according to claim 5, characterized in that, The feature map containing second- and third-level features described in step (3b3) The method to obtain it is as follows: ； in, For feature map The results of upsampling, , The outputs are the low-level and high-level feature maps of the channel attention block, respectively. This is the output of the spatial attention block.

8. The method according to claim 4, characterized in that, The network model loss value described in step (3c) The calculation formula is: ； in, Indicates the training dataset number 1 Each sample corresponds to the region of the model's predicted target. Indicates the training dataset number 1 Each sample corresponds to the region of the real target. This indicates the absolute value operation. , These represent the intersection and union operations, respectively. This indicates a summation operation.

9. The method according to claim 4, characterized in that, Step (3c) describes the adjustment of weights and bias parameters. , The update is performed using the chain rule, and the update formulas are as follows: ； ； in, Indicates bias parameters The update results express The update results Indicates the learning rate. express right Take the partial derivative, express right Take the partial derivative.

Images