An infrared dim small target detection method based on key point representation

Abstract

The application discloses an infrared dim small target detection method based on key point representation, first constructs a detection network main framework based on an encoder-decoder structure, and a feature extraction network based on VGG16; according to the position coordinates of a target center point, a suitable Gaussian radius is selected to calculate a key point heat map according to the Gaussian distribution of the center point coordinates, and a label map of a training image is converted into a corresponding key point heat map; after the training image is preprocessed, the training image is sent into the detection network, network training is performed, a loss function of the detection network is calculated, parameter updating is performed, repeated training is performed until training convergence is achieved, and finally, a network model is obtained; a test image is sent into the detection network, a heat map result is output, a peak area of the heat map is obtained by selecting a suitable threshold, and finally, a detection result is output. The method directly optimizes and positions a target center point coordinate, realizes end-to-end detection of an infrared dim small target level, and effectively improves the detection speed and detection precision of the infrared dim small target.

Description

Technical Field

[0001] This invention belongs to the field of image processing and pattern recognition technology, and particularly relates to access control technology for information security. It mainly relates to an infrared weak target detection method based on key point representation. Background Technology

[0002] Military security holds a very important position in national security, and all-weather detection and tracking of targets such as aircraft and tanks in the sky and on land has strong practical significance. As a new technology, infrared image weak target detection technology has become an effective supplement to traditional visible light detection systems because infrared imaging is only sensitive to temperature and material properties, which makes up for the shortcomings of visible light, such as being greatly affected by environmental factors such as light intensity. It can be applied to many military fields such as maritime surveillance, early warning systems, and precision guidance. However, due to the characteristics of infrared images themselves, the inherent characteristics of infrared weak targets are: (1) small target size: due to the long imaging distance, the target size is generally between 3×3 and 9×9 pixels; (2) weak target signal: the background information of the image is noisy, the signal-to-noise ratio of the target is low, and the target is easily mixed into the background; (3) unclear target structural features: the target does not have obvious texture features, and the shape of the target varies greatly from target to target. Therefore, traditional infrared weak target detection methods are not robust and have weak generalization ability.

[0003] In existing technologies, many scholars have begun to study infrared weak target detection. Related works include: Zhao Kun's spatial filter method utilizing grayscale differences (Zhao Kun, Kong Xiangwei. Suppression of Background Noise in Infrared Images of Small Targets and Discussion of Methods [J]. Optics and Optoelectronics Technology, 2004(02):9-12.), and Hou Xiaodi's detection method based on spectral residuals (HOU X, ZHANG L. Saliency detection: A spectral residual approach [C] / / 2007 IEEE Conference on computer vision and pattern recognition. IEEE, 2007:1-8.), etc. With the success of deep learning in target detection, semantic segmentation, and other fields, researchers are attempting to apply deep learning methods to the field of infrared weak target detection. Related works include: the densely connected detection network proposed by Li et al. (LIB, XIAO C, WANG L, et al. Dense Nested Attention Network for Infrared Small Target Detection[J]. arXiv preprint arXiv:2106.00487,2021.), and the Transformer-based infrared small target detection network proposed by Liu et al. (LIU F, GAO C, CHEN F, et al. Infrared Small-Dim Target Detection with Transformer under Complex Backgrounds[J]. arXiv preprint arXiv:2109.14379,2021.), etc. However, these networks often fail to meet the requirements of lightweight and real-time performance. At the same time, deep learning detection algorithms based on semantic segmentation are prone to "oversegmentation" and "undersegmentation" phenomena, which affect the detection results. Summary of the Invention

[0004] This invention addresses the problems in existing technologies by providing a keypoint-based infrared weak target detection method. First, it constructs a detection network framework based on an encoder-decoder structure, using a VGG16 feature extraction network. Then, based on the target center point coordinates, a suitable Gaussian radius is selected to calculate a keypoint heatmap according to a Gaussian distribution, converting the label map of the training image into the corresponding keypoint heatmap. The preprocessed training image is then fed into the detection network for training, calculating the network's loss function, updating parameters, and repeating the training process until convergence. The network structure and parameters are saved to obtain the final network model. The test image is then fed into the trained detection network, outputting the heatmap results. A suitable threshold is selected to obtain the peak region of the heatmap, and the final detection result is output. This method directly optimizes and locates the target center point coordinates, achieving end-to-end detection at the infrared weak target level. It effectively improves the detection speed and accuracy of infrared weak targets, offering a simple structure, high detection accuracy, and fast speed.

[0005] To achieve the above objectives, the technical solution adopted by this invention is: an infrared weak target detection algorithm based on key point representation, comprising the following steps:

[0006] S1, Constructing the detection network: Construct the main framework of the detection network based on the encoder-decoder structure, and the feature extraction network based on VGG16;

[0007] S2, Label Image Conversion: The binary label image of the training image is processed. Based on the position coordinates of the target center point, the key point heatmap is calculated according to the Gaussian distribution of the center point coordinates with Gaussian radius r, and the label image of the training image is converted into the corresponding key point heatmap.

[0008] S3, Training image input: For the training image, the training image is preprocessed and then sent to the detection network. The preprocessing step includes at least resizing the training image to the same size and converting it into a single-channel grayscale image.

[0009] S4, Network Training: Based on the input from step S3, calculate the loss function of the detection network and update the parameters. The loss function of the detection network, SmoothL1Loss, is shown in the following formula:

[0010] loss = SmoothL1Loss(S,G)

[0011] Where S and G represent the output of the detection network and the true value of the heatmap, respectively;

[0012] S5, Network Model Acquisition: Repeat steps S3 and S4 until training converges, save the network structure and parameters, and obtain the final network model;

[0013] S6, Test execution: The test image is fed into the detection network that has been trained through the above steps, the heat map result is output, and the peak region of the heat map is obtained according to the threshold T, and the final detection result is output.

[0014] Compared with existing technologies, this invention addresses the inherent characteristics of infrared weak targets by designing a keypoint detection-based method for detecting infrared weak targets. It transforms the detection problem into a keypoint detection problem, directly optimizing and locating the target's center point coordinates, effectively improving both detection speed and accuracy. Furthermore, the main framework of the detection network is designed based on an encoder-decoder structure. To prevent the loss of small targets, a VGG16-based feature extraction network is constructed. The convolutional neural network detection framework in this invention inputs the feature maps from the lower and middle layers of the encoder into a pre-designed feature fusion module, ensuring effective utilization of information at different scales. For higher-level feature maps, a pyramid pooling module is used to enhance the network's ability to extract contextual information. The SmoothL1 loss function is used to calculate the network loss function, reducing training difficulty and increasing training speed. The proposed method has a simple overall structure, high detection accuracy, and fast speed, and can be applied to many military fields such as maritime surveillance, early warning systems, and precision guidance, significantly improving both effectiveness and work efficiency. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the infrared weak target detection method based on key point representation of the present invention.

[0016] Figure 2 This is a schematic diagram of the detection network framework based on the encoder-decoder structure constructed in step S1 of the infrared weak target detection method based on key point representation of the present invention.

[0017] Figure 3 This is a schematic diagram of the feature fusion module constructed in step S1 of the infrared weak target detection method based on key point representation of the present invention.

[0018] Figure 4 This is a schematic diagram of the pyramid pooling module used in step S1 of the infrared weak target detection method based on key point representation of the present invention.

[0019] Figure 5 This is a visual heatmap of the experimental data labels in the test examples of this invention;

[0020] Figure 6 This is a graph showing the detection effect of experimental data in the test examples of this invention. Detailed Implementation

[0021] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0022] Example 1

[0023] A method for detecting weak infrared targets based on key point representation, such as Figure 1 As shown, it includes the following steps:

[0024] Step 1: Construct a detection network based on the encoder-decoder structure.

[0025] VGG-16 is used as the feature extraction network. Considering the inherent properties of infrared weak targets, to prevent the loss of small targets and to simplify the network structure, the last two VGG blocks in VGG-16 are discarded, so that the maximum downsampling of the network is 1 / 8. The design idea of ​​this network includes three points: (1) Based on the classic encoder-decoder structure, a fusion module (Bottom Salient to Top, BST) is designed, which effectively utilizes contextual information. The more accurate positional information of the low level is fused with the higher semantic information of the high level, which effectively increases the feature extraction and expression capabilities. (2) A pyramid pooling module (PPM) is used between the encoder and decoder to extract global features under different receptive fields, effectively fusion of global contextual information, and enhance the representation capability of features. (3) The network label is a target point heatmap calculated based on the coordinates of the target center point according to the Gaussian distribution. At the same time, heatmap regression is used instead of coordinate point regression to optimize the target center point coordinates, which has a stronger generalization ability.

[0026] The detection network in this embodiment includes at least a feature fusion module and a pyramid pooling module, such as Figure 2 As shown, the Feature Fusion Module (BST) and Pyramid Pooling Module (PPM) are as follows:

[0027] (1) BST Module: First, pooling layers of different sizes are used to extract and fuse low-level information to obtain multi-scale salient information with the highest response values ​​at the low level, reducing the impact caused by different target sizes. The extracted information is then used to perform attention weighting on high-level information with high semantic information, so that the high-level information has both high semantic information and relatively accurate location information. The final designed feature fusion module BST is as follows: Figure 3 As shown.

[0028] (2) PPM Module: This module employs four different max-pooling sizes to pool the feature maps to a specified size. Then, it uses 1×1 convolutions to reduce the number of channels in the feature maps and upsamples them to the original feature map size. Finally, it concatenates all feature maps with the original feature maps by channel. The PPM module pools the feature maps to different sizes during the pooling process, thereby acquiring global information at different scales. The final PPM module used is as follows: Figure 4 As shown.

[0029] Step 2: Based on the location of the target center point, select a Gaussian radius r = 5, and convert the label map of the training image into a corresponding heatmap for use in training the detection network.

[0030] Step 3: Input a batch of training images and adjust their size to 128×128.

[0031] Step 4: Calculate the loss function of the detection network and update the parameters. The input image is labeled I, the output image is labeled S, and the transformation between the input and output images is denoted as:

[0032] F(I)=S

[0033] The network loss function adopted is SmoothL1Loss, and the network loss function is shown in the following formula:

[0034] loss = SmoothL1Loss(S,G)

[0035] In the above formula, S and G represent the output result of the detection network and the true value of the heatmap, respectively.

[0036] After the loss function is calculated, the network parameters are updated by backpropagating the error using stochastic gradient descent, thus completing one training cycle.

[0037] Step 5: Proceed to Step 3 until the network converges, obtaining the final network model.

[0038] Step 6: In the testing phase, input the test image into the trained network model for infrared weak target detection and output the heatmap results. Set the threshold T=0.8 to filter out the peak areas of the heatmap, thus obtaining the final detection results.

[0039] Test case

[0040] This test was conducted on publicly available infrared weak target detection datasets (NJUST-SIRST and NUAA-SIRST) to verify the effectiveness of the invention. NJUST-SIRST contains 10,000 training samples and 100 test samples. The infrared weak target image samples in this dataset are randomly combined with actual or simulated weak targets and actual sky, sea, and building background images. NUAA-SIRST contains 256 training set samples, 85 validation set samples, and 86 test set samples. The infrared weak target image samples in this dataset were captured by real infrared cameras and manually labeled.

[0041] The evaluation indicators used are target detection rate and target false alarm rate.

[0042] Detection rate: Detection rate (P) d () is the target-level evaluation metric, which is the ratio of the number of correctly detected targets to the total number of targets. The calculation formula is shown below:

[0043]

[0044] Among them, T correct and T all These represent the number of correctly detected targets and the total number of targets, respectively. The method for determining whether a target point is correctly detected is as follows: if the pixel deviation between the predicted target center point coordinates and the actual target center point coordinates is less than a threshold, then the target is considered correctly predicted.

[0045] False alarm rate: False alarm rate (F) a Detection rate is another target-level evaluation metric. Simply pursuing a high detection rate is insufficient, as improving the detection rate inevitably increases the probability of false detections. Therefore, the false alarm rate (False Alarm Rate) is used to measure false detections. The False Alarm Rate is the ratio of the number of incorrectly predicted targets to the total pixel size of the image, calculated using the following formula:

[0046]

[0047] Among them, P correct P is the number of targets predicted incorrectly. all It is the pixel size of the entire image. When the pixel deviation between the predicted target center point coordinates and the actual target center point coordinates is greater than the threshold a (a=3), the target is considered to be predicted incorrectly.

[0048] The input to this invention is an infrared image and a heatmap of the target's center point, while the labels in the dataset are binarized images. Therefore, it is first necessary to calculate the coordinates of the target's center point. Then, based on the Gaussian radius r (r=5), the labels are calculated according to a Gaussian distribution of the center point coordinates, such as... Figure 5 As shown.

[0049] This test compared mainstream methods in the field of infrared weak target detection, including traditional methods such as GST, ILCM, LIG, NRAM, PSTNN, and TLLCM, as well as deep learning-based methods such as DNANet and MDvsFA. The results of different algorithms are shown in Table 1. Table 1 shows the P-values ​​of different algorithms on different datasets. d With F a The size of the indicator, P d The higher the value of the indicator, the better, F a The smaller the metric, the better. As shown in Table 1, this invention achieves the best balance between detection rate and false alarm rate on both datasets, resulting in the best detection performance.

[0050] Table 1 Comparison of results from different algorithms

[0051]

[0052] Figure 6 The detection effect of the present invention is shown. From left to right, the images are the input image, the binarized label image, and the detection result image. It can be seen intuitively that the present invention has a good detection effect.

[0053] This invention proposes a keypoint detection-based method for detecting weak infrared targets. It transforms the problem of detecting weak infrared targets into a keypoint detection problem. Given an image, an encoder is first designed to extract features, resulting in an intermediate feature map. Then, a fusion module is designed to fuse the features from the intermediate feature map, outputting a heatmap. Finally, the target's center point coordinates are determined by the peak values ​​in the heatmap. This method, by directly locating the target's center point coordinates, achieves end-to-end detection at the level of weak infrared targets, greatly simplifying the detection process, accelerating detection speed, improving detection accuracy, and enhancing robustness. During network training, the labels of the original training set need to be processed, i.e., a suitable Gaussian radius is selected to convert them into a keypoint heatmap suitable for network training. The keypoint detection-based method for detecting weak infrared targets proposed in this invention avoids the impact of "undersegmentation" and "oversegmentation" on target-level detection metrics, which is common in semantic segmentation-based detection methods, thus ensuring both detection speed and target-level detection accuracy.

[0054] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. An infrared weak target detection algorithm based on key point representation, characterized in that, Includes the following steps: S1, Constructing the Detection Network: Construct the main framework of the detection network based on the encoder-decoder structure, using a VGG16 feature extraction network. The main body of the detection network includes at least a feature fusion module and a pyramid pooling module. In the feature fusion module, the feature maps of the lower and middle layers in the encoder are input to the feature maps of the corresponding upper layer in the decoder, so that the more accurate positional information of the lower layer is fused with the more advanced semantic information of the higher layer. In the pyramid pooling module, the feature maps are pooled to different sizes during the pooling process to obtain global information at different scales. The specific working steps of the feature fusion module are as follows: using pooling layers of different sizes to extract and fuse low-level information to obtain multi-scale salient information with the highest low-level response value, and then using the extracted information to perform attention weighting on high-level information with high semantic information. The specific working steps of the pyramid pooling module are as follows: using four different max-pooling sizes, the feature maps are pooled to 1 respectively. 1,2 2, 3 3, 6 6. Four different sizes, then through 1 1. Convolution is used to reduce the number of channels in the feature map and upsample it to the original feature map size. Finally, all feature maps are concatenated with the original feature map by channel. S2, Label Image Conversion: The binary label image of the training image is processed. Based on the position coordinates of the target center point, the key point heatmap is calculated according to the Gaussian distribution of the center point coordinates with Gaussian radius r, and the label image of the training image is converted into the corresponding key point heatmap. S3, Training image input: For the training image, the training image is preprocessed and then sent to the detection network. The preprocessing step includes at least resizing the training image to the same size and converting it into a single-channel grayscale image. S4, Network Training: Based on the input from step S3, calculate the loss function of the detection network and update the parameters. The loss function of the detection network, SmoothL1Loss, is shown in the following formula: ； Where S and G represent the output of the detection network and the true value of the heatmap, respectively; S5, Network Model Acquisition: Repeat steps S3 and S4 until training converges, save the network structure and parameters, and obtain the final network model; S6, Test execution: The test image is fed into the detection network that has been trained through the above steps, the heat map result is output, and the peak region of the heat map is obtained according to the threshold T, and the final detection result is output.

2. The infrared weak target detection algorithm based on key point representation as described in claim 1, characterized in that, In step S2, the Gaussian radius r is set to 5; in step S3, the size of the preprocessed image is uniformly 128*128.

3. The infrared weak target detection algorithm based on key point representation as described in claim 2, characterized in that, After the loss function is calculated in step S4, the network parameters are updated by backpropagating the error using stochastic gradient descent, thus completing one training cycle.

4. The infrared weak target detection algorithm based on key point representation as described in claim 3, characterized in that, In step S6, the threshold T is set to 0.8.

Images