A target size prediction and estimation method based on infrared small target identification

By constructing a U-Net structured convolutional neural network and combining deep learning with loss function optimization, the problem of target size prediction in infrared small target recognition was solved, and the accuracy of recognition and prediction was improved.

CN118379538BActive Publication Date: 2026-11-10NANJING UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410360910.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-27
Publication Date
2026-11-10
Estimated Expiration
2044-03-27

AI Technical Summary

Technical Problem

Existing infrared small target recognition technology cannot predict or estimate the size of the target, especially under low signal-to-noise ratio or complex conditions, where the application effect of traditional visible light target recognition technology is poor.

Method used

A U-Net-based convolutional neural network with encoder-decoder and residual modules is constructed. Through annotation and training on high- and low-resolution paired infrared small target image datasets, deep learning is used for image processing to generate matting labels. The network is then optimized by combining Dice Loss and BCE Loss to achieve the recognition and size prediction of infrared small targets.

Benefits of technology

It improves the recognition accuracy of small infrared targets and the prediction accuracy of target size, especially under low-resolution conditions, achieving higher recognition and prediction results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118379538B_ABST
    Figure CN118379538B_ABST
Patent Text Reader

Abstract

This invention discloses a target size prediction and estimation method based on infrared small target recognition. The steps are as follows: (1) Construct a neural network for infrared small target recognition and target size prediction; (2) Construct a dataset of infrared small target images captured in pairs with high and low resolution, label the targets in the high-resolution images with 0 and 1 labels, and then downsample the labeled images with 0 and 1 labels to the same resolution size as the low-resolution images through bilinear interpolation, that is, generate the matting labels of the corresponding low-resolution images; at the same time, obtain the target size labels in the high-resolution images; (3) Train the neural network end-to-end using the matting labels, take the low-resolution images as network input, and map the target labels of the high-resolution images to the low-resolution labels; (4) Use the trained neural network to recognize and predict the size of small targets in the images. This invention achieves higher accuracy in the recognition and target size prediction and estimation of infrared small targets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of computational photography and deep learning, and in particular to a target size prediction and estimation method based on infrared small target recognition. Background Technology

[0002] Infrared small target recognition has a wide range of applications, such as national defense, environmental monitoring, security surveillance, and biological research. Infrared small target recognition has always been a challenging problem in the field of target recognition. In recent years, the widespread use of low-cost, low-resolution infrared cameras has brought even greater challenges to infrared small target recognition in scenarios with low signal-to-noise ratios or complex conditions.

[0003] Existing infrared small target recognition technologies are generally based on traditional visible light target recognition technologies, which greatly improve the accuracy of infrared small target recognition. However, these technologies cannot predict or estimate the size of the target. Summary of the Invention

[0004] In view of the above analysis of existing technologies and existing problems, the purpose of this invention is to propose a target size prediction and estimation method based on infrared small target identification.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A target size prediction and estimation method based on infrared small target identification includes the following steps:

[0007] Step 1: Construct a neural network for infrared small target recognition and target size prediction. This neural network is used to recognize infrared small targets and predict target size.

[0008] Step 2: Construct a dataset of real-world infrared small target images with paired high and low resolution images. Label the targets in the high-resolution images with 0 and 1 tags. Then, downsample the labeled images with 0 and 1 tags to the same resolution size as the low-resolution images using bilinear interpolation, thus generating the matting labels for the corresponding low-resolution images. At the same time, obtain the target size labels in the high-resolution images.

[0009] Step 3: Using the matting labels of the generated low-resolution images, train the neural network constructed in Step 1 end-to-end, taking the low-resolution images as network input and mapping the target labels of the high-resolution images to the low-resolution labels.

[0010] Step 4: Use the trained neural network to identify and predict the size of small targets in the image.

[0011] This invention is based on the idea of ​​optimizing infrared small target recognition methods in deep learning. First, it jointly collects high- and low-resolution paired infrared small target datasets, uses a novel method to annotate the datasets, and then performs image processing based on deep learning to construct a U-Net structured convolutional neural network based on an encoder-decoder and residual modules. This achieves higher accuracy in infrared small target recognition and target size prediction estimation. Attached Figure Description

[0012] Figure 1 This is a flowchart of the method of the present invention;

[0013] Figure 2 This is a schematic diagram of a binocular infrared camera imaging system in an embodiment of the present invention;

[0014] Figure 3 This is a schematic diagram of the real-shot dataset and the manual marking of infrared target tags in an embodiment of the present invention. Detailed Implementation

[0015] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0016] Reference Figure 1 The target size prediction and estimation method based on infrared small target identification in this embodiment has the following specific steps:

[0017] Step 1: Construct a neural network for infrared small target recognition and target size prediction. This neural network includes a U-Net structure convolutional neural network with an encoder / decoder and residual modules, which is used for infrared small target recognition and target size prediction.

[0018] Step 2: Acquire infrared small target images from real-world scenes, constructing a dataset of paired high- and low-resolution real-world infrared small target images. In this embodiment, the resolutions of the high- and low-resolution infrared images are 640*480 and 256*192, respectively. A binocular infrared camera imaging system composed of paired high- and low-resolution infrared cameras is designed, such as... Figure 2 As shown, this method is used to capture high- and low-resolution information of small targets. This embodiment utilizes an infrared thermal imager IRSV-Hinet-619 and an infrared camera InfiRay-Xmodule-S0 to capture high-resolution and low-resolution infrared images respectively, but is not limited to these two devices. To optimize the use of information from the captured image pairs, effective tagging is created, and the high-resolution and low-resolution images are aligned to the same viewpoint through binocular calibration. This process involves using an affine transformation to calibrate the field of view between the high-resolution and low-resolution infrared cameras, thereby ensuring that the images of both resolutions share the same viewpoint. Corresponding feature points are identified in the low-resolution and high-resolution images, their pixel coordinates are determined, and the parameters of the affine transformation are calculated. These parameters are then used to accurately generate high-resolution and low-resolution image pairs from the same viewpoint.

[0019] Targets in high-resolution and low-resolution infrared images are labeled. First, small targets in high-resolution images are manually labeled, with target areas marked as 1 and non-target areas (background areas) marked as 0. Simultaneously, the target size (number of pixels occupied) in the high-resolution image is also labeled. Besides the binary 0,1 labels indicating the presence or absence of a target, matting labels from matting algorithms in computer vision tasks can provide additional information about the distribution characteristics of the target's foreground and background. In image pairs, interference between the target foreground and background is negligible in high-resolution images due to the relatively clear boundaries of foreground targets (such as drones). However, in low-resolution images, because the feature size of the target is smaller than the pixel size, the radiant energy of the target foreground and background mixes at the target's edge regions. Furthermore, images acquired by low-quality infrared imaging devices make distinguishing edge pixels from foreground and background more complex. This ambiguity poses a challenge to accurately labeling small targets in low-resolution infrared images. Therefore, generating matting labels for low-resolution images can introduce effective supervision and better optimize the training of high-performance infrared small target recognition networks.

[0020] Bilinear interpolation is used to downsample the labeled high-resolution image (with 0,1 labels) to the same resolution as the low-resolution image, generating matting labels for the corresponding low-resolution infrared image. In bilinear interpolation, the target point value is calculated by weighting the pixel values ​​of its neighboring points. In the labeled low-resolution image generated by bilinear interpolation, the edge region of the target is blended with the background, producing a value between 0 and 1 representing the foreground opacity, which can represent the proportion of the region belonging to the foreground object, approximating the energy level of the region. In short, if a pixel belongs entirely to the background, its pixel value is labeled as 0; if it belongs entirely to the foreground (i.e., the target), its pixel value is labeled as 1. In the matting labeling of small targets in the low-resolution image, the central region of the small target will be labeled as 1, pixel regions belonging entirely to the background will be labeled as 0, and the edges of the small target will be labeled with values ​​between 0 and 1 according to the proportion of foreground and background.

[0021] Step 3: Using the matting labels obtained from the low-resolution infrared images, train the neural network constructed in Step 1 end-to-end. The low-resolution infrared images are used as network input, and the following loss function is employed to constrain the overall optimization of the network:

[0022]

[0023] Among them, Y b and Y′ bY represents the true value and predicted value of the (0,1) label, respectively. m and Y′ m λ represents the true and predicted values ​​of the matting label, respectively. matting , λ size The parameters are used to adjust the proportions of each loss function.

[0024] L Dice Dice Loss (Dice Coefficient Loss) is a loss function used for image segmentation tasks, such as semantic segmentation and medical image segmentation. The calculation formula is as follows:

[0025] L Dice = 1 - 2|Y∩Y′| / (|Y′|+|Y|)

[0026] Where Y is the true value, Y′ is the predicted value, and Y∩Y represents the intersection of the predicted result and the true result.

[0027] L BCE BCE Loss (Binary Cross-Entropy Loss) is a loss function used for binary classification tasks. It primarily measures the difference between the predicted values ​​of a binary classification model and the true labels. The calculation formula is as follows:

[0028]

[0029] Where N is the total number of sums, i is the starting value, n is the ending value, and Y is the sum of all possible sums. i Y represents the true value. i ′ represents the predicted value.

[0030] L matting This is the loss function used to optimize the network based on matting labels. It estimates the energy of the target by analyzing the pixel values ​​of the target in the matting labels. The calculation formula is as follows:

[0031] L matting (Y m ,Y′ m )=λ1L BCE (Y m ,Y′ m )+λ2L Energy (Y m ,Y′ m )

[0032]

[0033] Among them, E i This represents the sum of the pixel values ​​of the target in the matting tag. For the corresponding predicted values, the hyperparameters λ1 = 0.6, λ2 = 0.4, Y m Y represents the true value. m ′ represents the predicted value.

[0034] The loss function, which constrains the predicted target size, is calculated using the following formula:

[0035]

[0036] Where s gt s represents the true value of the target size. predict The predicted value for the target size; updated by labeling the target size. Constrain the size of the prediction target.

[0037] Step 4: Using the trained U-Net structure convolutional neural network, perform infrared small target recognition and target size prediction on the test set images acquired in Step 2, and output the recognition and prediction results.

Claims

1. A target size prediction and estimation method based on infrared small target recognition, characterized in that, Includes the following steps: Step 1: Construct a neural network for infrared small target recognition and target size prediction. This neural network includes a U-Net structure convolutional neural network with an encoder-decoder and residual modules, used to recognize infrared small targets and predict target size. Step 2: Construct a dataset of paired high- and low-resolution infrared small target images. Acquire high-resolution and low-resolution infrared images, and use affine transformation to calibrate the field of view between the high-resolution and low-resolution infrared images, ensuring they align with the same viewpoint. Label the targets in the high-resolution infrared images with 0 and 1 tags. Then, downsample the labeled images with 0 and 1 tags to the same resolution as the low-resolution infrared images using bilinear interpolation, generating matting labels for the corresponding low-resolution infrared images. In the matting label image of the low-resolution infrared images generated by bilinear interpolation, the edge regions of the targets will blend with the background, producing values ​​between 0 and 1 to represent foreground opacity, indicating the ratio of the edge region to the foreground object. The central region of the small target is labeled as 1, and pixel regions completely belonging to the background are labeled as 0. Simultaneously, obtain the target size and the number of pixels it occupies in the high-resolution infrared images. Step 3: Using the matting labels of the generated low-resolution infrared images, train the neural network constructed in Step 1 end-to-end, using the low-resolution infrared images as input to the neural network. Step 4: Use the trained neural network to identify and predict the size of small targets in the image; In step 3, the following loss function is used to constrain the overall optimization of the network: in, and These represent the actual and predicted values ​​for labels 0 and 1, respectively. and These represent the actual and predicted values ​​of the matting label, respectively. Here is the loss function used for image segmentation tasks. Here is the loss function used for binary classification tasks. Let be the loss function used to optimize the network based on matting labels. A loss function to constrain the size of the predicted target; , The parameter is used to adjust the proportion of its corresponding loss function; loss function The calculation formula is as follows: in, This represents the sum of the pixel values ​​of the target in the matting tag. For the corresponding predicted value, ; loss function The calculation formula is as follows: in The true value of the target size. This is the predicted value for the target size.

2. The target size prediction and estimation method based on infrared small target recognition according to claim 1, characterized in that, loss function The formula used to measure the difference between the predicted value and the true label of a binary classification model is as follows: Where N is the total number of sums, i is the starting value, and n is the ending value. For the true value, These are predicted values.

Citation Information

Patent Citations

  • Interactive image matting using neural networks

    CN111754517A

  • Airborne infrared image super-resolution and noise removal algorithm based on twin network

    CN114170094A