Infrared weak and small target identification system based on unmanned flight platform

Abstract

According to the infrared weak and small target identification system based on the unmanned flight platform, the infrared small sample problem is solved from the aspect of deep learning, the characteristics of the infrared small target are considered, such as the characteristic that the infrared small target is easily submerged by the background and few inherent characteristics in a deep network, and the characteristics of different responses of positive and negative samples to a loss function, and the like. And a deep learning network LCARNet is selected as a detection method, so that the segmentation precision of the whole pair of infrared weak and small targets is effectively improved, and a leading level is achieved. Firstly, infrared image samples of target object models shot at high altitude at different distances and angles are obtained to serve as a data set, and then a training set needs to be labeled to obtain a labeled graph. In network training, an infrared image and an annotated image need to be input into the network, and a high-precision target segmentation network based on an infrared target image is obtained through training, so that the infrared weak and small target segmentation precision is improved. The method is characterized in that 1) infrared weak and small target detection is carried out; and 2) the robustness is strong, and the method adapts to any angle and height.

Description

Technical Field

[0001] The invention relates to the field of computer technology, in particular to the field of infrared image small target segmentation technology, and specifically to an infrared small target recognition system based on an unmanned aerial platform. Background Art

[0002] Traditional neural networks are typical shallow models because of their simple structure. Due to the variable size, direction and high shooting height of infrared small targets, it is difficult to obtain more detailed and effective recognition features using traditional deep learning methods, and the performance is poor in infrared image recognition. At the same time, in the downsampling process, the detection of small infrared targets will encounter imaging problems, missing texture features and disappearance of targets, resulting in low accuracy. Summary of the invention

[0003] The technical problems that need to be solved by the technical solution of the present invention are: the recognition features are not detailed, the performance in the infrared image recognition problem is poor, and the detection of small infrared targets during the downsampling process encounters problems such as imaging areas, missing texture features and disappearance of targets.

[0004] In order to solve the above technical problems, the technical solution of the present invention provides an infrared small target recognition system based on an unmanned aerial platform, comprising:

[0005] An unmanned aerial platform equipped with an infrared camera obtains an initial image corresponding to a small target and stores it in a first folder, annotates the initial image in the first folder to obtain an annotated image and a trained initial image, stores the annotated image in a second folder, stores the trained initial image in the first folder, and the initial image, the trained initial image and the annotated image constitute a data set;

[0006] Use the LCAGNet deep learning network architecture, set dynamic learning rate and loss function, and use the dataset as training data to obtain the initial segmentation network model;

[0007] Initialize a maximum standard IOU to 0. In each round of iteration, calculate the current IOU based on the output of the initial segmentation network model and the annotated image, and compare the current IOU with the maximum standard IOU loss. If the current IOU is larger than the maximum standard IOU, save the weight file of the initial segmentation network model, set the maximum standard IOU value to the current IOU value, and obtain the structure file and weight file of the final segmentation network model.

[0008] Preferably, the first folder and the second folder are in the same main folder.

[0009] Preferably, the data annotation method is as follows:

[0010] Use the naked eye to observe the specific location of the weak target in the initial image;

[0011] Connect the weak and small targets along the outer contour to obtain a closed connected area and a non-connected closed area;

[0012] All pixels in the connected closed area are assigned a pixel value of 0, and all pixels in the unconnected closed area are assigned a pixel value of 255;

[0013] Save the initial image after assigning pixels as a binary image, making sure the image channel is 1;

[0014] Use the pytorch framework for network training, choose to use the AdamW optimizer, initialize the learning rate to 1e-4, batchsize to 8, and perform preset rounds of training. After training, the initial image and annotated image are enhanced, scaled to a length and width of 256 pixels, and normalized, and stored in the first folder and the second folder respectively.

[0015] Preferably, the loss function formula is:

[0016]

[0017] Among them, y i,j ∈R H×W is a label map, whose values ​​are 0 and 1, p i,j ∈R H×W It is the network output value after being processed by the sigmoid activation function σ.

[0018] The technical solution of the present invention proposes an infrared small target recognition system based on an unmanned aerial platform. First, infrared image samples of target object models taken at different distances and angles from high altitude are obtained as a data set. Then the data set is divided into a non-overlapping training set and a test set. The infrared target image features are obtained through the LCAGNet deep neural network model. Finally, the extracted feature information is input to perform infrared target segmentation. The infrared small target recognition system can reduce the difficulty of infrared target image feature extraction, reduce the complexity of the model, and achieve good segmentation effect and real-time computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is the overall training flow chart;

[0020] Figure 2 This is the structural diagram of LCAGNet. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0022] The infrared small target recognition system based on the unmanned aerial platform provided by the embodiment of the present invention first collects relevant pictures through the flight equipment and annotates them to form a data set. The data in the data set is input into the deep learning network LCAGNet, and the network is trained by error back propagation to improve the segmentation accuracy of infrared small targets. Specifically, the infrared small target recognition system based on the unmanned aerial platform includes:

[0023] Use an airborne drone to take aerial photos of the target object. The target object is not limited to type, height, or angle. The camera is limited to an infrared camera. Data annotation is performed on the captured images. The original images captured by the camera are placed in a folder named images. Then the images in the images file are annotated. The annotated images are placed in a folder named masks. These two files are collectively referred to as data set D. That is, using a flying device equipped with an infrared camera, the object to be identified can be photographed at any angle, height, and distance. In the InfraredImage (main folder, for example), two subfolders named images (first folder, for example) and masks (second folder, for example) are created under the InfraredImage file. All the captured data images are stored in the InfraredImages / images / folder. Then all the data under the InfraredImages / images file is annotated. The annotation method is as follows:

[0024] Open the image and use your naked eyes to observe the specific location of the target to be identified in the image;

[0025] Use a brush to connect the target to be identified along the outer contour to obtain a closed connected area A;

[0026] All pixels in the connected closed area A are assigned black, that is, the pixel value is 0, and all pixels in the unconnected closed area B are assigned white, with a pixel value of 255;

[0027] Save the image as a binary image, make sure the image channel is 1, and the pixel value has only two values, 0 and 255;

[0028] Use the pytorch framework for network training, choose to use the AdamW optimizer, initialize the lr_base learning rate to 1e-4, batchsize to 8, and train a total of 300 rounds (preset rounds, here is an example), and use the Nvidia A800 graphics card;

[0029] Use the command pip install albumentations to install the Python data enhancement package albumentations for image enhancement;

[0030] Use RandomRotate90() from the albumentations data package on the source image;

[0031] Continue using albumentations.transforms.Flip() on the source image;

[0032] Continue to use albumentations.Resize(256,256) on the source image to scale the image to a length and width of 256 pixels;

[0033] Continue to use albumentations.transforms.Normalize() on the source image to normalize the image;

[0034] Use albumentations.Resize(256,256) to scale the annotated image to 256 pixels in length and width.

[0035] Continue to use albumentations.transforms.Normalize() on the annotated image to normalize the image.

[0036] The infrared small target detection model is trained using the LCAGNet deep learning network architecture on dataset D. The images and annotated images are input into the neural network, and the loss function of the network structure is defined:

[0037]

[0038] y i,j ∈R H×W is a label map, whose values ​​are 0 and 1; p i,j ∈R H×W It is the network output value after being processed by the sigmoid activation function σ.

[0039] The details are as follows:

[0040] Convert the source image and the annotated image into the tensor format required by pytoch;

[0041] The current round number is Ep_n, and the learning rate is set to a dynamic learning rate. The specific attenuation equation is: lr_base*(1-Ep_n / 300)^(0.9);

[0042] The loss function is set to

[0043] y i,j ∈R H×W is a label map, whose values ​​are 0 and 1; p i,j ∈R H×W It is the network output value after being processed by the sigmoid activation function σ;

[0044] Using the LCAGNet network, a high-precision target segmentation model M based on infrared images is obtained through a back-propagation method based on chain gradient descent.

[0045] The details are as follows:

[0046] Use chained gradient backpropagation to iteratively update deep learning network parameters;

[0047] The deep learning network LCAGNet is trained iteratively, and the weights are saved every 10 rounds during the iteration process. The weight saving rules are as follows:

[0048] Initialize a maximum standard IOU to 0;

[0049] In each iteration, the current IOU (intersection over union) is calculated based on the network output and the annotated image.

[0050] Compare the current IOU with the maximum standard IOU loss;

[0051] If the current IOU is larger than the maximum standard IOU, save the weight file of the current network and set the value of the maximum standard IOU to the current IOU value;

[0052] Complete the LCAGNet network training and save the weight file and model structure file of the trained network M.

[0053] The embodiment of the present invention solves the problem of small infrared samples from the perspective of deep learning, taking into account the characteristics of small infrared targets, such as the characteristics of being easily submerged by the background and the fewer inherent characteristics in the deep network, the different responses of positive and negative samples to the loss function, etc., and selects the deep learning network LCAGNet as the detection method, which effectively improves the segmentation accuracy of the entire pair of infrared weak targets and reaches the leading level. First, infrared image samples of target object models taken at different distances and angles from high altitudes are obtained as a data set, and then the training set needs to be annotated to obtain an annotated map. In network training, it is necessary to input the infrared image and the annotated image into the network, and train a high-precision target segmentation network based on the infrared target image, thereby improving the segmentation accuracy of infrared weak targets. Features of the embodiment of the present invention: 1) Infrared weak target detection. 2) Strong robustness, adaptable to any angle and height.

Claims

1. The infrared small target recognition system based on unmanned aerial platform is characterized by: include: An unmanned aerial platform equipped with an infrared camera obtains an initial image corresponding to a small target and stores it in a first folder, annotates the initial image in the first folder to obtain an annotated image and a trained initial image, stores the annotated image in a second folder, stores the trained initial image in the first folder, and the initial image, the trained initial image and the annotated image constitute a data set; Use the LCAGNet deep learning network architecture, set dynamic learning rate and loss function, and use the dataset as training data to obtain the initial segmentation network model; Initialize a maximum standard IOU to 0. In each round of iteration, calculate the current IOU based on the output of the initial segmentation network model and the annotated image, and compare the current IOU with the maximum standard IOU loss. If the current IOU is larger than the maximum standard IOU, save the weight file of the initial segmentation network model, set the maximum standard IOU value to the current IOU value, and obtain the structure file and weight file of the final segmentation network model.

2. The infrared small target recognition system based on unmanned aerial platform as claimed in claim 1, characterized in that: The first folder and the second folder are in the same main folder.

3. The infrared small target recognition system based on unmanned aerial platform as claimed in claim 1, characterized in that: The data annotation method is as follows: Use the naked eye to observe the specific location of the weak target in the initial image; Connect the weak and small targets along the outer contour to obtain a closed connected area and a non-connected closed area; All pixels in the connected closed area are assigned a pixel value of 0, and all pixels in the unconnected closed area are assigned a pixel value of 255; Save the initial image after assigning pixels as a binary image, making sure the image channel is 1; Use the pytorch framework for network training, choose to use the AdamW optimizer, initialize the learning rate to 1e-4, batchsize to 8, and perform preset rounds of training. After training, the initial image and annotated image are enhanced, scaled to a length and width of 256 pixels, and normalized, and stored in the first folder and the second folder respectively.

4. The infrared small target recognition system based on unmanned aerial platform as claimed in claim 1, characterized in that: The loss function formula is: Among them, y i,j ∈R H×W is a label map, whose values ​​are 0 and 1, p i,j ∈R H×W It is the network output value after being processed by the sigmoid activation function σ.

Images