Progressive feature fusion method for infrared small target detection

Abstract

The application provides a progressive feature fusion method for infrared small target detection, and relates to the technical field of infrared target detection. First, the application collects data images and crops and scales them to 256*256 size, then performs mask annotation on the data images, and divides the collected data set. After image preprocessing of the training set and random initialization of the network structure, feature extraction is performed. Three feature maps of different sizes are generated, and are respectively transmitted to FSM and PFM to enhance the high-frequency feature response amplitude of the target. The enhanced results are transmitted to the segmentation head to obtain the final prediction segmentation map. Finally, the loss calculation is performed on the labeled data set and the prediction segmentation map, the model parameters are optimized according to the loss value through back propagation until the loss converges, and the model and weight file are saved. The application solves the false detection and missed detection problems caused by the fuzzy target contour and few texture detail features in the infrared small target detection, and improves the accuracy and detection speed of the infrared small target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of infrared target detection, and particularly relates to a progressive feature fusion method for infrared small target detection. BACKGROUND

[0002] Infrared small target detection is a main link in the field of computer vision target detection, and its purpose is to detect a target of interest in an infrared image and determine the position of the target. Compared with general detection targets, infrared small targets have characteristics such as darkness, weakness, small size and great shape change. Compared with targets in visible light images, targets in infrared images have more difficulties, such as lack of shape and texture features, and weakening or even loss of high-frequency amplitude of small targets after filtering and convolution calculation. Infrared small target detection has been a research hotspot in the field of infrared target detection technology. Thanks to the development of CNN methods, more data-driven methods are applied to infrared small target detection. Data-driven methods are suitable for more complex real scenes and are less affected by target size, shape and background changes. This method needs a large amount of data to show strong model fitting ability, and has achieved better detection performance than traditional methods.

[0003] At present, data-driven infrared target detection algorithms are divided into two categories: bounding box detection and pixel-level segmentation detection. Among them, the bounding box detection: from two stages to one stage, the idea is to match the prediction box with the labeled box to obtain multiple confidence scores, and then through the Non Maximum Suppression method, the prediction target with the maximum score value is represented by a rectangular box. This anchor-based method is divided into two tasks, namely prediction and detection, and cannot obtain accurate results when directly applied to infrared small targets. The pixel-level segmentation detection method regards target detection as a semantic segmentation task, and directly classifies and predicts pixels. This end-to-end single-task structure is more suitable for small target detection and can obtain excellent detection results.

[0004] Because visible light imaging will seriously affect the target detection effect when the light source is insufficient or encounters bad weather, infrared imaging can well solve this problem. Infrared detectors have all-weather working ability and important applications in airspace monitoring, sea monitoring, anti-missile, missile guidance, etc. Traditional infrared small target detection methods rely heavily on manual features. These methods are computationally friendly and do not require training and learning, but the design of manual features and the tuning of hyperparameters require expert knowledge and a lot of engineering effort. In deep learning algorithms, performance can be improved by building shallow networks, but the contradiction between high-level semantic features and high-resolution cannot be solved. In general, due to the large variation in the size of the unmanned aerial vehicle target and the extremely low pixel ratio in the infrared image, the number of negative samples in the image is too large, resulting in the loss of most of the available information in the algorithm running; most negative samples are easy to classify, and the algorithm cannot be optimized in the desired direction. Therefore, it is very difficult to directly use a network designed for normal objects to detect infrared small targets, and a special deep learning detection algorithm for infrared small targets needs to be designed.

[0005] In order to detect infrared small targets, many traditional methods have been proposed in recent decades. Traditional detection methods measure the non-coherence between the target and the background to achieve SIRST detection. Including filter-based methods, which can only suppress uniform and flat background clutter, and have a high false alarm rate for complex backgrounds, and unstable performance; HVS-based methods use the gray scale ratio between each pixel position and its adjacent region as an enhancement factor, which can effectively enhance the real target, but cannot effectively suppress the clutter in the background; low-rank representation-based methods can adapt to low-SCR infrared images, but still have a high false alarm rate on images with small targets and complex backgrounds.

[0006] The first segmentation-based SIRST detection method ACM designs a semantic segmentation network using an asymmetric context module, and then ALC introduces an expanded local contrast to improve their model, combines traditional methods with deep learning methods and uses a bottom-up local attention modulation module to embed subtle low-level details into higher layers, thereby achieving excellent detection performance. MDvsFA achieves a balance between the two metrics of missed detection and false alarm, applies a cGAN network to use MD and FA as two sub-tasks to build a model as a generator, and then uses a discriminator for image classification to distinguish the outputs of the two generators and the ground-truth of the three types of images. AGPCNet uses an attention mechanism to guide the pyramid context network to detect targets, first calculates the local correlation by dividing the feature map, then uses global context attention to calculate the correlation between semantics, and finally fuses decoding maps of different scales to improve detection effect. Although they all solve the problem of infrared small target detection from different angles, there are still many deficiencies. First, the high-frequency response amplitude of the infrared small target decreases or even disappears after multiple downsampling operations. On the other hand, due to the fuzzy outline of the infrared small target and the lack of texture detail features, it is easy to cause false detection and missed detection problems. SUMMARY

[0007] The problem to be solved by the present application is to solve the problems of the prior art, provide a progressive feature fusion method for infrared small target detection, to solve the problem of the decrease or even disappearance of the high-frequency response amplitude of the infrared small target in the CNN method and the false detection and missed detection problem caused by the fuzzy outline of the infrared small target and the lack of texture detail features in the infrared small target detection, and improve the accuracy and detection speed of the infrared small target detection.

[0008] In order to achieve the above-mentioned purpose of the present application, the technical scheme adopted by the present application is:

[0009] A progressive feature fusion method for infrared small target detection, comprising the following steps:

[0010] Step 1: Use an infrared imaging instrument to capture an infrared image containing a foreground target, and use a labeling tool to label the target drone in the image to obtain an infrared small target dataset.

[0011] The step specifically comprises:

[0012] Step 1.1: Uniformly crop and scale the obtained infrared small target dataset image to a picture with a size of 256x256.

[0013] Step 1.2: Use a labeling tool to label along the target outline to generate a pixel mask image and save it, obtain an infrared small target dataset, and divide the infrared small target dataset into a training set and a test set according to a set proportion.

[0014] Step 2. Data augmentation is performed on the infrared small target dataset.

[0015] The step specifically includes:

[0016] Step 2.1. Resize the input network picture, keeping the aspect ratio unchanged, and output image x.

[0017] Step 2.2. Randomly crop the output image of step 2.1, and the cropping satisfies where λ represents the cropping coefficient, x is the output image of step 2.1, and y1 represents the output image of step 2.2.

[0018] Step 2.3. Randomly translate the output image y1 of step 2.2, and the translation satisfies where y1 is the output image of step 2.2, and y2 represents the output image of step 2.3.

[0019] Step 2.4. Numerically normalize the output image y2 of step 2.3 and output image.

[0020] Step 2.5. Enlarge the resolution of the output image of step 2.4 to 256x256 size, and take 0 in the expanded area numerically.

[0021] Step 3. Design the network structure so that given an input image, classify each pixel through end-to-end convolution of this network structure to determine whether it is a drone target, then output a segmentation result of the same size, and finally use CE Loss and SoftIoU Loss loss functions to optimize in the desired direction. The above network structure is divided into two parts: a global feature extractor and a feature fusion network, which is further divided into two modules: Neck and Head, where Neck includes Pool Pyramid Fusion Model (PFM) and Feature Selection Model (FSM).

[0022] Step 3 specifically includes:

[0023] Step 3.1. Use Swin-Transformer as a global feature extractor to perform 3 times down-sampling on the output image of step 2.5, generating 64x64, 32x32, and 16x16 three different scale encoding maps.

[0024] Step 3.2. Take the 16x16 size encoding map as the input of PFM and output the decoding map, and the above PFM is a pyramid network composed of four adaptive pooling layers with different output sizes in parallel.

[0025] Step 3.2 specifically includes:

[0026] Step 3.2.1 Set the input feature map , where C, W, H represent the image dimensions, width, and height, respectively. The input feature map Input is passed into the pyramid pooling module in parallel for decoding, producing four different size encoding structures of 1x1, 2x2, 3x3, and 6x6.

[0027] Step 3.2.2 Reduce the feature dimension to using a 1x1 convolution, and then upsample the four different scale size feature maps through bilinear interpolation and concatenate them in the channel dimension with the input feature map.

[0028] Step 3.2.3 Perform convolution calculation on the output feature map through a 3x3 convolution to form a context pyramid with five feature maps of the same dimension but different scales.

[0029] Step 3.3: After upsampling the output decoding map of step 3.2, concatenate it with the 32x32 encoding Figure One in step 3.1 and pass it into the FSM module to output the decoding map F2, then upsample the output encoding map F2 and pass it into the FSM module again with the 64x64 encoding map in step 3.1 to output F1. The above FSM module is composed of a location selection LSM module and a channel selection CSM module, which are improved based on the CBAM module. Apply LSM and CSM to low-level sampling output and high-level sampling output respectively to obtain target position information and contour information. Then multiply the output of LSM and the input with the input and output of CSM respectively to aggregate the output feature maps of different sampling layers. At the same time, use a convolution kernel size of 5x5 for convolution operation on the output feature map.

[0030] The calculation formula of the output encoding map is as follows:

[0031]

[0032] where F is the final output of the FSM module, is the deep feature that includes high-level semantic information, is the shallow feature that contains rich image contour information and position information, and ⊕ represent vector element multiplication and addition respectively, and represent the CSM and LSM modules respectively.

[0033] Step 3.3 specifically includes:

[0034] Step 3.3.1 Firstly, the CSM module is used to average pool and max pool the input feature map X to generate different 3D tensors The global information of the feature map X is soft-coupled in its internal channels, and then the importance of each channel is evaluated using a 1x1 convolution to calculate the corresponding weight. The calculation formula is as follows:

[0035]

[0036]

[0037] wherein is the output of the CSM, is the function expression of the CSM module, X i represents different pooling operations of the same input image, when i = 1 is the feature vector obtained by average pooling, when i = 2 is the feature vector obtained by max pooling, w and h represent the width and height of the feature map respectively, is a point-wise convolution with two convolution kernels of 1x1 but different dimensions, δ represents a Sigmoid function, and σ represents a Rectified Linear Unit.

[0038] Step 3.3.2 The LSM module is used to calculate the maximum and average values of the input feature map X respectively and perform concatenation operation in the dimension direction, and then perform convolution operation on the feature map after splicing. A 7x7 convolution is used to further expand the receptive field of the convolution kernel, to capture regions with higher local response amplitude from the low-level network, and then calculate the accurate position of the unmanned aerial vehicle target in the entire feature map. The calculation formula is as follows:

[0039]

[0040]

[0041] wherein L is the final output of the LSM module, is the calculation method for the feature map dimension, when i = 1 the average value of the feature map X is output , when i = 2 the maximum value of the feature map X is output , represents the dimension concatenation operation, represents a point-wise convolution with a 7x7 convolution kernel.

[0042] Step 3.4 takes the output images of step 3.2 and step 3.3 as inputs of the head, and uses the segmentation head to fuse feature maps of different sizes to realize information superposition between high and low layers. The output images of step 3.2 and step 3.3 are respectively processed through the Ghost Model, and the same number of texture information encoding maps are generated through linear calculation. After processing through the Ghost Model, the high-layer encoding map is doubled in size and element addition operation is performed, and then the fused output image is calculated through convolution and then up-sampled and dimension concatenated. A multi-scale fusion strategy is used to gradually fuse feature maps of different sizes, and the fused feature map is finally threshold segmented to obtain the confidence map O.

[0043] Step 3.5 uses the CELoss and SoftIoULoss loss functions to calculate the total loss to realize the optimization of the algorithm in the expected direction, and the calculation formula is:

[0044]

[0045]

[0046]

[0047] Wherein, Pixels represents the calculation for each pixel, and alpha and beta represent loss weight coefficients, and smooth has no actual meaning, avoids the value of 0, and makes the calculation stable, and T and P represent the pixel values corresponding to the real target and output prediction, respectively.

[0048] Step 4 uses the designed progressive feature fusion network PFFNet to train the labeled infrared small target data set until the network converges, and saves the best training model and weight file. The test set is detected using the best training model, and the test results of the test set are obtained.

[0049] The specific steps include:

[0050] Step 4.1 uses the SGD optimizer to optimize PFFNet, sets the momentum, weight decay coefficient and initial learning rate, and uses the ploy attenuation strategy for training until the network converges, and saves the best training model and weight file.

[0051] Step 4.2 uses the model weight obtained in step 4.1 to test the results of the test set, that is, under the same hardware conditions, PFFNet and other infrared small target detection algorithms are compared and tested on the public infrared small target data set.

[0052] Compared with the prior art, the technical scheme adopted by the present application has the following technical effects:

[0053] Considering that there are rich target contour features in high-level semantic features and accurate target position information in low-level semantic features, the application provides a progressive feature fusion method for infrared small target detection, which has small model parameters, strong robustness, good detection effect and short detection time.

[0054] The CSM module in the FSM module enhances the target region response amplitude, solves the problem that the target of the unmanned aerial vehicle is easy to lose or even weaken the target region response value in the up-sampling process. Meanwhile, the LSM module quickly locates the local region with visual saliency, and overcomes the problem that the infrared small target is easy to introduce interference signals in the feature extraction process of the target due to the extremely low pixel number of the infrared small target in the infrared image. The FSM module fused by the above two modules can realize information interaction between different encoding maps of each dimension semantic information, effectively increase the utilization rate of the down-sampling and up-sampling output encoding maps and ensure the effectiveness of the multi-scale feature fusion.

[0055] The PFM module for infrared small target detection is used to process the encoding map of the highest down-sampling layer, different global adaptive pooling layer structures can compress the spatial dimension, and the mean value of the corresponding dimension is extracted, the feature representation of the small target in the deep network is enhanced, and the problem that the feature information of the unmanned aerial vehicle target such as propeller and arm is weakened or even lost due to the increase of the number of convolution down-sampling is solved.

[0056] A lightweight segmentation head more suitable for infrared small targets is designed, and the problem that the target feature response is weakened in the deepest layer of the convolution network is solved. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The whole structure flow chart of the progressive feature fusion method for infrared small target detection in the embodiment of the application is shown in the figure.

[0058] Figure 2 The PFM module structure diagram in the embodiment of the application is shown in the figure.

[0059] Figure 3 The FSM structure diagram in the embodiment of the application is shown in the figure.

[0060] Figure 4 The CSM structure diagram in the embodiment of the application is shown in the figure.

[0061] Figure 5 The LSM structure diagram in the embodiment of the application is shown in the figure.

[0062] Figure 6 The lightweight segmentation head structure diagram in the embodiment of the application is shown in the figure.

[0063] Figure 7Figure 1 is a schematic diagram of the PFFNet structure in the embodiments of the present application. DETAILED DESCRIPTION

[0064] The specific embodiments of the present application are described in further detail below in conjunction with the accompanying drawings and examples.

[0065] As Figure 1 shown, the method of the present embodiment is as follows:

[0066] Step 1: Use an infrared imaging instrument to capture an infrared image containing a foreground target, and use a labeling tool to perform data labeling on the target drone in the image to obtain an infrared small target data set.

[0067] The step specifically includes:

[0068] Step 1.1: Uniformly crop and scale the acquired infrared small target data set image into a picture with a size of 256x256.

[0069] Step 1.2: Use a labeling tool to label along the target contour to generate a pixel mask image and save it, and divide the infrared small target data set into a training set and a test set according to a ratio of 8:2.

[0070] Step 2: Perform data augmentation on the infrared small target data set.

[0071] The step specifically includes:

[0072] Step 2.1: Perform a Resize operation on the picture input into the network, keeping the aspect ratio unchanged.

[0073] Step 2.2: Randomly crop the image output by step 2.1, with the cropping satisfying where λ represents the cropping coefficient, x is the output image of step 2.1, and y1 represents the output image of step 2.2.

[0074] Step 2.3: Randomly translate the output image of step 2.2, with the translation satisfying where y1 is the output image of step 2.2, and y2 represents the output image of step 2.3.

[0075] Step 2.4: Numerically normalize the output image of step 2.3 and output the image.

[0076] Step 2.5: Enlarge the resolution of the output image of step 2.4 to a size of 256x256, and numerically take 0 for the expanded area.

[0077] ​Step 3: Design the network structure so that given an input image, each pixel is classified as a UAV target or not by the end-to-end convolution of this network structure, and then a segmentation result of the same size as the input image is output. Finally, the CE Loss and SoftIoU Loss loss functions are used to optimize the algorithm in the desired direction, accelerate the model convergence speed, and improve the prediction accuracy. The above network structure is divided into two parts: a global feature extractor and a feature fusion network. The global feature extractor extracts the basic features of the input infrared image by viewing the entire image, and obtaining these basic features can effectively reduce image redundancy information. The feature fusion network is divided into two modules: Neck and Head. The Head is used to realize the progressive fusion of feature maps of different sizes and generate a segmentation mask image. The Neck includes a Pool Pyramid Fusion Model (PFM) and a Feature Selection Model (FSM). The former is used to enhance the feature response amplitude in the deep network of the infrared UAV target, and the latter serves as a bridge for information exchange between high and low layers to improve the utilization rate of the down-sampled output encoding map.

[0078] Step 3 specifically includes:

[0079] Step 3.1: Use Swin-Transformer as a global feature extractor to perform 3 times down-sampling on the output image of step 2.5, generating 64x64, 32x32, and 16x16 encoding maps of different sizes.

[0080] Step 3.2: Use the 16x16 size encoding map as the input of PFM and output the decoding map, and enhance the high-frequency response amplitude of the deep target through PFM. The above PFM is a pyramid network composed of four adaptive pooling layers with different output sizes.

[0081] Step specifically includes:

[0082] Step 3.2.1: As shown in Figure 2 , let the input feature map be , where C, W, and H represent the image dimensions, width, and height, respectively. The input feature map Input is parallelly transmitted into the pyramid pooling module for decoding, producing 1x1, 2x2, 3x3, and 6x6 encoding structures of different sizes.

[0083] Step 3.2.2: Reduce the feature dimension to using a 1x1 convolution, and then up-sample the four feature maps of different sizes using bilinear interpolation and concatenate them in the channel dimension with the input feature map.

[0084] Step 3.2.3: Perform convolution calculation on the output feature map using a 3x3 convolution , prompting five feature maps of the same dimension but different scales to constitute a context pyramid.

[0085] Step 3.3. The output decoded map of step 3.2 is up-sampled and combined with the 32x32 encoded map in step 3.1 Figure One and input into the FSM module, and output the decoded map F 2， The output encoded map F2 is then up-sampled and combined with the 64x64 encoded map in step 3.1 again, and input into the FSM module and output F1. As Figure 3 shown, the FSM module is composed of a location selection LSM module and a channel selection CSM module, and the deep features containing high-level semantic information and the shallow features containing rich image contour information and location information can be fully integrated, thereby improving the utilization of the output encoded map. Based on the CBAM module, LSM and CSM modules are designed. LSM and CSM are applied to low-level sampling output and high-level sampling output respectively to obtain target location information and contour information. Then the output and input of LSM and the input and output of CSM are multiplied element by element to aggregate the output feature maps of different sampling layers. At the same time, a convolution operation with a convolution kernel size of 5x5 is performed on the output feature map. The FSM module utilizes the semantic information of each dimension to realize information interaction between different encoded maps, which can effectively increase the utilization of the down-sampling and up-sampling output encoded maps, and ensure the effectiveness of multi-scale feature fusion by positioning and enhancing the high-frequency response amplitude region. To preserve the feature representation of small targets in deep networks while not losing the spatial details of target positions, CSM is used to enhance information interaction between high and low layers, and LSM is used to obtain target location information.

[0086] The calculation formula of the output encoded map is as follows:

[0087]

[0088] where F is the final output of the FSM module, is a deep feature containing high-level semantic information, is a shallow feature containing rich image contour information and location information, and ⊕ represent vector element multiplication and addition, respectively, and represent the CSM and LSM modules.

[0089] Step 3.3 specifically includes:

[0090] Step 3.3.1 As Figure 4 shown, first use the CSM module to perform average pooling and max pooling operations on the input feature map X to generate different 3D tensors The global information of the feature map X is soft-coupled in its internal channels, and then the importance of each channel is evaluated using a 1x1 convolution to calculate the corresponding weight. The calculation formula is as follows:

[0091]

[0092]

[0093] wherein is the output of the CSM, is the function expression of the CSM module, X i represents different pooling operations of the same input image, when i = 1 is the feature vector obtained by average pooling, when i = 2 is the feature vector obtained by maximum pooling, w and h represent the width and height of the feature map respectively, is a point-wise convolution with two convolution kernels of 1x1 but different dimensions, δ represents a Sigmoid function, and σ represents a Rectified Linear Unit.

[0094] Step 3.3.2 As shown in Figure 5 , the maximum and average values of the input feature map X are calculated using the LSM module respectively, and a concatenation operation is performed in the dimension direction, and then a convolution operation is performed on the feature map after splicing. A 7x7 convolution is used to further expand the receptive field of the convolution kernel, to capture regions with higher local response amplitude from the low-level network, and then to calculate the accurate position of the unmanned aerial vehicle target in the entire feature map. The calculation formula is as follows:

[0095]

[0096]

[0097] wherein L is the final output of the LSM module, is the calculation method for the feature map dimension, when i = 1 the average value of the feature map X is output , when i = 2 the maximum value of the feature map X is output , represents a dimension concatenation operation, represents a point-wise convolution with a 7x7 convolution kernel.

[0098] Step 3.4 As shown in Figure 6As shown, the output images of step 3.2 and step 3.3 are taken as the input of the head, and the segmentation head is used to fuse feature maps of different sizes, realize information superposition between high and low layers, and enhance the target high-frequency response amplitude. The output images of step 3.2 and step 3.3 are respectively processed by the Ghost Model. Since the infrared small target occupies a very small proportion in the image, and a large number of feature maps with the same texture information will be generated after ordinary convolution calculation, the Ghost Model can generate the same number of texture information encoding maps through linear calculation, thereby reducing the convolution parameter quantity, improving the training and inference efficiency. After processing by the Ghost Model, the high-level encoding map is doubled in size and the elements are added. This process greatly simplifies the task of small target detection by sharing the same weight for all convolution blocks, and reduces the network inference time while reducing the P algorithm parameter quantity by using element summation operation. Then, the fused output image is upsampled and dimension concatenated after convolution calculation, and a multi-scale fusion strategy is adopted to gradually fuse feature maps of different sizes, and the final threshold segmentation is performed on the fused feature map to obtain the confidence map O.

[0099] Step 3.5 uses CE Loss and SoftIoU Loss loss functions to calculate the total loss, and realizes the optimization of the algorithm in the expected direction, and the calculation formula is:

[0100]

[0101]

[0102]

[0103] Where T and P represent the pixel values corresponding to the real target and output prediction respectively, Pixels represents the calculation for each pixel, and a, β represent the loss weight coefficient, smooth has no actual meaning, avoids the value of 0, and makes the calculation stable. According to the loss value at the beginning of training, a = 3 and β = 1 are set to balance the influence of individual loss on total loss to realize the optimization of the algorithm in the expected direction, and in order to ensure the stability of the calculation, smooth = 1 is set in this paper. Different weight balances may affect the performance index.

[0104] Step 4 uses the progressive feature fusion network PFFNet shown in Figure 7 to train the labeled infrared small target dataset until the network converges, and the best training model and weight file are saved. The best training model is used to detect the test set, and the test results of the test set are obtained.

[0105] The specific steps include:

[0106] Step 4.1. The PFFNet is optimized using the SGD optimizer, wherein the momentum and the weight decay coefficient are set to 0.9 and 0.0001 respectively, the initial learning rate is 0.05, and the network is trained until convergence using the ploy decay strategy, and the best trained model and weight file are saved.

[0107] Step 4.2. The model weight obtained in step 4.1 is used to test the test set.

[0108] Under the same hardware conditions, the PFFNet and other infrared small target detection algorithms are compared and tested on the public infrared small target data set, and the test results are shown in Table 1:

[0109] Table 1. Comparison of detection performance of the method of the present application and other infrared small target detection algorithms

[0110]

[0111] As can be seen from Table 1, the IoU of PFFNet is 1.8 percentage points higher than that of ALC, the running speed is improved by 0.047s, and although the detection speed is 0.006s slower than the fastest ACM, the detection performance is obviously improved. Table 1 fully demonstrates the effectiveness of the method of the present application, which can realize real-time infrared small target detection.

Claims

1. A progressive feature fusion method for infrared small target detection, characterized in that, The method comprises the following steps: Step 1, using an infrared imaging instrument to capture an infrared image containing a foreground target, and using a labeling tool to label the target drone in the image to obtain an infrared small target data set; Step 2, data augmentation is performed on the infrared small target data set; Step 3, a network structure is designed, so that given an input image, each pixel is classified by an end-to-end convolution of the network structure to determine whether it is a drone target, and then a segmentation result of the same size is output, and finally a CE Loss and a SoftIoU Loss loss function are used to optimize in the expected direction, the above network structure is divided into two parts: a global feature extractor and a feature fusion network, wherein the feature fusion network is divided into two modules: a Neck and a Head, wherein the Neck includes a Pool Pyramid Fusion Model (PFM) and a Feature Selection Model (FSM); The FSM module is composed of a position selection LSM module and a channel selection CSM module, the deep features containing high-level semantic information and the shallow features containing image contour information and position information can be fully fused, thereby improving the utilization rate of the output decoding map, the LSM and the CSM are improved based on the CBAM module, and the LSM and the CSM are applied to low-level sampling output and high-level sampling output to obtain target position information and contour information; Step 4, using the designed progressive feature fusion network PFFNet to train the labeled infrared small target data set until the network converges, and saving the best training model and weight file, using the best training model to detect the test set, and obtaining the test result of the test set; Step 4.1, using the SGD optimizer to optimize the PFFNet, setting the momentum, weight decay coefficient and initial learning rate, and using the ploy attenuation strategy for training until the network converges, and saving the best training model and weight file; Step 4.2, using the model weight obtained in step 4.1 to test the test set, that is, under the same hardware conditions, the PFFNet and other infrared small target detection algorithms are compared and tested on the public infrared small target data set.

2. The progressive feature fusion method for infrared small target detection according to claim 1, characterized in that, The step 1 specifically comprises the following steps: Step 1.1, uniformly cropping and scaling the obtained infrared small target data set image into a picture with a size of 256x256; Step 1.2, using a labeling tool to label along the target contour to generate a pixel mask image and save it, and obtain the infrared small target data set, and divide the infrared small target data set into a training set and a test set according to a set proportion.

3. The progressive feature fusion method for infrared small target detection according to claim 1, characterized in that, The step 2 specifically comprises the following steps: Step 2.1, performing a Resize operation on the input network picture, keeping the aspect ratio unchanged, and outputting an image x; Step 2.

2. Randomly crop the output image of Step 2.1 to meet where λ represents a cropping coefficient, x is the output image of Step 2.1, and y1 represents the output image of Step 2.

2. Step 2.

3. Randomly shift the output image y1 of step 2.2 by a shift that satisfies where y1 is the output image of step 2.2, denotes the output image of step 2.

3. Step 2.4, performing numerical normalization on the output image y2 of step 2.3 and outputting an image; Step 2.5, enlarging the resolution of the output image of step 2.4 to 256x256, and taking 0 in the expansion area in value.

4. The progressive feature fusion method for infrared small target detection according to claim 1, characterized in that, The step 3 specifically comprises the following steps: Step 3.1, using Swin-Transformer as a global feature extractor, the output image of step 2.5 is down-sampled 3 times to generate 64x64, 32x32, 16x16 three different scale encoding maps; Step 3.2, taking the 16x16 size encoding map as the input of PFM and outputting the decoding map, the above PFM is composed of four adaptive pooling layers with different output sizes in parallel to form a pyramid network; Step 3.3, the output decoding map of step 3.2 is up-sampled and input into the FSM module together with the 32x32 encoding map in step 3.1, and the output decoding map F2 is output, then the output decoding map F2 is up-sampled and input into the FSM module again with the 64x64 encoding map in step 3.1, and the output decoding map F1 is output, the output and input of LSM in FSM module are multiplied with the input and output of CSM respectively, which is used to aggregate the output feature maps of different sampling layers, and a convolution operation with a 5x5 convolution kernel is used on the output feature map; The calculation formula of the output decoding map is as follows: ； where F is the final output of the FSM module, are deep features that contain high-level semantic information, are shallow features that contain image contour information and location information, and ⊕ represent vector element multiplication and addition, respectively, and denote the CSM and LSM modules, respectively. Step 3.4, taking the output images of step 3.2 and step 3.3 as the input of the head, using the head to fuse feature maps of different sizes to realize information superposition between high and low layers, the output images of step 3.2 and step 3.3 are respectively processed by GhostModel, and the same number of texture information encoding maps are generated through linear calculation, the high layer encoding map is processed by GhostModel, and the size is doubled and the elements are added, then the fused output image is calculated by convolution, and then up-sampling and dimension concatenation are performed, a multi-scale fusion strategy is used to gradually fuse feature maps of different sizes, and the fused feature map is finally threshold segmented to obtain the confidence map O; Step 3.5, using CE Loss and SoftIoU Loss loss function to calculate the total loss, realizing the optimization of the algorithm in the expected direction.

5. The progressive feature fusion method for infrared small target detection according to claim 4, characterized in that, The step 3.2 specifically comprises the following steps: Step 3.2.1, set the input feature map wherein C, W, H represent the image dimensions, width, height respectively, the input feature map Input is transmitted into the pyramid pooling module in parallel for decoding to generate four encoding structures with different sizes of 1x1, 2x2, 3x3, 6x6; Step 3.2.2, reduce the feature dimension to Then, the four feature maps with different scale sizes are up-sampled by bilinear interpolation and concatenated with the input feature map in the channel dimension. Step 3.2.3, the output feature map is calculated by 3x3 convolution , prompting five feature maps of the same dimension but different scales to form a context pyramid.

6. The progressive feature fusion method for infrared small target detection according to claim 5, characterized in that, The step 3.3 specifically comprises the following steps: Step 3.3.1, first average pooling and max pooling operations are performed on the input feature map X using the CSM module to produce different 3D tensors The global information of the feature map X is soft-coupled in its internal channels, and then the importance of each channel is evaluated using a 1x1 convolution to calculate the corresponding weight, which is calculated as follows: ； ； where is the output of CSM, is the function expression of CSM module, X i denote different pooling operations of the same input image, when i = 1 is the feature vector obtained by average pooling, when i = 2 is the feature vector obtained by maximum pooling, w and h represent the width and height of the feature map respectively, is point-wise convolution with two convolution kernels of 1x1 but different dimensions, δ represents Sigmoid function, and σ represents Rectified Linear Unit; Step 3.3.2, using LSM module to calculate the maximum and mean value of input feature map X respectively and performing concatenation operation in dimension direction, then performing convolution operation on the spliced feature map, using 7x7 convolution to further expand the receptive field of convolution kernel, capturing the region with higher local response amplitude from low layer network, and then calculating the accurate position of unmanned aerial vehicle target in the whole feature map, the calculation formula is as follows: ； ； where L is the final output of the LSM module, is the way of calculation for feature map dimension, when i = 1 taking mean value output for feature map X , when i = 2 taking maximum value output for feature map X , denotes dimension concatenation operation, represents point-wise convolution with 7x7 convolution kernel.

Images