Infrared small target detection method and system based on frequency domain attention and storage medium

Abstract

The invention discloses an infrared small target detection method and system based on frequency domain attention and a storage medium, and belongs to the technical field of infrared small target detection, and the method comprises the steps: obtaining to-be-detected infrared image data, and carrying out the preprocessing of the to-be-detected infrared image data, and obtaining a standard infrared image; inputting the standard infrared image into a pre-trained infrared small target detection model to obtain a detected infrared small target; the acquisition method of the pre-trained infrared small target detection model comprises the following steps: acquiring a historical infrared image sample set; preprocessing each infrared image data in the historical infrared sample set to obtain a standard historical sample set; the standard historical sample set is input into a pre-constructed detection model, a trained infrared small target detection model is obtained, a decoding module of the detection model reconstructs a target image by fusing a frequency domain attention weighted feature map and a spatial domain feature map, the calculation complexity is reduced, and the detection accuracy is improved. And the infrared small target detection precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of infrared small target detection, and in particular relates to an infrared small target detection method, system and storage medium based on frequency domain attention. Background Art

[0002] Infrared imaging technology is widely used in target detection, monitoring, and environmental monitoring. By capturing infrared radiation emitted from the surface of an object, the infrared thermal imaging system can generate an image corresponding to the temperature distribution. This imaging method does not rely on external light sources, which makes it perform well at night or in bad weather conditions. But at the same time, because of this special imaging method, infrared small target detection has become a special task in the field of computer vision, with the following difficulties: 1. Due to the long imaging distance, the infrared image occupies a small pixel area in the image, resulting in unclear target features; 2. Severe background interference. Most areas in the infrared image are background pixels. The target is usually dim, with a low signal-to-noise ratio, and the target accounts for a small proportion in the image, resulting in an uneven distribution between the target and the background, affecting the detection effect; 3. Limited shape features. Due to the small size, uneven surface material and temperature distribution of the infrared small target, its shape features in the image are not obvious, which brings challenges to feature extraction.

[0003] In recent years, with the rapid development of deep learning, neural network models have been used for infrared small target detection and have shown good results. However, these models still face problems such as too large parameters, high computational cost, inaccurate detection targets, and high false detection rate. Summary of the invention

[0004] The present invention provides a method, system and storage medium for infrared small target detection based on frequency domain attention. The small target is treated as a high-frequency component in an infrared image, and the frequency domain attention correlation and spatial domain feature map are used in the decoding module of the infrared small target detection model to cross-channel fusion to reconstruct the target image, thereby reducing the computational complexity, simplifying the model parameters, and achieving efficient and accurate infrared small target detection.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides an infrared small target detection method based on frequency domain attention, comprising:

[0007] Acquire infrared image data to be tested;

[0008] Preprocessing the infrared image data to be measured to obtain a standard infrared image;

[0009] Inputting the standard infrared image into a pre-trained infrared small target detection model to obtain a detected infrared small target;

[0010] The method for obtaining the pre-trained infrared small target detection model includes:

[0011] Acquire a historical infrared image sample set, wherein the historical infrared image sample set includes historical infrared image data and corresponding labels;

[0012] Preprocessing each infrared image data in the historical infrared image data set to obtain a standard historical sample set;

[0013] The standard historical sample set is input into a pre-built detection model to obtain a trained infrared small target detection model, wherein the decoding module of the detection model reconstructs the target image by fusing the frequency domain attention weighted feature map and the spatial domain feature map.

[0014] Optionally, the pre-built detection model includes an encoding module, an embedding module and a decoding module connected in sequence;

[0015] The encoding module includes a multi-level residual connection encoder, each level of the encoder includes a convolutional residual block, a spatial channel attention layer and a downsampling layer connected in sequence;

[0016] The embedding module includes a residual block, and the residual block includes multiple convolutional layers connected in sequence;

[0017] The decoding module includes a multi-stage residual connection decoder, and each stage of the decoder includes an upsampling layer, a frequency domain attention layer, and a feedforward layer connected in sequence.

[0018] Optionally, the preprocessing of each infrared image data in the historical infrared image data set includes: adjusting the number of channels of each infrared image data to be consistent to obtain a corresponding standard infrared image.

[0019] Optionally, inputting the standard historical sample set into a pre-built detection model to obtain a trained infrared small target detection model includes:

[0020] The extracted feature map is obtained according to the input standard historical image through the encoding module;

[0021] The extracted feature map is subjected to deep feature extraction and fusion through the embedding module to obtain a fused feature map;

[0022] The reconstructed image is obtained according to the fused feature map through the decoding module;

[0023] Cross-channel fusion of the reconstructed image and the extracted feature map to obtain a target image;

[0024] The parameters of the detection model are adjusted according to the loss between the target image and the corresponding label to obtain a trained infrared small target detection model.

[0025] Optionally, the extracting the feature map by the encoding module according to the input standard historical image includes:

[0026] Performing a convolution operation on the standard historical image through the convolution residual blocks of encoders at each level to obtain a convolved feature map;

[0027] The spatial channel attention layer of the corresponding level encoder performs spatial channel attention weighted fusion on the convolution feature map to obtain an attention weighted feature map;

[0028] The resolution of the attention-weighted feature map is reduced through the downsampling layer of the corresponding level encoder to obtain the extracted feature map.

[0029] Optionally, obtaining the reconstructed image through the decoding module according to the fused feature map includes:

[0030] The fused feature map is deconvolved through the upsampling layers of each level of decoder to obtain the feature map with restored resolution. ;

[0031] The attention weight in the frequency domain is calculated for the feature map restored by the frequency domain attention layer of the corresponding decoder to obtain the feature map after the frequency domain attention weighting. The calculation formula is as follows:

[0032] ,

[0033] ,

[0034] in, is the attention weight in the frequency domain, It is a feature map The query vector in the frequency domain, It is a feature map The key vector in the frequency domain, It is a feature map A numeric vector in the frequency domain, represents the fast Fourier transform, express The inverse transformation of express The conjugate transpose of and The element-wise multiplication is performed between them. is the frequency domain attention weighted feature, Represents layer normalization operation layer norm;

[0035] The reconstructed image is obtained by linearly transforming the frequency domain attention weighted feature map through the feedforward layer of the corresponding level decoder , the calculation formula is as follows:

[0036] ,

[0037] ,

[0038] ,

[0039] ,

[0040] in, Represented as frequency domain attention weighted feature The feature map output by the upsampling layer Features after skip connection, represents the convolution operation, for Activation function, Represents the features transferred into the frequency domain after convolution, is a set of learnable parameters in the frequency domain, Representing the features in the frequency domain With learnable parameters After multiplication, the features are converted back to the spatial domain.

[0041] Optionally, cross-channel fusing the reconstructed image and the extracted feature map to obtain a target image includes:

[0042] According to the feature map extracted by the i-th level encoder And the reconstructed image obtained by the i-th level decoder Generate pixel-level feature vectors and ,in, , , , X is or , , R represents the image dataset, C is the number of channels of the feature map, W is the width of the feature map, H is the height of the feature map, m is the height index of the feature map X, n is the width index of the feature map X, and K represents the level index of the encoder or decoder;

[0043] According to the pixel-level feature vector and Generate the i-th level channel weight , the calculation formula is as follows:

[0044] ,

[0045] in, , is a linear layer, , ;

[0046] According to the i-th level channel weight Get the extracted feature map Feature vector after channel attention weighting , ,in, represents a linear rectification function;

[0047] The feature vector after weighting the channel attention and reconstructed image Splice in the channel dimension to get the i-th level target image .

[0048] Optionally, adjusting the parameters of the detection model according to the loss between the target image and the corresponding label to obtain a trained infrared small target detection model includes:

[0049] The target images of different scales are scaled uniformly to obtain the outputs of the detection model at all levels. The calculation formula is as follows:

[0050] ,

[0051] in, represents upsampling, represents the i-th level target image, is the i-th level output of the detection model;

[0052] The final output of the detection model is obtained according to the outputs of each level of the detection model. The calculation formula is as follows:

[0053] ,

[0054] in, To detect the final output of the model, represents channel dimension fusion, the final output The gray value of each pixel in represents the probability distribution of the pixel belonging to the infrared small target. If the pixel gray value is greater than 0, it means that the pixel belongs to the target. If the pixel gray value is 0, it means that the pixel belongs to the background.

[0055] According to the true label and the output of each level and the final output of the detection model, the training target is obtained , the calculation formula is as follows:

[0056] ,

[0057] ,

[0058] ,

[0059] in, is the loss function, are network parameters, represents the true value of the sample, represents the output value of the jth level of the detection model, , It means soft cross and comparison;

[0060] According to the loss function The sum of the losses of target images at all levels is obtained, and the network parameters of the encoder and decoder are iteratively updated according to the sum of the losses. The calculation formula is as follows:

[0061] ,

[0062] in, are the updated network parameters, are the network parameters before updating, is the learning rate, is the gradient of the loss function.

[0063] In a second aspect, the present invention provides an infrared small target detection system based on frequency domain attention, comprising:

[0064] Test data acquisition module: used to acquire the infrared image data to be tested;

[0065] Data preprocessing module: used to preprocess the infrared image data to be measured to obtain a standard infrared image;

[0066] Detection result acquisition module: used for inputting the standard infrared image into a pre-trained infrared small target detection model to obtain the detected infrared small target;

[0067] The method for obtaining the pre-trained infrared small target detection model includes:

[0068] Acquire a historical infrared image sample set, wherein the historical infrared image sample set includes historical infrared image data and corresponding labels;

[0069] Preprocessing each infrared image data in the historical infrared image data set to obtain a standard historical sample set;

[0070] The standard historical sample set is input into a pre-built detection model to obtain a trained infrared small target detection model, wherein the decoding module of the detection model reconstructs the target image by fusing the frequency domain attention weighted feature map and the spatial domain feature map.

[0071] In a third aspect, the present invention provides a computer storage medium having a computer program stored thereon. When the computer program is executed by a processor, the infrared small target detection method based on frequency domain attention as described in any step of the first aspect is implemented.

[0072] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention converts the traditional spatial domain attention mechanism into frequency domain calculation through Fourier transform. For infrared small target images, small targets often reflect the high-frequency components in the image. The extracted features are reconstructed through the frequency domain attention decoder, which reduces the computational complexity and simplifies the model parameters. At the same time, the correlation of small target features in the frequency domain is adaptively obtained, and it is fused with the feature map of the spatial domain across channels to obtain the target image. The target images obtained based on the fusion at each level are respectively calculated with the label loss to realize the training of the entire network, which greatly improves the detection accuracy of the detection model and provides efficient and high-accuracy infrared small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 Shown is a flow chart of an infrared small target detection method based on frequency domain attention in an embodiment of the present invention;

[0074] Figure 2 Shown is a schematic diagram of the structure of a detection model in an embodiment of the present invention;

[0075] Figure 3 Shown is a flow chart of a method for training a detection model using historical infrared images in one embodiment of the present invention;

[0076] Figure 4 Shown is a frame of infrared small target image in the NUDT-SIRST data set in one embodiment of the present invention;

[0077] Figure 5 The infrared small target detection model of the present invention is used in one embodiment of the present invention. Figure 4 The detection result of the image shown in the figure;

[0078] Figure 6 In one embodiment of the present invention, Figure 4 The true object label of the image shown;

[0079] Figure 7 The figure shows a comparison of detection results of the infrared small target detection model of the present invention and other depth models for detecting the same frame of infrared small target image in one embodiment of the present invention. DETAILED DESCRIPTION

[0080] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0081] Example 1

[0082] like Figure 1 As shown, this embodiment provides an infrared small target detection method based on frequency domain attention, including:

[0083] Acquire infrared image data to be tested;

[0084] Preprocessing the infrared image data to be measured to obtain a standard infrared image;

[0085] Inputting the standard infrared image into a pre-trained infrared small target detection model to obtain a detected infrared small target;

[0086] The method for obtaining the pre-trained infrared small target detection model includes:

[0087] Acquire a historical infrared image sample set, wherein the historical infrared image sample set includes historical infrared image data and corresponding labels;

[0088] Preprocessing each infrared image data in the historical infrared image data set to obtain a standard historical sample set;

[0089] The standard historical sample set is input into a pre-built detection model to obtain a trained infrared small target detection model, wherein the decoding module of the detection model reconstructs the target image by fusing the frequency domain attention weighted feature map and the spatial domain feature map.

[0090] The target image is reconstructed by fusing the frequency domain attention weighted features with the spatial domain feature map. The detection model is trained based on the target image, which greatly reduces the computational complexity, simplifies the model parameters, and improves the detection accuracy of small infrared targets.

[0091] Example 2

[0092] Based on Example 1, this example also makes the following design.

[0093] The infrared small target detection method based on frequency domain attention is specifically divided into the following steps:

[0094] S1, build a detection model, such as Figure 2 As shown, the detection model includes an encoding module, an embedding module and a decoding module which are connected in sequence.

[0095] The encoding module includes a multi-level residual connected encoder, each level of the encoder includes a residual block, a spatial channel attention layer and a downsampling layer connected in sequence; the embedding module includes a multi-level residual connected convolutional layer; the decoding module includes a multi-level residual connected decoder, each level of the decoder includes an upsampling layer, a frequency domain attention layer and a feedforward neural network connected in sequence.

[0096] S2. Preprocess the historical infrared image data in the historical infrared image sample set, adjust the number of channels of each image data to be consistent, and obtain the corresponding standard infrared image, where X∈R H×W×C , H is the image height, W is the image width, C is the number of image channels, and R is the infrared image dataset.

[0097] S3, divide the preprocessed infrared small target sample set into a training set and a test set, Figure 3 The flowchart is for training the constructed detection model using the preprocessed infrared small target training set, specifically steps S31-S36, and then testing the trained infrared small target detection model using the test set, specifically step S37.

[0098] S31, Reference Figure 2 , the image features are extracted through the encoding module.

[0099] S311, use the two convolution layers in the residual block to extract features from the training set data of the input model, perform two convolution operations respectively, and after each convolution operation, perform BatchNormalization and linear rectification activation to enhance the nonlinear characteristics of the features. After the convolution operation, the extracted feature map F is obtained. ,in, is the convolution operation, is a linear rectification function, is batch normalization, and X is the input feature of each convolutional layer.

[0100] S312, using channel attention in spatial channel attention layer and spatial attention Perform attention calculation on the feature map F after feature extraction, that is, calculate the output of average pooling and maximum pooling under the two attention mechanisms respectively, then add or concatenate the outputs of average pooling and maximum pooling, and calculate the attention weight through the sigmoid activation function. The calculation formula is as follows:

[0101] ,

[0102] ,

[0103] In the formula, MLP is a multi-layer perceptron. represents the channel attention weight, represents the spatial attention weight, , , is the sigmoid activation function, is the average pooling operation, It is the maximum pooling operation.

[0104] S313, multiply the feature map by the calculated attention weight to obtain a weighted feature map, that is, for the input infrared small target image , after the channel attention module and the spatial attention module, we get ,in, is element-wise multiplication.

[0105] In this embodiment, the residual blocks in each encoder are residually connected to the spatial channel attention layer. Therefore, the encoders at each level extract the attention fusion weighted feature map , the calculation formula is as follows:

[0106] ,

[0107] in, .

[0108] S314: fusion of the attention weighted feature map Downsampling is performed to obtain a reduced feature map , the calculation formula is as follows:

[0109] ,

[0110] in, For the pooling operation, .

[0111] S32, Reference Figure 2 , construct multi-layer convolutional layers as embedding modules to embed between the encoding module and the decoding module to extract deep features.

[0112] S321, feature map Input the embedding module and get the feature map after passing through the embedding module once , the calculation formula is as follows:

[0113] ,

[0114] Among them, Mid consists of multiple CBR (Conv-BatchNorm-ReLU) layers connected in sequence. , X represents the input features of each CBR layer, represents dilated convolution, Represents batch normalization batch norm, The linear rectifier function is used as the activation function.

[0115] The feature map output by the i-th CBR layer And the feature map output by the j-th CBR layer symmetric Fusion is performed to obtain the extracted multi-layer features , ,in, , It is channel dimension fusion.

[0116] S322, multi-layer features Perform fusion and jump connection to obtain deep features , the calculation formula is as follows:

[0117] ,

[0118] Among them, Hin represents the fused features and input feature map Skip connections, .

[0119] S33, Reference Figure 2 , the deep features The input is decoded into a module, which first performs upsampling, and then passes the upsampled feature map through a frequency domain attention module to obtain a reconstructed image.

[0120] S331, for the input deep feature map Use deconvolution to upsample, expand the image resolution, and obtain the feature map of resolution restoration , the deep feature map of the input decoding module The calculation formula for deconvolution is:

[0121] ,

[0122] Among them, the output , W is the matrix weight, b is the bias, represents the convolution kernel size, represents the step length, Indicates the number of fill layers.

[0123] S332, upsampling the result Input frequency domain attention layer to generate query vectors in frequency domain , key vector , numeric vector According to the convolution theorem, the correlation or convolution of two signals in the spatial domain is equivalent to their element-wise multiplication in the frequency domain, that is, for the query vector obtained , key vector , numeric vector , use the fast Fourier transform FFT to estimate its correlation in the frequency domain, the calculation formula is as follows:

[0124] ,

[0125] in, is the attention weight in the frequency domain, represents the fast Fourier transform, represents its inverse transformation, express The conjugate transpose of and The multiplication is element-wise.

[0126] The numeric vector Multiply it with the frequency domain attention weight using element-wise multiplication to obtain the frequency domain attention weighted feature , the calculation formula is as follows:

[0127] ,

[0128] in, Represents layer normalization operation layer norm.

[0129] S333, frequency domain attention weighted features Input the feedforward neural network and get the reconstructed image as output , the calculation formula is as follows:

[0130] ,

[0131] ,

[0132] ,

[0133] ,

[0134] in, is a set of learnable parameters in the frequency domain, for Activation function, Weighted features for frequency domain attention Feature map before frequency domain attention weighting Feature map after skip connection, is the feature transferred into the frequency domain after convolution, is the feature in the frequency domain and learnable parameters in the frequency domain The spatial domain features converted back after multiplication.

[0135] S34, Reference Figure 2 , the reconstructed image output by each level of decoder and the feature map extracted by the corresponding level encoder are fused across channels to obtain the target image.

[0136] For the feature map extracted by the i-th level encoder , and the reconstructed image output by the i-th level decoder , generating multiple pixel-level feature vectors and ,in, , , X is or , m is the height index of feature map X, n is the width index of feature map X, and K is the level index of the encoder or decoder.

[0137] According to the pixel-level feature vector and Generate channel weights ,in, , , is a linear layer, .

[0138] The channel weights Acting on the feature map extracted by the i-th level encoder , get the feature vector after channel attention weighting , ,in, Represents the linear rectification function, and then and reconstructed image Splice in the channel dimension to get the target image of level i , .

[0139] S35, Reference Figure 2 , the losses of different scales are calculated by respectively comparing the target images of different scales at each level with the labels.

[0140] S351, according to the multi-scale target images at all levels , we can get the output of each level of the detection model with unified scale, and the calculation formula is as follows:

[0141] ,

[0142] in, represents upsampling, is the i-th level output of the detection model.

[0143] The outputs of each level of the detection model are spliced ​​to obtain the final output of the detection model , the calculation formula is as follows:

[0144] .

[0145] S352, according to the output of each level of the detection model and the final output Compare the loss of the real label Mask to get the training target , training objective The calculation formula is as follows:

[0146] ,

[0147] ,

[0148] ,

[0149] in, is the loss function, are network parameters, represents the true value of the sample, represents the output value of the jth level of the detection model, , Indicates soft cross and comparison.

[0150] S36, repeat steps S31-S35, based on Training the network, training and updating the network parameters of the encoder and decoder through back propagation and gradient descent algorithms, specifically including:

[0151] ,

[0152] in, are the updated network parameters, are the network parameters before updating, is the learning rate, is the gradient of the loss function.

[0153] S37, when the training rounds reach K, select the trained network parameters to detect the input infrared small target image to obtain the results, specifically including:

[0154] Select the segmentation threshold δ, and mark the pixel value greater than δ in the grayscale image as 1, indicating the target; otherwise, mark it as 0, indicating the background, and obtain a binary result image with only 0 and 1, and obtain the target detection result. Calculate the average intersection over union ratio mIoU, detection rate Pd, and false alarm rate Fa of the target detection results and labels of each sample in the test set.

[0155] ,

[0156] ,

[0157] ,

[0158] ,

[0159] Among them, K represents the number of samples in the test set, k represents the sample index in the test set, is the test result of the kth test set sample data With the true label TP is the number of true positive examples, FN is the number of false negative examples, FP is the number of false positive examples, and TN is the number of true negative examples.

[0160] S4. Input the preprocessed infrared image to be tested into the trained infrared small target detection model to obtain the corresponding small target detection result.

[0161] Specific application examples of this embodiment:

[0162] The infrared small target dataset used in this embodiment is the NUDT-SIRST (National University of Defense Technology-Single frame InfraRed Small Target) dataset, which provides 1,327 single-frame infrared small target images with various target types, rich postures, and multiple infrared backgrounds. The dataset is divided into two folders: Images, which is used to store real images; Masks, which is used to store target labels.

[0163] The detection model of the present invention is compared with various indicators of the existing infrared small target detection technology, wherein the infrared small target detection technology includes: Top-Hat, Relative Local Contrast Measure (RLCM), Asymmetric Contextual Modulation (ACM), Attention Local Contract Network (ALCNet), Receptive-field and Direction Induced Attention Network (RDIAN), Dense Nested Attention Network (DNANet), Shape Matters for Infrared Small Target Detection (ISNet), and U-Net in U-Net Network (UIU-Net). The comparison results are shown in Table 1.

[0164] Table 1: Comparison of various indicators of small target detection in the NUDT-SIRST dataset

[0165] method QUR Pd Fa Top-Hat 20.24 78.04 166.7 RLCM 15.14 16.35 162.99 ACM 64.855 96.72 28.587 ALCNet 61.14 97.24 29.09 RDIAN 82.12 98.83 14.85 DNANet 88.61 98.42 9.00 ISNet 81.23 97.77 6.34 UIUNet 91.52 98.31 8.34 The present invention 93.44 98.836 4.82

[0166] As can be seen from Table 1, the various indicators of the present invention are improved compared with the existing deep learning methods and traditional detection methods, and are superior to other methods, making it the detection algorithm with the best comprehensive performance.

[0167] In order to visualize the infrared small target detection results, such as Figure 4 , Figure 5 , Figure 6 As shown, they are respectively the real geographical location map, target detection result map, and target label Mask of a frame of infrared image in NUDT-SIRST. It can be seen that the detection result of the present invention has a high degree of overlap with the target label.

[0168] In order to visually compare the existing infrared small target detection methods, such as Figure 7 As shown, the detection result diagram of the present invention and the detection result diagram of other deep learning methods are specifically shown. It can be seen that the present invention can accurately detect the position and contour of small infrared targets.

[0169] Example 3

[0170] This embodiment provides an infrared small target detection system based on frequency domain attention, including:

[0171] Test data acquisition module: used to acquire the infrared image data to be tested;

[0172] Data preprocessing module: used to preprocess the infrared image data to be measured to obtain a standard infrared image;

[0173] Detection result acquisition module: used for inputting the standard infrared image into a pre-trained infrared small target detection model to obtain the detected infrared small target;

[0174] The method for obtaining the pre-trained infrared small target detection model includes:

[0175] Acquire a historical infrared image sample set, wherein the historical infrared image sample set includes historical infrared image data and corresponding labels;

[0176] Preprocessing each infrared image data in the historical infrared image data set to obtain a standard historical sample set;

[0177] The standard historical sample set is input into a pre-built detection model to obtain a trained infrared small target detection model, wherein the decoding module of the detection model reconstructs the target image by fusing the frequency domain attention weighted feature map and the spatial domain feature map.

[0178] Example 4

[0179] This embodiment provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the infrared small target detection method based on frequency domain attention as described in any step of Example 2 is implemented.

[0180] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0181] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0182] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0183] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0184] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A method for detecting small infrared targets based on frequency domain attention, characterized in that: include: Acquire infrared image data to be tested; Preprocessing the infrared image data to be measured to obtain a standard infrared image; Inputting the standard infrared image into a pre-trained infrared small target detection model to obtain a detected infrared small target; The method for obtaining the pre-trained infrared small target detection model includes: Acquire a historical infrared image sample set, wherein the historical infrared image sample set includes historical infrared image data and corresponding labels; Preprocessing each infrared image data in the historical infrared image data set to obtain a standard historical sample set; The standard historical sample set is input into a pre-built detection model to obtain a trained infrared small target detection model, wherein the decoding module of the detection model reconstructs the target image by fusing the frequency domain attention weighted feature map and the spatial domain feature map.

2. The infrared small target detection method based on frequency domain attention according to claim 1 is characterized in that: The pre-built detection model includes an encoding module, an embedding module and a decoding module connected in sequence; The encoding module includes a multi-level residual connection encoder, each level of the encoder includes a convolutional residual block, a spatial channel attention layer and a downsampling layer connected in sequence; The embedding module includes a residual block, and the residual block includes multiple convolutional layers connected in sequence; The decoding module includes a multi-stage residual connection decoder, and each stage of the decoder includes an upsampling layer, a frequency domain attention layer, and a feedforward layer connected in sequence.

3. The infrared small target detection method based on frequency domain attention according to claim 1 is characterized in that: The preprocessing of each infrared image data in the historical infrared image data set includes: adjusting the number of channels of each infrared image data to be consistent to obtain a corresponding standard infrared image.

4. The infrared small target detection method based on frequency domain attention according to claim 2 is characterized in that: The step of inputting the standard historical sample set into a pre-built detection model to obtain a trained infrared small target detection model comprises: The extracted feature map is obtained according to the input standard historical image through the encoding module; The extracted feature map is subjected to deep feature extraction and fusion through the embedding module to obtain a fused feature map; The reconstructed image is obtained according to the fused feature map through the decoding module; Cross-channel fusion of the reconstructed image and the extracted feature map to obtain a target image; The parameters of the detection model are adjusted according to the loss between the target image and the corresponding label to obtain a trained infrared small target detection model.

5. The infrared small target detection method based on frequency domain attention according to claim 4 is characterized in that: The step of extracting the feature map by the encoding module according to the input standard historical image comprises: Performing a convolution operation on the standard historical image through the convolution residual blocks of encoders at each level to obtain a convolved feature map; The spatial channel attention layer of the corresponding level encoder performs spatial channel attention weighted fusion on the convolution feature map to obtain an attention weighted feature map; The resolution of the attention-weighted feature map is reduced by the downsampling layer of the corresponding encoder to obtain the feature maps extracted by the encoders at each level.

6. The infrared small target detection method based on frequency domain attention according to claim 4 is characterized in that: The step of obtaining a reconstructed image according to the fused feature map by the decoding module includes: The fused feature map is deconvolved through the upsampling layers of each level of decoder to obtain the feature map with restored resolution. ; The attention weight in the frequency domain is calculated for the feature map restored by the frequency domain attention layer of the corresponding decoder to obtain the feature map after the frequency domain attention weighting. The calculation formula is as follows: ， ， in, is the attention weight in the frequency domain, It is a feature map The query vector in the frequency domain, It is a feature map The key vector in the frequency domain, It is a feature map A numeric vector in the frequency domain, represents the fast Fourier transform, express The inverse transformation of express The conjugate transpose of and The element-wise multiplication is performed between them. is the frequency domain attention weighted feature, Represents layer normalization operation layer norm; The frequency domain attention-weighted feature map is linearly transformed through the feedforward layer of the corresponding decoder to obtain the reconstructed image output by each level of decoder , the calculation formula is as follows: ， ， ， ， in, Represented as frequency domain attention weighted feature The feature map output by the upsampling layer Features after skip connection, represents the convolution operation, for Activation function, Represents the features transferred into the frequency domain after convolution, is a set of learnable parameters in the frequency domain, Representing the features in the frequency domain With learnable parameters After multiplication, the features are converted back to the spatial domain.

7. The infrared small target detection method based on frequency domain attention according to claim 6 is characterized in that: The cross-channel fusion of the reconstructed image and the extracted feature map to obtain a target image includes: According to the feature map extracted by the i-th level encoder And the reconstructed image obtained by the i-th level decoder Generate pixel-level feature vectors and ,in, , , , X is or , , R represents the image dataset, C is the number of channels of the feature map, W is the width of the feature map, H is the height of the feature map, m is the height index of the feature map X, n is the width index of the feature map X, and K represents the level index of the encoder or decoder; According to the pixel-level feature vector and Generate the i-th level channel weight , the calculation formula is as follows: ， in, , is a linear layer, , ; According to the i-th level channel weight Get the extracted feature map Feature vector after channel attention weighting , ,in, represents a linear rectification function; The feature vector after weighting the channel attention and reconstructed image Splice in the channel dimension to get the i-th level target image .

8. The infrared small target detection method based on frequency domain attention according to claim 7 is characterized in that: The parameters of the detection model are adjusted according to the loss between the target image and the corresponding label to obtain a trained infrared small target detection model, including: The target images of different scales are scaled uniformly to obtain the outputs of the detection model at all levels. The calculation formula is as follows: ， in, represents upsampling, represents the i-th level target image, is the i-th level output of the detection model; The final output of the detection model is obtained according to the outputs of each level of the detection model. The calculation formula is as follows: ， in, To detect the final output of the model, represents channel dimension fusion, the final output The gray value of each pixel in represents the probability distribution of the pixel belonging to the infrared small target. If the pixel gray value is greater than 0, it means that the pixel belongs to the target. If the pixel gray value is 0, it means that the pixel belongs to the background. According to the true label and the output of each level and the final output of the detection model, the training target is obtained , the calculation formula is as follows: ， ， ， in, is the loss function, are network parameters, represents the true value of the sample, represents the output value of the jth level of the detection model, , It means soft cross and comparison; According to the loss function The sum of the losses of target images at all levels is obtained, and the network parameters of the encoder and decoder are iteratively updated according to the sum of the losses. The calculation formula is as follows: ， in, are the updated network parameters, are the network parameters before updating, is the learning rate, is the gradient of the loss function.

9. An infrared small target detection system based on frequency domain attention, characterized in that: include: Test data acquisition module: used to acquire the infrared image data to be tested; Data preprocessing module: used to preprocess the infrared image data to be measured to obtain a standard infrared image; Detection result acquisition module: used for inputting the standard infrared image into a pre-trained infrared small target detection model to obtain the detected infrared small target; The method for obtaining the pre-trained infrared small target detection model includes: Acquire a historical infrared image sample set, wherein the historical infrared image sample set includes historical infrared image data and corresponding labels; Preprocessing each infrared image data in the historical infrared image data set to obtain a standard historical sample set; The standard historical sample set is input into a pre-built detection model to obtain a trained infrared small target detection model, wherein the decoding module of the detection model reconstructs the target image by fusing the frequency domain attention weighted feature map and the spatial domain feature map.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the infrared small target detection method based on frequency domain attention as described in any one of claims 1 to 8 is implemented.

Images