Infrared small target detection method and system based on self-adaptive partial self-attention

Abstract

The invention provides an infrared small target detection method and system based on self-adaptive partial self-attention, and the method comprises the steps: carrying out the processing of a single-frame infrared image through a double-residual attention block and a feature encoder, and obtaining a coding feature map; performing maximum pooling down-sampling processing on the coding feature maps of different levels to obtain coding features, and splicing the coding features in a channel dimension to obtain a multi-scale advanced coding feature map; obtaining a global context long-range feature map based on the multi-scale advanced coding feature map; obtaining a code-added feature map based on the global context long-range feature map; and inputting the decoded feature map into the prediction head to obtain a prediction result. According to the method, the salient small target in the infrared image is effectively extracted, the self-attention block of the self-adaptive part is utilized to integrate global context information, the long-distance dependence learning ability of the model is enhanced, and the perception ability of the position and the shape of the infrared small target is effectively enhanced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and target detection, and in particular to a method and system for detecting infrared small targets based on adaptive partial self-attention. Background Art

[0002] Infrared small target detection is an important research direction in the field of computer vision and target detection, and has a wide range of practical needs in military, security, rescue and other application scenarios. However, due to the weak signal of small infrared targets in images, they usually show low contrast, small size, and complex background, making it difficult for traditional detection methods to accurately identify and detect these targets. In the existing technology, infrared small target detection is usually performed through traditional methods based on filtering, background modeling and machine learning or deep learning methods such as convolutional neural networks (CNNs), but these methods still have significant limitations in complex environments.

[0003] Traditional infrared small target detection methods mainly rely on background suppression and filtering techniques, such as high-pass filtering, differential operators, and background prediction to enhance the signal of small targets. However, the detection effect of such methods is often poor in scenes with complex backgrounds, low contrast, and low signal-to-noise ratio. In addition, traditional methods lack the ability to express target features in depth and are difficult to adapt to the diverse target characteristics in infrared images, resulting in a high missed detection rate and serious false detection problems.

[0004] With the development of deep learning technology, infrared small target detection methods based on convolutional neural networks have received widespread attention. CNN performs well in image feature extraction. It can gradually obtain rich feature information from low to high layers by constructing a multi-layer convolution and pooling network structure. However, the CNN method is mainly based on local receptive fields and has shortcomings in long-distance dependency modeling and context information capture. Infrared small targets are usually small in size and weak in signal. It is difficult to effectively enhance the target signal by simply relying on convolution operations. In addition, due to the fixed feature extraction method of CNN, it is difficult to adapt to changes in the target, resulting in limited detection accuracy, especially when the background interference of the infrared image is strong, it is difficult to accurately identify the target. Summary of the invention

[0005] In view of the above situation, the main purpose of the present invention is to propose an infrared small target detection method and system based on adaptive partial self-attention to solve the above technical problems.

[0006] The present invention proposes an infrared small target detection method based on adaptive partial self-attention, the method comprising the following steps: Step 1: Process the single-frame infrared image through the dual residual attention block and the feature encoder in sequence to obtain a coded feature map; Step 2: Perform maximum pooling downsampling processing on the coding feature maps of different levels to obtain coding features, and concatenate the coding features in the channel dimension to obtain multi-scale high-level coding feature maps; Step 3: perform a channel segmentation operation on the multi-scale high-level coding feature map to obtain a segmented multi-scale high-level coding feature map, perform self-attention calculation on the segmented multi-scale high-level coding feature map to obtain a global attention map; The global attention map is concatenated with the multi-scale high-level coding feature map in the channel dimension to obtain a fused feature map, which is then processed by a convolutional layer and a multi-layer perceptron to obtain a global context long-range feature map. Step 4: Perform an upsampling operation based on transposed convolution on the global context long-range feature map and then merge it with the encoded feature map to obtain a decoded feature map of the intermediate layer, and input the decoded feature map of the intermediate layer into the feature decoder to obtain a decoded feature map; Step 5: Input the decoded feature map into the prediction head to obtain the potential prediction image, integrate the potential prediction image, and obtain the final detection image; The target is tracked based on the final detection image.

[0007] The present invention also proposes an infrared small target detection system based on adaptive partial self-attention, the system comprising: Feature encoding module, used to: The single-frame infrared image is processed by the dual residual attention block and the feature encoder in sequence to obtain the encoded feature map; Perform maximum pooling downsampling on the coded feature maps at different levels to obtain coded features, and concatenate the coded features in the channel dimension to obtain multi-scale high-level coded feature maps; Adaptive partial self-attention module for: Perform channel segmentation on the multi-scale high-level coding feature map to obtain a segmented multi-scale high-level coding feature map, perform self-attention calculation on the segmented multi-scale high-level coding feature map to obtain a global attention map; The global attention map is concatenated with the multi-scale high-level coding feature map in the channel dimension to obtain a fused feature map, which is then processed by a convolutional layer and a multi-layer perceptron to obtain a global context long-range feature map. Feature decoding module, used to: The global context long-range feature map is upsampled based on transposed convolution and then merged with the encoded feature map to obtain a decoded feature map of the middle layer. The decoded feature map of the middle layer is input into the feature decoder to obtain a decoded feature map. Image detection module for: Input the decoded feature map into the prediction head to obtain the potential prediction image, integrate the potential prediction images to obtain the final detection image; The target is tracked based on the final detection image.

[0008] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention enhances the modeling of long-range dependency relationships of small infrared targets through the architecture of adaptive partial self-attention module and multi-scale feature extraction, so that the system can accurately identify small targets in complex backgrounds and low-contrast environments, effectively reducing missed detections and false detections; 2. The present invention enables the system to achieve efficient real-time detection even in resource-constrained scenarios. The adaptive partial Transformer structure is used to perform attention calculations on only some channel features, which greatly reduces the computational cost; 3. The present invention enhances the stability of the system in complex environments such as low signal-to-noise ratio and dynamic background, and adapts to a variety of application scenarios to meet a variety of infrared small target detection needs.

[0009] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a flow chart of the infrared small target detection method based on adaptive partial self-attention proposed by the present invention; Figure 2 Schematic diagram of dual residual attention blocks of the infrared small target detection method based on adaptive partial self-attention proposed in the present invention; Figure 3 A schematic diagram of an adaptive partial self-attention module of the infrared small target detection method based on adaptive partial self-attention proposed by the present invention; Figure 4 This is a framework diagram of the infrared small target detection system based on adaptive partial self-attention proposed in the present invention. DETAILED DESCRIPTION

[0011] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0012] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to represent some ways of implementing the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0013] See also Figure 1 and Figure 2 The embodiment of the present invention proposes an infrared small target detection method based on adaptive partial self-attention, which includes the following steps: Step 1: Process the single-frame infrared image through the dual residual attention block and the feature encoder in sequence to obtain a coded feature map; In this step, the single-frame infrared image is processed by the dual residual attention block and the feature encoder in turn to obtain the encoded feature map. The relationship between the corresponding process is: ; in, represents a single frame infrared image, It means that it has been processed by a convolution layer with a convolution kernel size of 1×1. It means that it has been processed by the double residual attention block. Indicates The encoded feature map of the layer, It means that it has been processed by the maximum pooling layer with a window size of 2 and a step length of 2. Indicates The encoded feature map of the layer, Indicates the window size, represents the step length, represents the encoding feature map; The working principle of the dual residual attention block is expressed by the following relationship: ; in, Indicates The primary encoding feature map of the layer, It means that the convolution layer with a convolution kernel size of 3×3 is processed. It means that after being processed by Gaussian activation function, It means that it has been processed by batch normalization layer. Indicates The double residual attention block input features of the layer, Indicates Layer enhanced residual feature map, Indicates The primary encoding feature map of the layer is enhanced by channel attention to obtain the feature map. It means that after being processed by a multi-layer perception network, It means that after the maximum pooling process, It means that after average pooling, express The feature map obtained after spatial attention enhancement, It means that it has been processed by a convolution layer with a convolution kernel size of 7×7. represents the Sigmoid activation function, Represents a concatenation operation on the channel dimension.

[0014] Step 2: Perform maximum pooling downsampling processing on the coding feature maps of different levels to obtain coding features, and concatenate the coding features in the channel dimension to obtain multi-scale high-level coding feature maps; In step 2, the maximum pooling downsampling process is performed on the coded feature maps of different levels to obtain the coded features, and the coded features are spliced ​​in the channel dimension to obtain multi-scale high-level coded feature maps. The relationship between the corresponding process is: ; in, Indicates The encoded features of the layer, , , and All represent coding features. Represents a multi-scale high-level encoding feature map.

[0015] Step 3: perform a channel segmentation operation on the multi-scale high-level coding feature map to obtain a segmented multi-scale high-level coding feature map, perform self-attention calculation on the segmented multi-scale high-level coding feature map to obtain a global attention map; The global attention map is concatenated with the multi-scale high-level coding feature map in the channel dimension to obtain a fused feature map, which is then processed by a convolutional layer and a multi-layer perceptron to obtain a global context long-range feature map. See also Figure 3 In step 3, the multi-scale high-level coding feature map is subjected to channel segmentation to obtain the segmented multi-scale high-level coding feature map, and self-attention calculation is performed on the segmented multi-scale high-level coding feature map to obtain a global attention map. The process also includes the following sub-steps: Adaptive segmentation of multi-scale advanced coding feature maps, the corresponding process has the following relationship: ; in, Indicates that it is processed by SiLU activation function. It represents the multi-scale high-level coding feature map after being enhanced by convolution, batch normalization and SiLU activation function. represents the channel splitting factor, represents the features fed into multi-head self-attention after performing channel segmentation, A feature representing the number of remaining channels, , , and Both represent channel dimensions, represents the set of real numbers, Indicates the width of a single frame infrared image, Indicates the height of a single frame infrared image; The features sent to the multi-head self-attention after performing channel segmentation are sent to the projection convolution layer to generate the query weight matrix, key weight matrix and value weight matrix. The corresponding process has the following relationship: ; in, , and Represent the query weight matrix, key weight matrix and value weight matrix respectively, represents the channel dimension, Indicates the number of attention heads; Based on the query weight matrix, key weight matrix and value weight matrix, the query matrix, key matrix and value matrix are obtained, and then the query matrix, key matrix and value matrix are used to calculate the attention score. The relationship between the corresponding process is: ; in, Indicates The attention head score, It means that it has been processed by the Softmax function. Represents the dimension, Indicates participation The query matrix for attention head score calculation, Participate in The transpose of the key matrix for the score calculation of the attention heads, Indicates participation The value matrix of the attention head score calculation, Indicates the total score of attention investment; The self-attention calculation is performed on the segmented multi-scale high-level coding feature map to obtain the global attention map. The relationship between the corresponding process is: ; in, represents the position embedding matrix of the multi-scale high-level encoding feature map, represents the primary global attention map, represents the global attention map, represents the primary global attention map enhanced by residual connections; The global attention map and the multi-scale high-level coding feature map are spliced ​​in the channel dimension to obtain a fused feature map. The fused feature map is processed by the convolution layer and the multi-layer perceptron to obtain a global context long-range feature map. The corresponding process has the following relationship: ; in, represents the fused feature map, It means that it has been processed by the Dropout layer. Represents the global context long-range feature map.

[0016] Furthermore, in this step, the multi-scale gratitude encoding feature map is divided into two parts, one part participates in the self-attention calculation, and the other part is spliced ​​with the global attention map on the channel.

[0017] Step 4: Perform an upsampling operation based on transposed convolution on the global context long-range feature map and then merge it with the encoded feature map to obtain a decoded feature map of the intermediate layer, and input the decoded feature map of the intermediate layer into the feature decoder to obtain a decoded feature map; In step 4, the global context long-range feature map is subjected to an upsampling operation based on a transposed convolution and then merged with the encoded feature map to obtain a decoded feature map of the intermediate layer. The decoded feature map of the intermediate layer is input into the feature decoder to obtain a decoded feature map. The relationship between the corresponding process is: ; in, represents the encoded feature map, represents the upsampling operation based on transposed convolution, Indicates the step length is The convolution kernel is The transposed convolution of It means that after the transposed convolution processing with a convolution kernel size of 2 and a step length of 2, Indicates the number of decoder layers.

[0018] Step 5: Input the decoded feature map into the prediction head to obtain the potential prediction image, integrate the potential prediction image, and obtain the final detection image; Track the target based on the final detection image; In step 5, the decoded feature map is input into the prediction head to obtain the potential prediction image, and the potential prediction image is integrated to obtain the final detection image. The relationship between the corresponding process is: ; in, represents the potential prediction image, Indicates that it has been processed by the Sigmoid activation function. It means that the upsampling process is performed by bilinear difference. Represents the final detection image.

[0019] See also Figure 4 The embodiment of the present invention further provides an infrared small target detection system based on adaptive partial self-attention, the system comprising: Feature encoding module, used to: The single-frame infrared image is processed by the dual residual attention block and the feature encoder in sequence to obtain the encoded feature map; Perform maximum pooling downsampling on the coded feature maps at different levels to obtain coded features, and concatenate the coded features in the channel dimension to obtain multi-scale high-level coded feature maps; Adaptive partial self-attention module for: Perform channel segmentation on the multi-scale high-level coding feature map to obtain a segmented multi-scale high-level coding feature map, perform self-attention calculation on the segmented multi-scale high-level coding feature map to obtain a global attention map; The global attention map is concatenated with the multi-scale high-level coding feature map in the channel dimension to obtain a fused feature map, which is then processed by a convolutional layer and a multi-layer perceptron to obtain a global context long-range feature map. Feature decoding module, used to: The global context long-range feature map is upsampled based on transposed convolution and then merged with the encoded feature map to obtain a decoded feature map of the middle layer. The decoded feature map of the middle layer is input into the feature decoder to obtain a decoded feature map. Image detection module for: Input the decoded feature map into the prediction head to obtain the potential prediction image, integrate the potential prediction images to obtain the final detection image; The target is tracked based on the final detection image.

[0020] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0021] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0022] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting small infrared targets based on adaptive partial self-attention, the method is implemented based on a prediction model, the prediction model is composed of a feature encoding module, an adaptive partial self-attention module, a feature decoding module and an image detection module, characterized in that: The method comprises the following steps: Step 1: Process the single-frame infrared image through the dual residual attention block and the feature encoder in sequence to obtain a coded feature map; Step 2: Perform maximum pooling downsampling processing on the coding feature maps of different levels to obtain coding features, and concatenate the coding features in the channel dimension to obtain multi-scale high-level coding feature maps; Step 3: perform a channel segmentation operation on the multi-scale high-level coding feature map to obtain a segmented multi-scale high-level coding feature map, perform self-attention calculation on the segmented multi-scale high-level coding feature map to obtain a global attention map; The global attention map is concatenated with the multi-scale high-level coding feature map in the channel dimension to obtain a fused feature map, which is then processed by a convolutional layer and a multi-layer perceptron to obtain a global context long-range feature map. Step 4: Perform an upsampling operation based on transposed convolution on the global context long-range feature map and then merge it with the encoded feature map to obtain a decoded feature map of the intermediate layer, and input the decoded feature map of the intermediate layer into the feature decoder to obtain a decoded feature map; Step 5: Input the decoded feature map into the prediction head to obtain the potential prediction image, integrate the potential prediction image, and obtain the final detection image; The target is tracked based on the final detection image.

2. The infrared small target detection method based on adaptive partial self-attention according to claim 1 is characterized in that: In step 1, the single-frame infrared image is processed by the dual residual attention block and the feature encoder in sequence to obtain a coded feature map. The relationship between the corresponding processes is: ； in, represents a single frame infrared image, It means that it has been processed by a convolution layer with a convolution kernel size of 1×1. It means that it has been processed by the double residual attention block. Indicates The encoded feature map of the layer, It means that it has been processed by the maximum pooling layer with a window size of 2 and a step length of 2. Indicates The encoded feature map of the layer, Indicates the window size, represents the step length, represents the encoding feature map; The working principle of the dual residual attention block is expressed by the following relationship: ； in, Indicates The primary encoding feature map of the layer, It means that the convolution layer with a convolution kernel size of 3×3 is processed. It means that it has been processed by Gaussian activation function. It means that it has been processed by batch normalization layer. Indicates The double residual attention block input features of the layer, Indicates Layer enhanced residual feature map, Indicates The primary encoding feature map of the layer is enhanced by channel attention to obtain the feature map. It means that after being processed by a multi-layer perception network, It means that after the maximum pooling process, It means that after average pooling, express The feature map obtained after spatial attention enhancement, It means that it has been processed by a convolution layer with a convolution kernel size of 7×7. represents the Sigmoid activation function, Represents a concatenation operation on the channel dimension.

3. The infrared small target detection method based on adaptive partial self-attention according to claim 2 is characterized in that: In step 2, the coding feature maps of different levels are subjected to maximum pooling downsampling processing to obtain coding features, and the coding features are spliced ​​in the channel dimension to obtain multi-scale high-level coding feature maps. The relationship between the corresponding processes is: ； in, Indicates The encoded features of the layer, , , and All represent coding features. Represents a multi-scale high-level encoding feature map.

4. The infrared small target detection method based on adaptive partial self-attention according to claim 3 is characterized in that: In step 3, the multi-scale high-level coding feature map is subjected to a channel segmentation operation to obtain a segmented multi-scale high-level coding feature map, and a self-attention calculation is performed on the segmented multi-scale high-level coding feature map to obtain a global attention map. The process also includes the following sub-steps: Adaptive segmentation of multi-scale advanced coding feature maps, the corresponding process has the following relationship: ； in, Indicates that it is processed by SiLU activation function. It represents the multi-scale high-level coding feature map after being enhanced by convolution, batch normalization and SiLU activation function. represents the channel splitting factor, represents the features fed into multi-head self-attention after performing channel segmentation, A feature representing the number of remaining channels, , , and Both represent channel dimensions, represents the set of real numbers, Indicates the width of a single frame infrared image, Indicates the height of a single frame infrared image; The features sent to the multi-head self-attention after performing channel segmentation are sent to the projection convolution layer to generate the query weight matrix, key weight matrix and value weight matrix. The corresponding process has the following relationship: ； in, , and Represent the query weight matrix, key weight matrix and value weight matrix respectively, represents the channel dimension, Indicates the number of attention heads; Based on the query weight matrix, key weight matrix and value weight matrix, the query matrix, key matrix and value matrix are obtained, and then the query matrix, key matrix and value matrix are used to calculate the attention score. The relationship between the corresponding process is: ； in, Indicates The attention head score, It means that it has been processed by the Softmax function. Represents the dimension, Indicate participation The query matrix for attention head score calculation, Participate in The transpose of the key matrix for the score calculation of the attention heads, Indicate participation The value matrix of the attention head score calculation, Indicates the total score of attention investment; The self-attention calculation is performed on the segmented multi-scale high-level coding feature map to obtain the global attention map. The relationship between the corresponding process is: ； in, represents the position embedding matrix of the multi-scale high-level encoding feature map, represents the primary global attention map, represents the global attention map, Represents the primary global attention map enhanced by residual connections.

5. The infrared small target detection method based on adaptive partial self-attention according to claim 4 is characterized in that: In step 3, the global attention map and the multi-scale high-level coding feature map are spliced ​​in the channel dimension to obtain a fused feature map, and the fused feature map is processed by a convolutional layer and a multi-layer perceptron to obtain a global context long-range feature map. The relationship between the corresponding process is: ； in, represents the fused feature map, It means that it has been processed by the Dropout layer. Represents the global context long-range feature map.

6. The infrared small target detection method based on adaptive partial self-attention according to claim 5 is characterized in that: In step 4, the global context long-range feature map is subjected to an upsampling operation based on a transposed convolution and then merged with the encoded feature map to obtain a decoded feature map of the intermediate layer. The decoded feature map of the intermediate layer is input into the feature decoder to obtain a decoded feature map. The relationship between the corresponding process is: ； in, represents the encoded feature map, represents the upsampling operation based on transposed convolution, Indicates the step length is The convolution kernel is The transposed convolution of It means that after the transposed convolution processing with a convolution kernel size of 2 and a step length of 2, Indicates the number of decoder layers.

7. The infrared small target detection method based on adaptive partial self-attention according to claim 6 is characterized in that: In step 5, the decoded feature map is input into the prediction head to obtain a potential prediction image, and the potential prediction image is integrated to obtain the final detection image. The relationship between the corresponding process is: ； in, represents the potential prediction image, Indicates that it has been processed by the Sigmoid activation function. It means that the upsampling process is performed by bilinear difference. Represents the final detection image.

8. An infrared small target detection system based on adaptive partial self-attention, characterized in that: The system applies the infrared small target detection method based on adaptive partial self-attention as described in any one of claims 1 to 7 above, and the system comprises: Feature encoding module, used to: The single-frame infrared image is processed by the dual residual attention block and the feature encoder in sequence to obtain the encoded feature map; Perform maximum pooling downsampling on the coded feature maps at different levels to obtain coded features, and concatenate the coded features in the channel dimension to obtain multi-scale high-level coded feature maps; Adaptive partial self-attention module for: Perform channel segmentation on the multi-scale high-level coding feature map to obtain a segmented multi-scale high-level coding feature map, perform self-attention calculation on the segmented multi-scale high-level coding feature map to obtain a global attention map; The global attention map is concatenated with the multi-scale high-level coding feature map in the channel dimension to obtain a fused feature map, which is then processed by a convolutional layer and a multi-layer perceptron to obtain a global context long-range feature map. Feature decoding module, used to: The global context long-range feature map is upsampled based on transposed convolution and then merged with the encoded feature map to obtain a decoded feature map of the middle layer. The decoded feature map of the middle layer is input into the feature decoder to obtain a decoded feature map. Image detection module for: Input the decoded feature map into the prediction head to obtain the potential prediction image, integrate the potential prediction images to obtain the final detection image; The target is tracked based on the final detection image.

Images