An infrared target detection method and system

Abstract

The application discloses an infrared target detection method, comprising the following steps: S1, inputting an infrared image to be detected into a target detection network, wherein the target detection network comprises a U-Net structure, a global feature self-attention fusion module is integrated into a decoder part of the U-Net structure, and a third-order Taylor expansion module is integrated into at least one feature connection path of the U-Net structure; S2, based on features output by the global feature self-attention fusion module and edge features extracted by the third-order Taylor expansion module, generating a final target detection result through a segmentation head. The application effectively suppresses complex background noise, reduces false alarms, can more accurately model and enhance the edges and fine structures of targets, and improves the detection rate of small targets and the accuracy of segmentation boundaries. Experimental results show that the comprehensive evaluation index of the method of the application is better than that of existing mainstream methods on multiple public data sets.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to an infrared image processing method. Background Technology

[0002] Infrared target detection refers to the process of locating and classifying targets of interest in infrared images. Infrared images contain a large amount of noise and clutter in the background, resulting in low target contrast and signal-to-clutter ratio. Infrared target detection methods can be divided into model-driven and data-driven methods. Model-driven methods include those based on filtering such as TopHat, those based on the Human Visual System (HVS) such as Local Contrast Metric (LCM), and those based on low-rank matrix factorization and reconstruction such as Infrared Patch Image (IPI). With the development of deep learning, data-driven methods have made significant progress in infrared target detection. Asymmetric Context Modulation (ACM) networks introduce asymmetric feature fusion, which is an alternative to the traditional skip connections in U-Net. Densely Nested Attention Networks (DNANet) implement a nested architecture with progressive and adaptive interactions between multiple feature layers. Furthermore, UIUNet enhances the detection of local target contrast by integrating multiple U-Net structures and employing an interactive cross-attention mechanism for feature fusion.

[0003] While data-driven methods excel in infrared target detection, they often struggle to capture details and have weak global perception capabilities. Summary of the Invention

[0004] This invention provides an infrared target detection method and system to address the problems of weak detail capture and global perception capabilities in existing data-driven infrared target detection methods.

[0005] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0006] An infrared target detection method includes the following steps: S1. The infrared image to be detected is input into the target detection network. The target detection network includes a U-Net structure containing a decoder and an encoder. The decoder part of the U-Net structure integrates a Global Feature Self-Attention Fusion (GFSAF) module. The GFSAF module is used to perform global spatial and channel attention calculations on the deepest features output by the encoder and fuse them to generate a feature map with a global receptive field. The target detection network also includes a third-order Taylor expansion module. Each feature layer of the decoder in the U-Net structure is connected to the third-order Taylor expansion module. The third-order Taylor expansion module is used to perform residual learning on the feature map using the third-order Taylor expansion to extract the edge features of the target. S2. Based on the features output by the global feature self-attention fusion module and the edge features extracted by the third-order Taylor expansion module, the final target detection result is generated through the segmentation head.

[0007] As a further preferred embodiment of the above technical solution, the global feature self-attention fusion module is implemented in the following ways: The deepest features are spatially focused using a non-local attention module to obtain a reconstructed feature map and a global spatial module result. Then, the reconstructed feature map is focused on channel attention using a squeezing and excitation module to obtain a channel attention module result. Finally, the global spatial module result and the channel attention module result are fused using convolution to obtain the feature map with a global receptive field.

[0008] As a further preferred embodiment of the above technical solution, the spatial attention calculation of the deepest features using a non-local attention module specifically includes: First, a 1x1 convolution is applied to the deepest features. The number of channels in the middle is reduced from c to , , In ;in The encoder output is represented by Q, K, and V, which are the outputs of their respective 1x1 convolutions. R is the real number field, and the upper right subscript of R is the matrix dimension. c is the number of channels, and w and h represent the width and height of the features input to the non-local attention module, respectively. Secondly, the feature map is reshaped and calculated according to the following formula. Obtain the reconstructed feature map: ;in , , These are the results of reshaping Q, K, and V, respectively, that is, expanding the width / height dimensions of Q, K, and V into one dimension to obtain Q', K', and V'; softmax is the softmax layer. It is a convolutional layer using a 1x1 kernel to increase the number of channels; reshape means reshaping, that is, reorganizing a one-dimensional vector into a two-dimensional matrix format.

[0009] As a further preferred embodiment of the above technical solution, the implementation of the third-order Taylor expansion module includes: The network relationship is constructed based on the third-order Taylor finite difference equation, and the target output is represented as the sum of the input and multiple residual terms. Convolutional layers are used to transform derivatives of different orders in the residual term; At least one higher-order residual term is computed using gated convolution, the input of which includes features from the previous order and corresponding layer features from the encoder-decoder structure.

[0010] As a further preferred embodiment of the above technical solution, the output of the TOTE module... It is obtained through calculation in the following form: ; where u gate The term is implemented by gated convolution, and the input of the gated convolution fuses the intermediate feature state with the corresponding feature p(x) from the encoder; For the input values ​​of the TOTE module, , , Depend on Obtained by transformation using multiple convolutional layers; express and The residuals between express and The residuals between them.

[0011] As a further preferred embodiment of the above technical solution, the third-order Taylor expansion module is connected to each feature layer of the decoder.

[0012] As a further preferred embodiment of the above technical solution, the segmentation head is a convolution segmentation head.

[0013] Based on the same technical concept, the present invention also provides an infrared target detection system, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method as described in any one of claims 1 to 7.

[0014] Compared with existing technologies, the present invention has the following advantages: The detection method of this invention introduces a GFSAF module, which, through a parallel spatial and channel attention mechanism, fully aggregates the global contextual information of the image, effectively suppressing complex background noise and reducing false alarms. A TOTE module is also introduced, based on a high-precision Taylor expansion design, using residual learning and gated convolution to more accurately model and enhance the edges and fine structures of targets, improving the detection rate of small targets and the accuracy of segmentation boundaries. These two modules operate at different stages and feature layers of the network, respectively, and work together to achieve a significant performance improvement. Experimental results show that the method of this invention outperforms existing mainstream methods in terms of comprehensive evaluation metrics on multiple public datasets.

[0015] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall structure of the target detection network in Example 1.

[0017] Figure 2 This is a schematic diagram of the structure of the third-order Taylor expansion module in an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of the global feature self-attention fusion module in an embodiment of the present invention. Detailed Implementation

[0019] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0020] Example 1: An infrared target detection method includes the following steps: S1. Input the infrared image to be detected into the target detection network, such as... Figure 1 As shown, the object detection network uses a U-Net structure containing a decoder and an encoder as its basic framework. The decoder part of the U-Net structure integrates a global feature self-attention fusion module, which is used to perform global spatial and channel attention calculations on the deepest features output by the encoder and fuse them to generate a feature map with a global receptive field. Each feature layer of the decoder in the U-Net structure is connected with a corresponding number of third-order Taylor expansion modules, which are used to perform residual learning on the feature map using third-order Taylor expansion to extract the edge features of the target and obtain a rough target shape and fine edges. S2. Based on the features output by the global feature self-attention fusion module and the edge features extracted by the third-order Taylor expansion module, the coarse target shape is further refined by the convolution segmentation head according to the fine edge, and the final target detection result is generated to obtain the fine target shape.

[0021] The specific structure of the GFSAF module mentioned in S1 is as follows: Figure 3 As shown, it receives the deepest feature map output by the encoder as input, and through a dual-path parallel processing and fusion structure, it simultaneously captures global spatial dependence and channel importance, and finally outputs an enhanced feature with a global receptive field.

[0022] This module comprises three key steps: First, global spatial attention modeling of the main path: spatial attention is calculated on the feature map of the deepest layer of the decoder using a non-local attention module, whose size is significantly smaller than the input image; then, a 1x1 convolution is applied to... The number of channels in the middle is reduced from c to , , In c represents the number of channels, and w and h represent the width and height of the features input to the non-local attention module, respectively. The feature map is then reshaped and calculated according to the following formula. The results of the global spatial attention module are obtained: ; in , , These are the results of reshaping Q, K, and V, respectively. `softmax` refers to the `softmax` layer, while... It is a convolution using a 1x1 kernel to increase the number of channels.

[0023] Second, channel attention recalibration of branch paths: squeezing and excitation blocks are introduced to enhance channel perception. The input deep features are simultaneously fed into the squeezing and excitation modules to obtain the results of the channel attention module. The results are then subjected to an upsampling operation to match their spatial dimensions with the features processed by the main path, which facilitates subsequent fusion.

[0024] By concatenating the results of the global spatial and channel attention modules and fusing them using convolution, GFSAF can capture global feature representations and obtain feature maps with global receptive fields, which is crucial for effective object detection.

[0025] The structure of the TOTE module mentioned in S1 is as follows: Figure 2 As shown, the TOTE module is inspired by the idea of ​​using third-order Taylor expansions to achieve high-precision numerical solutions to ordinary differential equations (ODEs), and transforms this into a residual network module suitable for deep learning, focusing on extracting edge features. In fact, the connection between ODEs and neural networks has been studied by researchers before. It is worth noting that most existing ODE-based networks are designed based on the Euler method, but in reality, numerical solutions to ODEs based on the Taylor method often provide higher accuracy. Based on this characteristic, this invention applies the Taylor formula to obtain numerical solutions to ODEs and accordingly designs a novel network block to extract useful edge features from infrared targets.

[0026] Specifically, we discretize ordinary differential equations using finite difference equations, where partial derivatives can be replaced by a set of approximate differences. Since the Taylor finite difference method has higher accuracy than the Euler method, we use it to design the network block. This invention starts with the third-order Taylor expansion to construct the network block, hence the name TOTE module.

[0027] From a mathematical perspective, the third-order Taylor finite difference equation can be expressed as: ; Then rewrite it in addition form: ; To simplify the training of deep neural networks, we employ the idea of ​​residual learning, which directly maps... Convert to residual form , where H(x) and F(x) represent the target output and the learned residual, respectively. We rewrite the above formula as: ; In this invention, multiple convolutional layers are used to achieve the transition from... arrive as well as arrive The transformation is performed, and gated convolution is used to obtain the result. ,therefore, ; in, express and The residuals between express and The residuals between them. In this way, edge blocks inspired by ODEs can extract edge features through residual learning. It is worth noting that gated convolution can be regarded as partially learnable convolution, in which a soft gating mechanism is used to better learn the edge information of the target while suppressing background information. Specifically, The input is The sum of the corresponding features (denoted as p(x)) from U-Net.

[0028] The above constitutes Figure 2 The mathematical basis of the module structure shown is that, through this design, the TOTE module introduces the precision advantages of high-order numerical methods into the feature learning process, accurately modeling and enhancing the edge and detail features of targets in infrared images in a differentiable and trainable manner.

[0029] To verify the target recognition performance of the method in this embodiment, infrared images provided in international standard datasets (NUDT-SIRST dataset and IRSTD-1K dataset) were processed using both the method of this embodiment and conventional methods (ACM, RDIAN, ALCNet, DNANet, UIU-Net, and SCTransNet). Performance was evaluated using four metrics: mIoU(%), F1(%), Pd(%), and Fa(1e-6). The results are shown in Table 1 below (in Table 1, the arrows next to the performance metrics indicate that higher values ​​are better, and lower values ​​are better). For details on the relevant evaluation metrics, please refer to the following explanation: TP (True Positive): The true value is the target, and the prediction is also the number of pixels of the target; TN (True Negative): The number of pixels that are both true and predicted to be the background. FP (False Positive): The number of pixels predicted as the target when the ground truth is background; FN (False Negative): The number of pixels predicted as background when the true value is the target; IoU = Intersection / Union = TP / (TP + FP + FN), mIoU (mean Intersection over Union) is the average IoU ratio. It is calculated for each category and then averaged to measure the accuracy of the model for each target segmentation region. The higher the better. Pd (Probability of Detection), the detection rate / recall rate, Pd = Recall = TP / (TP + FN), the proportion of true positives correctly detected by the model. A higher Pd indicates fewer false negatives. Fa (False Alarm Rate), calculated as Fa = FP / (TN + FP), represents the proportion of negative cases that are incorrectly predicted as positive cases. A lower rate is better, indicating fewer false alarms. The unit is 1e-6, or one in a million. Precision: The proportion of pixels predicted as targets that are actually targets, reflecting the false alarm rate. Precision = TP / (TP + FP). A higher value indicates that the algorithm misclassifies background as targets less often. F1 = 2 × (Precision × Recall) / (Precision + Recall), which balances precision and recall to comprehensively evaluate the quality of target detection; the higher the better.

[0030] Table 1: Comparison of Infrared Target Recognition Performance between Example 1 and Conventional Methods

[0031] The infrared target detection system of this embodiment includes at least one processor and a memory. The memory stores executable instructions, which, when executed by the processor, control the processor to perform the steps described in this embodiment. This system can be integrated into a drone, monitoring equipment, vehicle-mounted computing platform, or cloud server.

[0032] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. The above are merely preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. For those skilled in the art, improvements and modifications obtained without departing from the technical concept of the present invention should also be considered within the protection scope of the present invention.

Claims

1. An infrared target detection method, characterized in that, Includes the following steps: S1. The infrared image to be detected is input into the target detection network. The target detection network includes a U-Net structure containing a decoder and an encoder. The decoder part of the U-Net structure integrates a global feature self-attention fusion module. The global feature self-attention fusion module is used to perform global spatial and channel attention calculation on the deepest features output by the encoder and fuse them to generate a feature map with a global receptive field. The target detection network also includes a third-order Taylor expansion module. Each feature layer of the decoder in the U-Net structure is connected to the third-order Taylor expansion module. The third-order Taylor expansion module is used to perform residual learning of the third-order Taylor expansion on the feature map to extract the edge features of the target. S2. Based on the features output by the global feature self-attention fusion module and the edge features extracted by the third-order Taylor expansion module, the final target detection result is generated through the segmentation head.

2. The infrared target detection method according to claim 1, characterized in that, The implementation methods of the global feature self-attention fusion module include: The deepest features are spatially focused using a non-local attention module to obtain a reconstructed feature map and a global spatial module result. Then, the reconstructed feature map is focused on channel attention using a squeezing and excitation module to obtain a channel attention module result. Finally, the global spatial module result and the channel attention module result are fused using convolution to obtain the feature map with a global receptive field.

3. The infrared target detection method according to claim 2, characterized in that, The spatial attention computation for the deepest features using a nonlocal attention module specifically includes: First, a 1x1 convolution is applied to the deepest features. The number of channels in the middle is reduced from c to , , In ;in, The encoder output is represented by Q, K, and V, which are the outputs of their respective 1x1 convolutions. R is the real number field, and the upper right subscript of R is the matrix dimension. c is the number of channels, and w and h represent the width and height of the features input to the non-local attention module, respectively. Secondly, the feature map is reshaped and calculated according to the following formula. Obtain the reconstructed feature map: ;in , , These are the results of reshaping Q, K, and V, respectively; softmax is the softmax layer; It is a convolutional layer using a 1x1 kernel to increase the number of channels; reshape means reshaping, that is, reorganizing a one-dimensional vector into a two-dimensional matrix format.

4. The infrared target detection method according to any one of claims 1-3, characterized in that, The implementation methods of the third-order Taylor expansion module include: The network relationship is constructed based on the third-order Taylor finite difference equation, and the target output is represented as the sum of the input and multiple residual terms. Convolutional layers are used to transform derivatives of different orders in the residual term; At least one higher-order residual term is computed using gated convolution, the input of which includes features from the previous order and corresponding layer features from the encoder-decoder structure.

5. The infrared target detection method according to claim 4, characterized in that, The output of the TOTE module It is obtained through calculation in the following form: ; where u gate The term is implemented by gated convolution, and the input of the gated convolution fuses the intermediate feature state with the corresponding feature p(x) from the encoder; For the input values ​​of the TOTE module, , , Depend on Obtained by transformation using multiple convolutional layers; express and The residuals between express and The residuals between them.

6. The infrared target detection method according to any one of claims 1-3, characterized in that, The third-order Taylor expansion module is connected to each feature layer of the decoder.

7. The infrared target detection method according to any one of claims 1-3, characterized in that, The segmentation head is a convolution segmentation head.

8. An infrared target detection system, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method as described in any one of claims 1 to 7.

Images