A method for tracking small infrared targets in complex backgrounds

Abstract

To address the difficulties of extracting effective features from small infrared targets in complex backgrounds and their susceptibility to interference from surrounding objects, the present invention proposes a method for tracking small infrared targets in complex backgrounds. The network model of the present invention first passes the input reference area and the area to be tracked into the dual feature extraction module to obtain fused feature maps. The similarity calculation module then performs similarity calculation on the fused feature maps, and the output similarity map contains the classification and regression information of the target. Finally, the refinement module and the head network output the predicted position and bounding box of the target in the current frame image to achieve robust tracking of small infrared targets in complex backgrounds. The present invention can effectively and robustly track real targets in complex scenes, reduce the influence of interference from the target's surroundings, improve tracking performance, and provide accurate location information for subsequent target feature extraction and key event determination.

Description

Technical Field

[0001] The present invention belongs to the field of infrared image target tracking, that is, using an end-to-end deep network model to achieve accurate tracking of infrared dim small targets in different complex backgrounds, specifically a method for tracking infrared dim small targets in complex backgrounds. Background Art

[0002] Infrared small target tracking technology is primarily used in enemy target early warning and long-range guided weapons. Accurately tracking small infrared targets is a critical challenge. Key challenges include: 1) Small infrared targets are located very far from the infrared sensor, resulting in a small number of infrared target pixels occupying the image, typically between 2×2 and 9×9 pixels. These pixels lack edge contours and texture information, making it difficult to extract key features. 2) During the tracking process, sensor jitter can occur, causing the target's trajectory to break, with the target suddenly jumping from one image location to another, causing the tracker to lose the target. 3) As the target adjusts its posture or switches on and off, its grayscale value changes. If the target encounters a bright background, it may be submerged in the background, leading to tracking failure. 4) Interference objects may appear around the target, whose grayscale values ​​exhibit a Gaussian distribution, similar to the true target. This can cause the tracker to deviate from the target during the discrimination process, potentially leading to loss of the true target. Therefore, developing a method for tracking small infrared targets in complex backgrounds is an urgent and challenging task.

[0003] At present, existing infrared dim target tracking methods can be divided into two categories: one is the mathematical modeling method based on model drive (referred to as "mathematical modeling method"), and the other is the data-driven deep learning method (referred to as "deep learning method"). The mathematical modeling method uses online learning correlation filters to establish a target appearance model, and applies it to the target candidate area to obtain a similarity map for tracking. The position of the maximum response in the similarity map is the center position of the real target in the current frame. The mathematical modeling method will update the target model in real time when tracking the target, which can effectively reduce the impact of posture adjustment, brightness change, etc. of the infrared dim target on the tracker. The method also collects the background information in front of the target, so that its tracking performance is improved. However, there are also the following defects: 1) The method often uses a multi-scale pyramid for scale estimation, and different scale parameters need to be set in advance to adjust the size of the candidate area to be tracked. The process is cumbersome and computationally complex, resulting in a decrease in tracking speed. It is only applicable to changes in scale of equal proportions and is difficult to apply to actual tracking scenarios. 2) The cyclic operation method introduces a boundary effect that reduces the performance of the tracker. In the process of training the filter, multiple cyclically shifted images are used, which leads to the presence of real target information in the negative samples, seriously reducing the performance of the tracker and reducing the ability to distinguish similar interference objects around the real target. The deep learning method currently commonly used is the twin network architecture, which extracts the deep features of the reference area and the area to be tracked through a backbone network with shared parameters, and then performs a cross-correlation operation to obtain a similarity map, and finally outputs the center position and bounding box of the predicted target through the classification and regression head network. It has a simple structure, does not need to set a large number of hyperparameters, and can achieve an effective balance between accuracy and speed. Therefore, the present invention is improved on the basis of the twin network architecture to achieve accurate tracking of weak infrared targets in complex scenes. Summary of the Invention

[0004] Aiming at the problems that it is difficult to extract effective features of infrared dim small targets under complex background conditions and they are easily affected by surrounding interference objects, the present invention proposes a method for tracking infrared dim small targets under complex backgrounds. The method is suitable for tracking infrared dim small targets under complex backgrounds such as forests, plains, and ridges, and can achieve high accuracy and precision while meeting real-time requirements.

[0005] The present invention adopts the following technical solution: a method for tracking infrared small targets under complex backgrounds is used to robustly track infrared small targets under different background environments, and the steps are as follows:

[0006] Step 1: Input an infrared image sequence Z containing a small infrared target;

[0007] Step 2: Select the target area in the first frame of the infrared image sequence Z as the reference area T;

[0008] Step 3: Input the reference region T into the dual feature extraction module to obtain the fused feature map cat(T);

[0009] Step 4: The subsequent frames of the infrared image sequence Z are taken as the origin, and the target center position of the previous frame is used to obtain the tracking area X. i ,i∈[2,n], n represents the total number of frames in T sequence;

[0010] Step 5: Set the area to be tracked to X i Input into the dual feature extraction module to obtain the fusion feature map cat(X i );

[0011] Step 6: Fusion of two features Figure 1 Input them into the similarity calculation module to obtain the similarity graph R;

[0012] Step 7: The similarity map R is refined by the module to obtain a similarity map up(R) that is consistent with the size of the area to be tracked.

[0013] Step 8: The similarity graph up(R) outputs the target center point position and bounding box size through the head network to obtain the tracking result of the current frame i;

[0014] Step 9: Replace the target area tracked in the current frame i with the reference area T, and continue with steps 3 to 9 until the sequence ends.

[0015] This method requires the construction of a dual-feature extraction module (DEM), a similarity calculation module (SCM), and a refinement module (RM) to form an infrared dim target tracking network. During the testing phase, a region update module (RUM) is added to adapt to changes in the target and surrounding background. The DEM extracts features from the infrared dim target and part of its background, effectively extracting key features. The SCM measures the similarity between the feature maps of the reference region and the region to be tracked, generating a similarity map containing target classification and regression information. RM addresses the problem of the similarity map being small in size, which introduces a large amount of background when projected back to the region to be tracked, resulting in reduced tracking accuracy. The similarity map is amplified by a neural network and aligned with the region to be tracked, ensuring a one-to-one correspondence between pixels and improving tracking accuracy. During the testing phase, the RUM always uses the previous image frame as the reference region to update the feature map information. The combination of the above modules ultimately constitutes a method for tracking infrared dim targets in complex backgrounds. This method uses publicly available datasets on the Internet and adds operations such as cropping, rotation, blurring, and mirroring to improve network robustness. It also uses multi-loss joint training for network optimization. The input of this method is an infrared image sequence, and the output is the coordinates of the upper left and lower right corners of the predicted target position in each frame.

[0016] In this method, the dual feature extraction module includes a deep feature extractor and an oriented gradient histogram feature extractor. The deep feature extractor, after inputting an infrared image, extracts both shallow details and deep semantic features from the image. The oriented gradient histogram feature extractor evenly divides the image into multiple image blocks, calculates the direction and magnitude of the grayscale gradient of the pixels in each image block, and finally integrates each image block to form a grayscale gradient histogram feature for the entire image.

[0017] The similarity calculation module is a modified transformer network. Through its self-attention mechanism, it enhances the key information of the target and background in the feature maps of the reference area and the area to be tracked. It uses the cross-attention mechanism to find the area most similar to the enhanced reference area feature map in the enhanced feature map of the area to be tracked. Benefiting from the transformer network's ability to extract global context information, it adaptively pays attention to the dependency of similar parts in the two feature maps from a global perspective.

[0018] The refinement module splits the U-Net network, removing the downsampling network on the left and retaining the upsampling network on the right. Its input is the similarity map obtained by the similarity calculation module, and its output is a refined similarity map that matches the size of the area to be tracked. The purpose of this module is to: when using the dual feature extraction module, the convolution and pooling operations in the deep network downsample the image area, reducing the size of the feature map. This introduces a lot of background information when the receptive field of each pixel is mapped to the area to be tracked, reducing tracking performance. Adding the refinement module enriches the classification and regression information in the similarity map, making it easier to locate the real target.

[0019] The region update module continuously updates the reference region during the method's testing. In infrared target tracking, partial background regions are often included to improve tracking accuracy. However, in complex environments, where the target's background region is constantly changing, using only the first frame of an infrared sequence as the target region can degrade tracker performance during subsequent tracking. Therefore, adding the region update module during the method's testing improves tracking accuracy.

[0020] In the dual feature extraction module described above, the deep feature extractor uses a ResNet-18 network model, with the final average pooling and fully connected layers removed. The input is a 127×127×3 reference region T and a 255×255×3 region to be tracked X. After passing these two regions through a five-layer residual network, the resulting deep feature maps res(T) and res(X) are 15×15×512 and 31×31×512, respectively. The Histogram of Oriented Gradients (HOG) feature extractor uses the same input as the deep feature extractor, and the outputs are a 15×15×8 HOG feature map hog(T) and a 31×31×8 HOG feature map hog(X). Finally, res(T) and hog(T) are concat-ed to obtain cat(T). The same operation is performed on the region to be tracked to obtain cat(X).

[0021] The similarity calculation module described above improves the transformer network structure used in natural language processing, making it suitable for target tracking. The similarity calculation module consists of two parts: an encoder and a decoder. In the encoder stage, the nn.Embeding function in the PyTorch deep learning library is first used to spatially encode the feature map cat(T) to obtain the encoded map P(T). P(T) is then added to cat(T), and the view function is used to flatten each channel of the added feature map to obtain a 520×255 multi-dimensional feature vector f(T) as the input to the transformer encoder. After passing f(T) through the first encoder layer, the target feature information in f(T) is enhanced using a multi-head attention mechanism. The specific calculation process is as follows: MultiHead(Q,K,V)=ConCat(head1,...,head n )W O , head i =Attention(QW i Q ,KW i K ,VW i V ), where Q, K, and V are identical and equal to f(T), W i Q 、W i K 、W i V The weight matrix i∈[1,8] represents the weight matrix that focuses on different information. The attention mechanism focuses on information at eight locations, mapping the encoded content to eight spaces and enhancing the model's representational capabilities. After f(T) passes through the multi-head attention mechanism, the encoded feature enc(T) is obtained. Then, f(T) and enc(T) are directly added together, followed by a normalization layer and a feedforward neural network (FFN). The feedforward neural network consists of two linear layers and one normalization layer. These operations ultimately yield the first encoded feature enc'(T), which can be expressed as: enc'(T) = FFN(Norm(f(T) + enc(T))). The reference region features are then further enhanced through the second encoder layer, following the same process as the first encoder layer. Finally, cat(T) passes through the encoder of the similarity calculation module to obtain the encoded feature enc"(T). In the decoder stage, the fused feature map cat(X) of the area to be tracked is input into the decoder in the similarity calculation module (SCM). The decoder also has two decoder layers. After adding spatial position encoding and flattening operations to the feature map, it passes through the first decoder layer. The decoder layer contains an attention module consisting of a multi-head attention mechanism and a normalization layer, which is used to enhance the target information and key background information in the feature map of the area to be tracked, and output the first decoded feature dec'(X). In the second decoder layer, it contains an attention module and a feedforward neural network, taking dec'(X) and the encoded feature enc"(T) as input. When passing the multi-head attention mechanism into the attention module, the Q, K, and V variables are no longer exactly the same, but Q = dec'(X), K = V = enc"(T). Finally, the second decoded feature dec"(X) is output through the normalization layer and the feedforward neural network. Afterwards, the view function in the Pytorch deep learning library is used to scale the dec”(X) feature map into a similarity map R of size 31×31×520.

[0022] The refinement module uses a U-Net upsampling network with five layers. The first and second layers each contain a transposed convolution with a 3×3 kernel and a 2×2 stride, followed by a double convolution block. The double convolution block consists of two convolution blocks, each containing a 3×3 kernel, 1 padding, and a stride of 1 convolution layer, a normalization layer, and an activation function layer. The main difference between the other three layers and the first two is the size and stride of the transposed convolution kernels: the third layer has a 2×2 kernel and a stride of 2, the fourth layer has a 2×2 kernel and a stride of 1, and the fifth layer has a 1×1 kernel and a stride of 1. Finally, a 2D convolution with a 1×1 kernel is performed. After the similarity map R passes through the refinement module, the refined similarity map up(R) with a size of 1×255×255 is obtained.

[0023] The above-mentioned region update module is applied to the test process of the present invention. When calculating the similarity between the fused feature maps of the area to be tracked and the reference area, the reference area always uses the infrared image of the target and part of the background area of ​​the previous frame to obtain a new reference area T′. Finally, the new encoding feature enc”(T′) is obtained through the encoder part in the dual feature extraction module and the similarity calculation module.

[0024] The training set image sequence for this method of tracking small infrared targets in complex backgrounds is obtained through the following process: The LaTOT dataset is used as the base dataset for model training. Each image in the sequence is first cropped from the target's center point to a width 10 times the diagonal length of the bounding box, resulting in a new 511×511 image. Subsequently, the new image is translated, scaled, blurred, and mirrored to offset the target's original position. The training reference images are from the first to the last frame of the sequence, while the images to be tracked are randomly selected within 30 frames before and after the reference image. The training set used by this method consists of 104,726 images.

[0025] The image sequence of the test set of the above-mentioned infrared dim target tracking method under complex background is obtained by the following process: the DIRST dataset is selected as the test set. In some infrared sequences, the target is only 1×1 in size, and it is impossible to extract useful depth feature maps and HOG feature maps. Therefore, when the target is too small, a simulated infrared dim target in the range of 5×5-7×7 is added to the real target, and its grayscale value is Gaussian distributed. The label is labeled using Dark Label software.

[0026] In order to solve the problem of robust tracking of infrared dim small targets in different background environments, the present invention proposes an infrared dim small target tracking method in complex backgrounds. By designing an end-to-end deep learning network, the tracking problem is decomposed into target classification and regression tasks. The method can effectively and robustly track real targets in complex scenes, reduce the influence of interference objects around the target, improve tracking performance, and provide accurate location information for subsequent extraction of target features and judgment of key events. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a diagram of the network model structure of the present invention.

[0028] Figure 2 This is a structural diagram of the dual feature extraction module of the present invention.

[0029] Figure 3 This is a structural diagram of the similarity calculation module of the present invention.

[0030] Figure 4 This is a refined module structure diagram of the present invention.

[0031] Figure 5 Schematic diagram of an embodiment of the present invention, where (a) shows the reference area of ​​the previous frame input by the tracking network, (b) shows the area to be tracked in the current frame input by the tracking network, and (c) shows the annotation result of the bounding box output by the tracking network in the current source image. DETAILED DESCRIPTION

[0032] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0033] Reference Figure 1 In this embodiment, a method for tracking a small infrared target under a complex background includes the following steps:

[0034] Step 1: Input an infrared image sequence Z to be tracked, which contains n frames of images;

[0035] Step 2: Manually mark the target area to be tracked on the first frame of the infrared image sequence Z, and fill the surrounding background area with twice the diagonal length of the target area to obtain the reference area T∈R 127×127×3 ;

[0036] Step 2.1: Send the reference area T to the dual feature extraction module (DEM) for feature extraction, referring to Figure 2 The model framework in [1] includes two parts: deep feature extraction and oriented gradient histogram feature extraction;

[0037] Step 2.2: Deep feature extraction for DEM dual feature extraction: A ResNet-18 architecture is used, consisting of five network blocks. The reference region T passes through these five network blocks in sequence to obtain the reference region deep feature map res(T). The first network block consists of a convolutional layer with a 7×7 kernel, a stride of 2, and a padding of 1. The second network block consists of a max pooling layer with a 3×3 kernel, a stride of 1, and a padding of 1, and two residual blocks. The residual block structure is composed of two convolutional layers with a 3×3 kernel, a stride of 1, and a padding of 1, but the output is concatenated with the input of the residual block at the output position to form a residual connection. The third to fifth network blocks each consist of two residual blocks. The residual block structure is similar to that of the residual block in the second network block, but the stride of the first convolutional layer in each residual block is set to 2 to downsample the image. Finally, the last average pooling layer and linear layer are deleted to obtain the reference region depth feature map res(T) of size 15×15×512.

[0038] Step 2.3: Extraction of oriented gradient histogram features in DEM dual feature extraction: First, divide the reference area T into multiple cell units, each cell unit is 8×8 pixels in size; then calculate the gradient size and direction of each pixel in each cell unit, and use 8 orientation statistics in the cell unit to calculate the gradient information of the pixels in the cell unit; finally, synthesize a larger block with 2×2 cell units, and concatenate and normalize the gradient information of each cell unit in the block to form the gradient histogram feature in the block; finally, merge all blocks to obtain a 15×15×8 oriented gradient histogram feature map hog(T).

[0039] Step 2.4: Fusion of dual features in DEM dual feature extraction: Concat the res(T) output from the deep feature extractor and the hog(T) output from the oriented gradient histogram feature extractor in the channel dimension to obtain the fused feature map cat(T).

[0040] Step 3: Input the fused feature map cat(T) of the reference region into the encoder in the similarity calculation module (SCM), and refer to Figure 3The encoder structure in . First, use the nn.Embeding function in the pytorch deep learning library to perform 256-dimensional spatial position encoding on the pixels in each row of the feature map cat(T), and do the same for the pixels in each column. Then, perform a dimension-wise concat operation on the spatial position encoding of each row and the spatial position encoding of each column to obtain a 15×15×512 coded map P(T). Because the number of width and height channels of P(T) and cat(T) are consistent, the corresponding pixels are directly added so that each pixel information of cat(T) contains the spatial information of its location. Then, use the view function to flatten each channel of the added feature map to obtain a 520×255 multi-dimensional feature vector f(T) as the input of the encoder. In the encoder, there are two encoder layers. After the first encoder layer inputs f(T), the multi-head attention mechanism is used to enhance the target feature information in f(T). The specific calculation process is as follows: MultiHead(Q,K,V)=ConCat(head1,...,head n )W O , head i =Attention(QW i Q ,KW i K ,VW i V ), where Q, K, and V are identical and equal to f(T), W i Q 、W i K 、W i V represents the weight matrix that focuses on information in different directions. i∈[1,8] indicates that the multi-head attention mechanism focuses on information in eight locations, mapping the encoded content to eight spaces and enhancing the model's representational capabilities. After passing f(T) through the multi-head attention mechanism, the feature enc(T) is obtained. Then, f(T) and enc(T) are directly added together, followed by a normalization layer and a feedforward neural network (FFN), where the FNN consists of two linear layers and a normalization layer. These operations ultimately yield the first encoded feature enc'(T), which can be expressed as: enc'(T) = FFN(Norm(f(T) + enc(T))). The reference region features are then further enhanced through the second encoder layer, following the same process as the first encoder layer. Finally, cat(T) passes through the encoder of the similarity calculation module to obtain the encoded feature enc"(T).

[0041] Step 4: Input 2-n frames of infrared image sequence, take the center position of the target in the previous frame as the origin, and fill the background area with 4.5 times the diagonal length of the target bounding box in the previous frame to form the tracking area X i ,i∈[2,n].

[0042] Step 5: Set the area to be tracked to X i The dual feature extraction module (DEM) is sent to extract features to obtain the fusion feature map cat(X i ).

[0043] Step 6: The fusion feature map cat(X i ) is input into the decoder in the Similarity Computation Module (SCM), refer to Figure 3 The decoder structure in . The decoder also has two decoder layers. The first decoder layer includes an attention module, and the second decoder layer includes an attention module and a feedforward neural network. The attention module is composed of a multi-head attention mechanism and a normalization layer. Feature map cat(X i ) After spatial position encoding and flattening operations, it passes through the first decoder layer to enhance the target information and key background information in the fusion feature map of the tracking area, and outputs the first decoding feature dec'(X i ). In the second decoder layer, dec'(X i ) and the encoded features enc”(T) as input, when passing into the multi-head attention mechanism, the Q, K, V variables are no longer exactly the same, but Q = dec'(X i ), K=V=en c”(T), and finally the second decoder layer outputs the second decoding feature dec”(X i ). Then, use the view function in the Pytorch deep learning library to perform dec" (X i )The feature map is scaled and transformed into a similarity map R of size 31×31×520.

[0044] Step 7: Pass the similarity graph R into the refinement module (RM) to obtain the refined similarity graph up(R). Its network structure is referenced Figure 4. Among them, the refinement module contains 5 layers of networks. The first and second layers of the network both contain a transposed convolution with a convolution kernel of 3×3 and a stride of 2×2, and a double convolution block. The double convolution block is composed of two convolution blocks. Each convolution block contains a convolution layer with a convolution kernel of 3×3, a padding of 1, and a stride of 1, a normalization layer, and an activation function layer. The main difference between the other three layers of the network and the previous two layers is the size and stride of the transposed convolution kernel, which is mainly reflected in: the transposed convolution kernel of the third layer is 2×2 and the stride is 2, the transposed convolution kernel of the fourth layer is 2×2 and the stride is 1, and the transposed convolution kernel of the fifth layer is 1×1 and the stride is 1. Finally, through a 2D convolution with a 1×1 convolution kernel, a 1×255×255 refined similarity map up(R) is obtained.

[0045] Step 8: When tracking targets in the third frame and beyond, the reference region is updated using the Region Update Module (RUM). First, the previous infrared image and the predicted bounding box are input. The infrared image is then expanded in all directions, using the predicted bounding box's center as the origin, by a distance twice the bounding box's diagonal. This results in a new reference image, T'. Finally, the new encoded features, enc" (T'), are obtained through the dual feature extraction module and the encoder portion of the similarity calculation module.

[0046] Step 9: Establishment of training data set: The present invention uses the modified LaTOT data set (104,726 images) as the basic data set for training, but in order to improve the robustness and accuracy of the network of the present invention, the LaTOT data set is modified: First, for an image in the LaTOT data set, the target center is used as the origin and expanded to all sides to obtain a basic image; then, random translation, scaling, blurring, and mirroring operations are added to generate a new image. All LaTOT images go through the above two steps to obtain a modified LaTOT data set. When the present invention inputs training images, each image is input and cropped to obtain a reference image. The image to be tracked is a frame of image randomly selected 30 frames before and after the position of the image sequence where the reference image is located, and is obtained through cropping.

[0047] Step 10: Test Dataset Creation: This paper uses the modified DIRST dataset (13,655 images) as the test set. In the original DIRST dataset, the target size in some image sequences is 1×1 pixels, and it is impossible to extract effective depth features or oriented gradient histogram features. To this end, a new target of 5×5-7×7 pixels is overlaid on the 1×1 pixel target, and the grayscale of the new target conforms to the two-dimensional Gaussian distribution.

[0048] Step 11: Model Training: The proposed method for tracking small infrared targets in complex backgrounds is an end-to-end network model. Two cropped infrared images are fed into the network model as the reference area and the area to be tracked. Multiple iterations of optimization are performed. The classification results, center deviation results, and bounding box results output by the head network are compared with the corresponding labels, and the network parameters are optimized using gradient derivatives. The test results are evaluated using IOU and precision (center point Euclidean distance).

[0049] Step 12: Model training parameter settings: The model was trained on a Windows server with an NVIDIA RTX 3090 graphics card and 24GB of video memory. The test software used was PyCharm 2021.2.2. The total number of training iterations was 50. The learning rate was initialized to 0.01 and decreased exponentially during the iterations until it reached 0.0005. The stochastic gradient descent (SGD) optimizer was used, and the network framework was PyTorch 1.8.0.

Claims

1. A method for tracking small infrared targets in complex backgrounds, characterized by: The following steps are involved: Step 1: Input an infrared image sequence Z containing a small infrared target; Step 2: Select the target area in the first frame of the infrared image sequence Z as the reference area T; Step 3: Input the reference region T into the dual feature extraction module to obtain the fused feature map cat(T); Step 4: The subsequent frames of the infrared image sequence Z are taken as the origin, and the target center position of the previous frame is used to obtain the tracking area X. i ,i∈[2,n], n represents the total number of frames in the Z sequence; Step 5: Set the area to be tracked to X i Input into the dual feature extraction module to obtain the fusion feature map cat(X i ); Step 6: Input the two fused feature maps together into the similarity calculation module to obtain the similarity map R; Step 7: The similarity map R is refined by the module to obtain a similarity map up(R) that is consistent with the size of the area to be tracked. Step 8: The similarity graph up(R) outputs the target center point position and bounding box size through the head network to obtain the tracking result of the current frame i; Step 9: Replace the target area tracked in the current frame i with the reference area T, and continue with steps 3 to 9 until the sequence ends.

2. The method for tracking small infrared targets under complex backgrounds according to claim 1, wherein: The dual feature extraction module consists of two parts: a deep feature extraction network and a grayscale gradient feature extraction method. The deep feature extraction network uses deep learning and gradient descent to enable the network to adaptively learn shallow detail features and deep semantic features in the image. The grayscale gradient feature extraction method uses mathematical calculations to extract the grayscale gradient histogram features of local areas in the image. The similarity calculation module includes a Transformer network structure, which uses the self-attention mechanism to enhance the target information of the feature map and suppress the background information. The cross-attention mechanism calculates the similarity between feature maps from a global level to obtain a similarity map. The refinement module modifies the U-Net network in the image segmentation task to reduce the receptive field of each pixel in the similarity map and maintain a one-to-one correspondence with the pixels in the area to be tracked, thereby increasing the amount of information in the similarity map and improving the accuracy of the head network output. The region update module is specifically reflected in the testing process, using the predicted target region of the target tracked in the previous frame as the new reference region.

3. The method for tracking small infrared targets in complex backgrounds according to claim 2, wherein: The dual feature extraction module consists of a ResNet-18 deep network and a histogram of oriented gradients feature extractor. The reference area and the area to be tracked are respectively passed into the ResNet-18 network and the histogram of oriented gradients feature extractor to obtain feature maps, which are then concatenated at the dimensional level to obtain a fused feature map.

4. The method for tracking small infrared targets under complex backgrounds according to claim 2, wherein: The similarity calculation module adds position encoding to the reference area feature map, and changes its shape into a multi-dimensional vector through the view function in pytorch, so that each pixel in the feature map records spatial information. Then, through the self-attention mechanism in the transformer, the target information in the feature map is enhanced and the background area is suppressed. The feature map of the area to be tracked is obtained by the same operation to obtain a one-dimensional vector with added position encoding. Then, through the multi-head attention mechanism, the target information in the feature map of the area to be tracked is enhanced. Finally, the enhanced reference area feature map and the feature map of the area to be tracked are jointly passed into the cross-attention mechanism of the transformer, and the position most similar to the reference area feature map is searched in the enhanced feature map of the area to be tracked to generate a similarity map.

5. The method for tracking small infrared targets under complex backgrounds according to claim 2, wherein: The refinement module is split from the U-Net network in the image segmentation task, retaining only the upsampling network. The input of the refinement module is the output similarity map of the similarity calculation module, and finally a refined similarity map up(R) with the same size as the area to be tracked is obtained.

6. The method for tracking small infrared targets under complex backgrounds according to claim 2, wherein: The region update module enables the tracking method to continuously use the target region and part of the background region of the previous frame of the tracked image as reference regions during the testing process.

7. The method for tracking small infrared targets under complex backgrounds according to claim 2, wherein: The image sequence of the training set is obtained by the following process: the LaTOT dataset is selected as the basic dataset for model training. First, each image in the image sequence is cropped from the origin to the surrounding area with the target center point as the origin and the cropping width is 10 times the diagonal length of the bounding box to obtain a new image of 511×511; then, the new image is translated, scaled, blurred, and mirrored to offset the target from the original position. The training reference image is from the first frame to the last frame of the sequence, and the image to be tracked is randomly selected within 30 frames before and after the reference image.

8. The method for tracking small infrared targets under complex backgrounds according to claim 7, characterized in that: The image sequence of the test set is obtained by the following process: the DIRST dataset is selected as the test set. In the case where the target is too small, a simulated infrared weak target in the range of 5×5-7×7 is added to the real target, whose grayscale value is Gaussian distributed, and the target is labeled.

Images