A transformer-based anti-interference target tracking algorithm

Abstract

The application discloses an anti-interference target tracking algorithm based on a transformer, wherein the algorithm comprises the following steps: video image acquisition; feature extraction and training on sample data; continuous tracking on a selected target; coping strategy mechanism for phenomena such as appearance change, similar object interference and occlusion; an attention module increases the extraction capability of the network for target features; training on the built target tracking network; if no interference phenomenon occurs, the algorithm directly returns the tracking result and displays the tracking result on a computer; if the algorithm judges that interference occurs, the algorithm adopts an interference coping strategy, returns the tracking result after excluding the interference influence and displays the tracking result on the computer. The application has high application value in the fields of unmanned aerial vehicle tracking, video monitoring, intelligent driving and the like, and can more continuously and accurately track interested objects appearing in related field tasks.

Description

Technical Field

[0001] This invention relates to a target tracking algorithm, belonging to the field of deep learning. Background Technology

[0002] With the gradual development of single-object tracking technology, the introduction of deep learning has significantly improved tracking accuracy. However, these tracking algorithms have not demonstrated better adaptability when facing problems such as changes in the appearance of the tracked object, occlusion, and interference from similar objects. Therefore, developing an interference-resistant target tracking algorithm is extremely important. Existing interference-resistant target tracking algorithms using deep learning often suffer from insufficient interference resistance and significantly reduced tracking speed. In view of this, this invention proposes an interference-resistant target tracking algorithm, aiming to improve the interference resistance and tracking speed of target tracking to meet practical application needs.

[0003] CN202411599372.0 discloses a target tracking method based on an improved ResNet network. The method includes acquiring video data of the scene to be tested and performing frame segmentation to obtain several images; constructing an improved ResNet network by replacing the ordinary convolutions in stage 0 with partial convolutions and introducing CBAM after the partial convolutions; improving the Bottleneck in stages 1 to 4 using SAAN; training the improved ResNet network using a training set and converging the model using a loss function. This invention utilizes a cGAN network to synthesize different image data to augment the dataset; replaces standard convolutions with partial convolutions in the network structure and introduces CBAM after the convolutions; and integrates SAAN into the residual blocks to capture long-range dependencies and improve the accuracy of continuous tracking.

[0004] CN202510157654.3 discloses a target tracking method, specifically an interference-resistant rotating target tracking method, which solves the problems of existing methods' inability to accurately track rotating targets and computational complexity. This method achieves interference-resistant target tracking through the following algorithmic flow: acquiring a first frame image patch; performing affine rotation; acquiring the target rotation angle in the second frame; acquiring the final position of the target in the second frame; acquiring the second frame as the current frame; acquiring the current frame image patch; performing affine rotation; acquiring the target rotation angle in the next frame; determining whether there is occlusion; if occlusion occurs, updating the parameters by using the response map position as the final position of the target in the next frame; if occlusion still occurs, fusing Kalman filtering to obtain the final position of the target in the next frame. This invention can effectively reduce the interference effects during rotating target tracking while maintaining a relatively fast tracking speed.

[0005] CN202510128063.3 discloses a low-energy-consumption 3D point cloud single-target tracking method based on a spiking neural network, comprising: acquiring 3D point cloud data of the target in the previous and current frames; designing a voxel mesh module to divide the point cloud into 3D voxels; designing a feature extraction module that uses a three-stage spiking convolutional block as a feature extractor to extract voxelized point cloud features and adding a Transformer encoder layer to capture the global context information of the point cloud through a self-attention mechanism; designing a partial-to-part motion modeling module that models and encodes the continuous point cloud features in the time series part by part through a two-dimensional spiking convolutional block to obtain the feature information changes in the corresponding spatial parts of the previous and current frames; and designing a regression prediction module that infers the relative motion of the target through a spiking regression prediction module. This invention reduces energy consumption and improves efficiency while ensuring accurate target tracking in diverse scenes and motion states.

[0006] CN202510059022.3 discloses a target tracking method and system based on dual-path attention and channel feature optimization. This method includes filtering out interactions with low query key matching scores through efficient self-attention branches to reduce the impact of irrelevant information; considering all query key pairs through comprehensive self-attention branches to obtain attention scores, ensuring necessary information flow in the network; utilizing channel aggregation capabilities to reduce redundant information in feature representation; fusing features together using adaptively adjustable weights through two types of attention, and feeding the fused features into a channel feature optimization module to further enhance feature representation and reduce computational redundancy. This invention constructs a Transformer-based feature fusion network by fully combining the advantages of efficient and comprehensive self-attention, achieving feature aggregation through adaptively adjusting branch outputs, better focusing on important features and ignoring irrelevant information, thus reducing computational burden.

[0007] CN202411828432.1 discloses an efficient pedestrian following method resistant to long-term occlusion, comprising two parts: constructing a behavior discriminator based on pedestrian re-identification using target tracking data and a novel dynamic update mechanism for pedestrian features. The main steps of this method are: initializing the pedestrian tracker by acquiring target detection information of selected image frames; constructing a behavior discriminator to monitor changes in the motion state between pedestrians; predicting target information under continuous observation state by associating inter-frame trajectories through the tracker; recovering the target identity representation under discrete observation point state based on the pedestrian re-identification network; dynamically adjusting the feature update weights of historical frames and the current frame according to the degree of target occlusion to weaken noise interference introduced by occlusion and optimize pedestrian following effect. This invention is applicable to various tracking algorithms to improve the performance resistant to long-term occlusion, and its tracking stability and computational cost are superior to the current method of tracking and then re-identifying pedestrians frame by frame. It has been successfully applied in robot following tasks. Summary of the Invention

[0008] This invention provides an anti-interference target tracking algorithm, including a target tracking network model and an anti-interference countermeasure mechanism:

[0009] First, video information of the target is collected using video recording equipment; the target is selected and fed into the tracking network for target initialization and model training; if the tracking is not interfered with, the initial target is continuously tracked and predicted in subsequent videos; if the model determines that the tracking is interfered with, the type of interference is determined and different anti-interference strategies are adopted according to the current type of interference; the final target tracking result is returned to the computer for display.

[0010] This application provides an anti-interference target tracking algorithm, comprising:

[0011] 1. Acquiring sample data: A large amount of training sample data was collected using various video shooting devices under various interference conditions. The video was converted into images frame by frame, and finally each sample data was labeled to complete the production of the training dataset.

[0012] 2. Construct a target tracking network: Build a target tracking network to initialize and continuously track the target from the received video images;

[0013] Among them, the target tracking network is based on the Transformer design of deep learning. It increases the receptive field by dividing the input image into blocks and performing linear projection operations on each block, thereby speeding up the model's information processing speed.

[0014] In this process, after block segmentation and linear projection, the obtained information is concatenated into a vector, and after adding the corresponding positional encoding, it is fed into the encoding layer of the model. By using a global multi-head attention module, the ability to extract features of the tracking target is enhanced, thereby effectively capturing complex features in the image;

[0015] After the above processing, the obtained features are sent to the early candidate elimination module to delete unimportant information in order to increase the model's tracking speed.

[0016] Specifically, the information from the early candidate outputs is fed into a multilayer perceptron to further transform the data and enhance the nonlinear expressive power of the model; and the output of the multilayer perceptron is residually connected with the output of the global multi-head attention module, which helps with model gradient flow and model training.

[0017] The integrated features are padded back to their original length with zeros and then fed into the model's prediction head to perform tracking and prediction of the target.

[0018] 3. Construct an anti-interference countermeasure mechanism: Build a generative adversarial image dehazing network to dehaze received foggy video images;

[0019] Among these methods, scoring is performed on the predicted target area of ​​the tracked object to determine whether the tracking is interfered with;

[0020] Specifically, when it is determined that the target is not disturbed, normal high-efficiency target tracking is used and the target tracking model is updated based on the tracking results;

[0021] Specifically, when tracking is found to be interfered with, the type of interference is determined by listing possible target areas; when there are multiple possible targets, it is determined to be interference from similar objects; when there is only one possible target, it is determined to be a change in the appearance of the tracking target; when there are no possible targets, it is determined to be that the tracking target is occluded.

[0022] In response to the interference from similar objects, the model combines historical information to establish a corresponding relationship for each similar object in order to eliminate interference;

[0023] Specifically, the model is selectively updated to reduce the dilution of features by low-quality samples in response to changes in the appearance of the tracked target.

[0024] Specifically, when the tracked target is occluded, the model performs motion estimation to predict the possible location of the target;

[0025] 4. Perform image preprocessing and data augmentation on the sample data to obtain training and test sets;

[0026] The preprocessing process involves scaling the image by 2x or 4x, rotating it by 90, 180, or 270 degrees, translating it, and cropping it.

[0027] This application uses the following image enhancement techniques:

[0028] (1) Rotation and flipping: By randomly rotating and flipping the image (horizontally or vertically), the model can learn objects at different angles and directions;

[0029] (2) Scaling and cropping: These operations can change the scale and size of an image, helping the model to recognize objects of different sizes. For example, an image can be scaled to different sizes or randomly cropped;

[0030] (3) Image noise: Gaussian noise and salt-and-pepper noise are added to the image to augment the data and improve the robustness and generalization ability of the model.

[0031] 5. Train the parameters of the target tracking network using the training set, wherein the test set is used to test the performance of the target tracking model on unseen data.

[0032] The beneficial effects of adopting the above technical solution are as follows:

[0033] This application can be used for real-time target tracking in fields such as UAV monitoring and video surveillance. It exhibits better anti-interference and tracking adaptability for targets whose appearance changes, are interfered with by similar objects, or are occluded. Furthermore, compared to ordinary tracking algorithms, it offers better tracking accuracy and faster tracking speed. Attached Figure Description

[0034] Figure 1 This application provides a schematic diagram of the anti-interference target tracking algorithm process;

[0035] Figure 2 This application provides a network diagram for anti-jamming target tracking;

[0036] Figure 3 This application provides a schematic diagram of the Global Channel Multi-Head Attention Mechanism (GCMHAM) module in an anti-jamming target tracking network.

[0037] Figure 4 This is a schematic diagram illustrating the operation of the anti-interference countermeasure mechanism of the anti-interference target tracking algorithm provided in this application;

[0038] Figure 5 This is a diagram showing the results of the proposed algorithm performing target tracking on the OTB-100 target tracking dataset;

[0039] Figure 6 This is a graph showing the results of the proposed algorithm performing target tracking on the VOT2018 target tracking dataset. Detailed Implementation

[0040] To facilitate a clearer understanding and use of this application, the following detailed description, in conjunction with the accompanying drawings and embodiments, will enhance the understanding of those skilled in the art and those who use the application. These embodiments are merely partial illustrative examples of this application. Any other embodiments of this application obtained by the recipient without inventive effort are within the scope of this application and are protected.

[0041] Figure 1 A schematic diagram of the anti-interference target tracking algorithm is provided for this application.

[0042] Step 101 First, select the target to be tracked, and then initialize and train the tracking model based on it; Figure 2 This is a schematic diagram of the constructed tracking model structure;

[0043] Step 201 involves inputting image pairs into the network, including initializing the target region and the tracked image during subsequent tracking.

[0044] Step 202 involves preprocessing the image before it is input into the network, including slicing the image into blocks, linearly projecting it into vectors, and adding positional encoding to obtain the final input vector.

[0045] 203 is the encoder of the network, which includes the normalization layer (Layer Norm), the global channel multi-head attention module (GCMHAM), the early candidate elimination module (Early Candidate Elimination), and the multilayer perceptron (MLP);

[0046] 204 is used to tokenize the output vector to ensure that the lengths of the input and output vectors are equal; then the vector is fed into the prediction header of the model.

[0047] Step 205 predicts the target region in the current tracking image based on the input vector to obtain the tracking result.

[0048] Step 102 calculates the confidence score of the possible regions of the prediction results during the tracking process to obtain the probability score that the region may be the tracking target.

[0049] Step 103 determines whether the current tracking is being interfered with, based on the number of possible areas currently existing.

[0050] Step 104 involves the model eliminating interference based on its type when it determines that the current tracking is being interfered with; a detailed diagram of the elimination steps is shown below. Figure 4 As shown;

[0051] 401 is the response strategy when there are multiple possible regions, i.e., when similar objects interfere; the algorithm obtains the probability that each region is the tracking target by listing the current possible regions as a candidate region set and associating and scoring them with regions in historical information.

[0052] In this process, a similarity score is obtained by associating each region in the historical information with the candidate regions in the candidate region set. When the similarity score of the current candidate region with a region other than the target region in a certain historical information is higher than that of the historical target region, it is judged as an interference object; otherwise, it is judged as the tracking target. This process is expressed by the formula shown in Equation (1):

[0053]

[0054] In the formula: a certain subsequent region is Candidate region set s t With s t-1 These are the current region similarity score and the historical region similarity score, respectively. c A score threshold used to measure the degree of change in the target score;

[0055] In the case where the candidate region set list is empty or there is no candidate region that satisfies the condition of being judged as a tracking target (401), the algorithm uses a motion estimation method to calculate the possible motion trajectory of the target based on several historical frames. The actual position of the target in the previous frame is used as the base position, and the bounding box is moved and changed to obtain the predicted bounding box. The detailed calculation process of the predicted bounding box is shown in equation (2).

[0056]

[0057] In the formula: A1 and A2 are both hyperparameter coefficient weights, used to adjust the influence of different frames on the overall result in motion estimation.

[0058] Step 105 is when the model determines that the current tracking is not disturbed and continues to perform normal, efficient tracking.

[0059] Step 106 is to predict the current position of the target when there is no interference or the interference has been eliminated, and update the model based on the prediction result.

[0060] Step 107 is that after obtaining the prediction results, the model will feed back the tracking results to the computer and display them by generating region bounding boxes.

[0061] The following is combined with Figure 2 The present application will be further described in detail with reference to the embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the application. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the accompanying drawings.

[0062] The server is configured with a 12th Gen Intel(R) Core(TM) i7-12700H CPU, 2.30GHz, and 32GB of RAM. The test platform is Windows 11 Home Edition and Python 3.8.

[0063] The network input is a pair of images (201), namely the initial sample image patch and the current search region image.

[0064] The image is segmented and flattened into a bag sequence (202), which is then projected into the latent space using a parameterized trainable linear projection layer; learnable position embeddings are added to the bag embeddings of the template and search region to produce the final template label embedding and search region label embedding, which are then fed into the Transformer encoder layer.

[0065] In the encoder layer (203), the input vector first passes through a normalization layer, and then is fed into the GCMHAM module. The detailed structure of the module is as follows: Figure 3 .

[0066] In GCMHAM, the input tensor is split and then subjected to max pooling and average pooling to obtain a tensor of shape (301); these two vectors are then multiplied by the Values ​​vector obtained in step 303 after passing through the SoftMax activation function.

[0067] The result vector obtained in step 301 will be restored to its original length through a linear transformation, then normalized by a Sigmoid operation and transformed into a global channel coefficient matrix (302), as shown in equation (3):

[0068] y = linear(Concat(V) max V Avg )).sigmoid (3)

[0069] Simultaneously with step 301, the original tensor is split into three tensors: Query, Keys, and Values ​​(303). The resulting Query and Keys vectors are multiplied by the scale, and the result is multiplied by the Values ​​tensor after Softmax and Dropout operations to obtain the output at that position. The calculation process is shown in equation (4).

[0070] z'=dropout(softmax((Query'·Keys'.transpose(-2,-1))*scale))·Values'(4)

[0071] The original shape is restored by performing a linear transformation on the result obtained in step 303 to facilitate subsequent operations (304).

[0072] Finally, the results obtained from 302 and 304 are used to calculate the Hadamard product and then passed through the Dropout layer to obtain the final output (305). The calculation process is shown in equation (5).

[0073]

[0074] Continuing with step 202, the output of GCMHAM is residually concatenated with the original input and then fed into the network's Early Candidate Elimination module. This module allows each block region in each search region image to calculate a similarity score with the sample image as its score. The candidate regions with the highest scores are retained, while the remaining candidate regions are discarded. Then, the output is fed into the normalization layer, and the result is fed into the Multilayer Perceptron (MLP) and residually concatenated to obtain the final output of the encoder.

[0075] At the head of the network (204), the network reinterprets the search region label sequence after candidate elimination and zero-padding as a 2D spatial feature map, and then feeds it into a fully convolutional network, which consists of several stacked Conv-BN-ReLU layers. The output contains the object classification score map and local offsets to compensate for discretization errors caused by reduced resolution and normalized bounding box size.

[0076] The position with the highest classification score in step 204 is the target position (205).

[0077] To demonstrate the improved tracking performance of the proposed model and to verify the effectiveness of the specific improvements, a comparative experiment was conducted on the OTB-100 dataset using the proposed algorithm and other mainstream tracking algorithms. The overall results of the experiment are shown in Table 1.

[0078] Table 1: Test Results of the OTB-100 Dataset

[0079]

[0080] The experimental results are shown in Table 1. The results demonstrate that the algorithm in this application achieves improvements in all metrics compared to the current mainstream algorithms on the OTB-100 dataset. Specifically, AUC (Area under curve): the area under the curve of the curve representing the intersection-union ratio of pixels within the predicted bounding box and the target ground truth bounding box as different thresholds representing tracking success change; OP (Overlap precision): the overlap ratio, i.e., the intersection-union ratio of pixels within the predicted bounding box and the target ground truth bounding box, where OP50 and OP75 represent the success rates when the thresholds representing tracking success are set to 0.5 and 0.75, respectively (i.e., the predicted bounding box and the target ground truth bounding box have more than 50% overlap and more than 75% overlap); Precision: the average Euclidean distance between the centers of the tracked predicted bounding box and the target ground truth bounding box, with a threshold of 20 (i.e., a center distance within 20 is considered successful); Norm Precision: normalized precision, which, considering the scale of the target box, calculates the ratio of the Euclidean distance between the center points of the predicted bounding box and the target ground truth bounding box to the hypotenuse of the target ground truth bounding box.

[0081] To evaluate the generalization ability of the algorithm, this application also conducted experiments using the VOT2018 target tracking dataset. The results are shown in Table 2. The algorithm in this application outperforms other mainstream algorithms on the VOT2018 dataset.

[0082] Table 2: Test Results of VOT2018 Dataset

[0083]

[0084] To demonstrate the detection effectiveness of the model in this application, experimental results of the algorithm in this application and the classic target tracking algorithm OSTrack are presented, such as... Figure 5 , Figure 6 As shown, the two algorithms perform target tracking on the OTB-100 dataset and the VOT2018 dataset. Figure 5 , Figure 6 As shown, during the tracking process, when faced with some occlusion phenomena and interference from similar objects, the algorithm of this application can effectively avoid these interferences and maintain normal tracking.

[0085] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A target tracking method, characterized in that: It can achieve efficient and accurate anti-interference target tracking; First, real-time video images are acquired using various shooting devices; target regions are selected based on the acquired video information; a network model is constructed and initialized based on target region samples; the target is continuously tracked and the trained network model is updated; and it is determined whether the tracking process is interfered with. Depending on the type of interference, an interference mitigation strategy is used to eliminate its impact; if the target cannot be tracked normally, motion estimation is used to predict the target's position; the tracking results are then returned and displayed on the computer. The specific steps of this method are as follows: S1: Acquire video images; S2: Acquire sample data and select tracking targets based on video images; S3: Build and initialize the target tracking network; S4: Target tracking and real-time network model updates; S5: Determine whether the current tracking is being interfered with based on the method, and use the corresponding anti-interference strategy to eliminate the interference effect based on the interference situation; S6: Perform motion estimation for targets that cannot be tracked normally; S7: The system returns the target tracking results in real time after eliminating interference effects; The step S3 of constructing and initializing the target tracking network includes: Extract target features: 1) Image segmentation: By dividing the image into smaller segments, the attention module's ability to focus on each region of the image is enhanced, thereby improving the model's global perspective; 2) Combining sample images and search region images: By combining sample images and search region images, a single-stream Transformer mode is implemented, enhancing the model's computational speed and efficiency; 3) Write positional encoding: By adding positional encoding to the input image tensor after the above combination, positional information of different blocks is provided to the model for subsequent operations; The constructed target tracking network coding layer: 1) Normalization layer: By standardizing the input or output of each layer, it stabilizes training, accelerates convergence, and alleviates gradient problems; 2) Global Channel Multi-Head Attention Module: By splitting the input tensor into query, key, and value multi-head vectors, and performing global max pooling, global average pooling, linear transformation, Dropout, Softmax, Sigmoid, and Hadamard product to extract features comprehensively and in detail from multiple angles; 3) Residual connections: Residual connections directly add the layer's input to the output, avoiding the gradient vanishing problem in deep networks and allowing the model to train deeper architectures more efficiently; 4) Early candidate elimination module: This module can remove useless feature information, effectively shortening the model training and prediction forward time; 5) Multilayer perceptron: Performs nonlinear transformation and dimensionality adjustment on the features of self-attention output to enhance the model's expressive power, and works with residual connections to stabilize training; The constructed target tracking network head: 1) Split the feature vector: Split the features of the sample image and the search region image into the feature vector for subsequent operations; 2) Token filling and reshaping: Fill in the portions deleted in the early candidate elimination module to ensure that the lengths of the input and output tensors are consistent; 3) Target prediction: Based on the tensor information mentioned above, the target is predicted to obtain the planar coordinate information of the target area.

2. The target tracking method according to claim 1, characterized in that, Step S3 includes: 1) Input the target region sample of the video image into the target tracking network to extract the target region features; 2) Construct the network model based on the initialized weights; 3) Use the above target region features as template features, save and adjust the model parameters.

3. The target tracking method according to claim 2, characterized in that, The step S5 includes: 1) Generate a probability score for the candidate regions within the current search area; 2) Based on the above scores, list the candidate region set, construct and initialize the target tracking network; 3) Establish the association between the candidate regions in the candidate region set and the regions in historical information to determine the current interference type; 4) Based on the above judgment, filter candidate regions and eliminate corresponding interference effects.

4. The target tracking method according to claim 2, characterized in that, The step S6 includes: 1) Record the location information of prediction results in historical data; 2) Calculate the movement direction and speed trend of the target using the historical information mentioned above; 3) Estimate the possible motion state of the currently lost target and predict its current position based on the movement direction and speed trend of the tracked target.

5. The target tracking method according to claim 3, characterized in that, The anti-interference strategy includes: 1) Generate scores for prediction results: Make predictions based on the sample template and generate similarity scores; 2) Generate scores for other candidate regions and summarize them into a candidate region set: Select candidate regions with higher scores through an adaptive candidate threshold to form a candidate region set; 3) Eliminate similar objects: Combine historical information to establish an association between each candidate region set and the regions in the historical information. Candidate regions that are not the target region are identified as interference and deleted. 4) Return to the candidate region after the above filtering.

6. The target tracking method according to claim 1, characterized in that, The motion estimation S6 includes: 1) Obtain historical target location information: Based on the saved historical target location information, obtain the coordinates and area size information of past targets for later use; 2) Calculate the motion state based on historical information; 3) Obtain and return the motion estimation results.

Images