Transform-based anti-interference target tracking algorithm

Abstract

The invention discloses an anti-interference target tracking algorithm based on Transform. The algorithm comprises the following steps: collecting a video image; performing feature extraction and training on the sample data; continuously tracking the selected target; a coping strategy mechanism for appearance change, similar object interference, shielding and other phenomena; an attention module is used for increasing the extraction capability of the network on target features; the built target tracking network is trained; and if the interference phenomenon does not occur, the algorithm directly returns and displays the tracking result on the computer. If the algorithm judges that interference occurs, the algorithm adopts an interference coping strategy, and after interference influence is eliminated, a tracking result is returned and displayed on the computer. The method has high application value in the fields of unmanned aerial vehicle tracking, video monitoring, intelligent driving and the like, and interested objects appearing in tasks in the related fields can be tracked more continuously and accurately.

Description

Technical Field

[0001] The present invention relates to a target tracking algorithm, belonging to the field of deep learning Background Art

[0002] With the gradual development of single target tracking technology, the introduction of deep learning single target tracking technology has significantly improved the accuracy of tracking. However, when faced with problems such as appearance changes, occlusion, and interference from similar objects in tracking tasks, these tracking algorithms do not show better adaptability. Therefore, it is extremely important to develop an interference-resistant target tracking algorithm. Existing interference-resistant target tracking algorithms using deep learning often have problems such as insufficient anti-interference ability and significantly reduced tracking speed. In view of this, the present invention proposes an interference-resistant target tracking algorithm, which aims to improve the target tracking anti-interference ability and tracking speed to meet the needs of practical applications.

[0003] CN202411599372.0 discloses a target tracking method based on an improved ResNet network, including collecting video data of the scene to be tested and performing frame processing to obtain several images; constructing an improved ResNet network, replacing the ordinary convolution in stage 0 of the ResNet network with partial convolution, and introducing CBAM after the partial convolution; and using SAAN to improve the Bottleneck in stages 1 to 4; using a training set to train the improved ResNet network, and using a loss function to converge the model. The invention uses a cGAN network to synthesize different image data to expand the data set; replacing the standard convolution with partialconv in the network structure and introducing CBAM after the convolution; integrating SAAN in the residual block to capture long-distance dependencies and improve the accuracy of continuous tracking.

[0004] CN202510157654.3 discloses a target tracking method, which relates to an anti-interference rotating target tracking method, and solves the problem that the existing method cannot accurately track the rotating target and the calculation is complex. The method realizes anti-interference target tracking through the following algorithm process design: obtaining the first frame image block; performing affine rotation; obtaining the angle of rotation of the second frame target; obtaining the final position of the second frame target; obtaining the second frame as the current frame; obtaining the current frame image block; performing affine rotation; obtaining the angle of rotation of the next frame target; judging whether it is occluded, if it is occluded, the response image position is used as the final position update parameter of the next frame target; if it is occluded, the Kalman filter is integrated to obtain the final position of the next frame target. The invention can better reduce the interference effect in the rotating target tracking process while maintaining a faster tracking speed.

[0005] CN202510128063.3 discloses a low-energy three-dimensional point cloud single target tracking method based on a pulse neural network, including: obtaining the three-dimensional point cloud data of the previous frame and the current frame of the target, designing a voxel grid module to divide the point cloud into 3D voxels; designing a feature extraction module, using a three-stage pulse convolution block as a feature extractor to extract voxelized point cloud features, and adding a Transformer encoder layer to capture the global context information of the point cloud through a self-attention mechanism; designing a part-to-part motion modeling module, modeling and encoding the continuous point cloud features on the time series part by part through a two-dimensional pulse convolution block, and obtaining the feature information changes of the previous frame and the current frame in the corresponding spatial part; designing a regression prediction module, and inferring the relative motion of the target through the pulse regression prediction module. While reducing energy consumption and improving efficiency, this invention ensures that the network can accurately track the target in a variety of scenarios and motion states.

[0006] CN202510059022.3 discloses a target tracking method and system based on dual-path attention and channel feature optimization, which includes filtering out interactions with low query key matching scores through an efficient self-attention branch to reduce the impact of irrelevant information; considering all query key pairs through a comprehensive self-attention branch to obtain attention scores to ensure the necessary information flow in the network; utilizing channel aggregation capabilities to reduce redundant information in feature representation; the two types of attention fuse features together through adaptively adjustable weights, and send the fused features to a channel feature optimization module to further enhance feature representation and reduce computational redundancy; the present invention fully combines the advantages of efficient self-attention and comprehensive self-attention to construct a Transformer-based feature fusion network, realizes feature aggregation through the output of the adaptive adjustment branch, better focuses on important features and ignores irrelevant information, and reduces the computational burden.

[0007] CN202411828432.1 discloses an efficient pedestrian following method that is resistant to long-term occlusion, including two parts: constructing a behavior discriminator based on pedestrian re-identification for target tracking data and a new pedestrian feature dynamic update mechanism. The main steps of this method are: obtaining target detection information of selected image frames to initialize the pedestrian tracker, constructing a behavior discriminator to supervise the changes in motion state between pedestrians, predicting target information under continuous observation state by associating inter-frame trajectories of the tracker, and restoring the target identity representation under discrete observation point state based on the pedestrian re-identification network; dynamically adjusting the feature update weights of historical frames and current frames according to the degree of target occlusion, weakening the noise interference introduced by occlusion, and optimizing the pedestrian following effect; this invention is applicable to a variety of tracking algorithms to improve the performance against long-term occlusion, and the tracking stability and computational cost are better than the current tracking and then frame-by-frame pedestrian re-identification method, and has been successfully applied in robot following tasks. Summary of the Invention

[0008] The present invention provides an anti-interference target tracking algorithm, including a target tracking network model and an anti-interference countermeasure mechanism: First, video information of the tracking target is collected through a video capture device; the tracking target is selected and sent to the tracking network for tracking target initialization and model training; if the tracking is not disturbed, the initial tracking target is continuously tracked and predicted in subsequent videos; if the model determines that the tracking is disturbed, the type of interference is determined, and different anti-interference countermeasures are adopted according to the current interference type; the final target tracking result is returned to the computer for display.

[0009] This application provides an anti-interference target tracking algorithm, including: 1. Obtain sample data: Use various video capture devices to collect a large amount of training sample data under various interference scenarios, convert the video into each frame of image, and finally label each sample data to complete the production of training data sets; 2. Build a target tracking network: Build a target tracking network to perform initial training and continuous tracking of the received video images; The target tracking network is based on the deep learning Transformer design. It divides the input image into blocks and performs linear projection operations on each block to increase the receptive field, thereby speeding up the model's information processing speed. After block segmentation and linear projection, the obtained information is concatenated into a vector, the corresponding position code is added, and then fed into the encoding layer of the model. The ability to extract tracking target features is enhanced by using a global multi-head attention module, effectively capturing complex features in the image. After the above processing, the obtained features are sent to the early candidate elimination module to delete unimportant information to increase the tracking speed of the model; Among them, the information of early candidate outputs is fed into a multi-layer perceptron to further transform the data and enhance the nonlinear expression ability of the model; and the output of the multi-layer perceptron is residually connected with the output of the global multi-head attention module to facilitate model gradient flow and model training; The integrated features are padded with zeros to their original length and then fed into the prediction head of the model to perform tracking prediction on the target. 3. Build an anti-interference countermeasure mechanism: Build a generative adversarial image dehazing network to dehaze received foggy video images; Among them, by scoring the predicted target area of the tracking object, it is determined whether the tracking is interfered; Among them, when it is judged that the target is not interfered with, normal high-efficiency target tracking is used and the target tracking model is updated according to the tracking results; When the tracking is judged to be interfered with, the type of interference is determined by listing possible target areas; when there are multiple possible targets, it is judged as interference from similar objects; when there is only one possible target, it is judged as a change in the appearance of the tracking target; when there is no possible target, it is judged as occlusion of the tracking target; Among them, in order to address the interference phenomenon of similar objects, the model combines historical information to establish a corresponding relationship for each similar object to eliminate interference; Specifically, in response to changes in the appearance of the tracking target, the model is selectively updated to reduce the dilution of features by low-quality samples; Among them, when the tracking target is blocked, the model performs motion estimation to predict the possible position of the target; 4. Perform image preprocessing and data enhancement on the sample data to obtain a training set and a test set; Among them, the preprocessing uses operations such as 2x and 4x scaling, 90 degree, 180 degree, and 270 degree rotation, translation, and cropping of the image; This application uses the following image enhancement techniques: (1) Rotation and flipping: By randomly rotating and flipping the image (horizontally or vertically), the model can learn objects at different angles and directions; (2) Scaling and cropping: These operations can change the scale and size of the image, which helps the model recognize objects of different sizes. For example, the image can be scaled to different sizes or randomly cropped; (3) Image noise: Gaussian noise and salt and pepper noise are added to the image to perform data augmentation and improve the robustness and generalization ability of the model; 5. Train the parameters of the target tracking network using the training set, wherein the test set is used to test the performance of the target tracking model on unseen data.

[0010] The beneficial effects of adopting the above technical solution are: This application can be used for real-time target tracking in areas such as drone monitoring and video surveillance. It offers improved anti-interference capabilities and tracking adaptability for targets with changing appearance, interference from similar objects, and occlusions. It also offers superior tracking accuracy and faster tracking speed than conventional tracking algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This application provides a flowchart of the anti-interference target tracking algorithm; Figure 2 This is a network diagram of anti-interference target tracking provided by this application; Figure 3 This application provides a schematic diagram of the global channel multi-head attention mechanism module (GCMHAM) in the interference-resistant target tracking network; Figure 4 This is a schematic diagram of the operation of the anti-interference countermeasure mechanism of the anti-interference target tracking algorithm provided by this application; Figure 5 This is the target tracking result of the algorithm proposed in this application on the target tracking dataset OTB-100; Figure 6 This is the target tracking result of the algorithm proposed in this application on the target tracking dataset VOT2018. DETAILED DESCRIPTION

[0012] In order to facilitate a more straightforward further understanding and use of this application, the following is a clear and complete description of this application in conjunction with the accompanying drawings and examples to enhance the understanding of this application by technical personnel and users. This embodiment is only a partial illustrative embodiment of this application. Any individual or unit that obtains other embodiments of this application without creative work shall fall within the scope of the rights of this application and be protected.

[0013] Figure 1 A flowchart of an anti-interference target tracking algorithm is provided for this application.

[0014] Step 101 first selects a tracking target and initializes and trains the tracking model based on it; Figure 2 It is a schematic diagram of the tracking model structure constructed; Step 201 is to input an image pair into the network, including the initialization target area and the tracked image in the subsequent tracking process; Step 202 is the preprocessing of the image before inputting it into the network, including slicing the image, linearly projecting it into a vector, and adding position encoding to obtain the final input vector. 203 is the encoder of the network, which includes the normalization layer (Layer Norm), the global channel multi-head attention module (GCMHAM), the early candidate elimination module (Early Candidate Elimination), and the multi-layer perceptron (MLP); 204 is to perform token padding on the output result vector to ensure that the length of the input and output vectors are equal; then the vector is sent to the prediction head of the model; In step 205 , the target area in the current tracking image is predicted according to the input vector to obtain a tracking result.

[0015] Step 102 calculates the confidence score of the possible area of the prediction result during the tracking process to obtain a probability score that the area may be the tracking target.

[0016] Step 103 determines whether the current tracking is interfered with, and the determination is made based on the number of possible areas currently existing.

[0017] Step 104 is when the model determines that the current tracking is interfered with, and eliminates the interference effect according to the type of interference; the detailed elimination step diagram is as follows: Figure 4 As shown; 401 is the coping strategy when there are multiple possible regions, that is, when similar objects interfere. The algorithm lists the current possible regions as a candidate region set and associates and scores them with the regions in the historical information to obtain the probability of each region being the tracking target. Among them, the corresponding similarity scores are obtained by associating each region in the historical information with the candidate regions in the candidate region set. When the similarity score of the current candidate region to the region other than the target region in a certain historical information is higher than that of the historical target region, it is judged as an interference object, otherwise it is judged as a tracking target. This process is expressed in the formula (1): Where: A subsequent region is The candidate region set is s t With s t-1 are the current region similarity score and the historical region similarity score, respectively. c The score threshold is used to measure the degree of change in the target score; When the candidate region set list is empty and no candidate region satisfies the condition to be determined as a tracking target (401), the algorithm uses a motion estimation method to calculate the possible motion trajectory of the target based on several historical frames. The actual position of the target in the previous frame is used as the base position, and the bounding box is moved to obtain the predicted bounding box. The detailed calculation process of the predicted bounding box is shown in Equation (2). Where: A1 and A2 are hyperparameter coefficient weights, which are used to adjust the impact of different frames on the overall result in motion estimation.

[0018] Step 105 is the model determining that the current tracking is not disturbed and continuing normal efficient tracking.

[0019] Step 106 is to predict the current position of the target when interference has not occurred or has been eliminated, and update the model according to the prediction result.

[0020] Step 107 is that after the prediction result is obtained, the model will feed back the tracking result to the computer in the form of generating a region border and display it.

[0021] The following combination Figure 2The specific implementations described herein are only used to explain the related applications and are not intended to limit the application. It should also be noted that, for ease of description, only the parts related to the present application are shown in the accompanying drawings.

[0022] The server configuration is a computer with a 12th Gen Intel(R) Core(TM) i7-12700H CPU, 2.30GHz, and 32GB of RAM. The test platform is Windows 11 Home Edition and Python 3.8.

[0023] The input of the network is a pair of images (201), namely the initialization sample image patch and the current search area image.

[0024] The image is segmented and flattened into a bag sequence (202), which is then projected into a latent space using a trainable linear projection layer with parameters; the learnable position embedding is added to the bag embedding of the template and the search region to produce the final template label embedding and search region label embedding and fed into the Transformer encoder layer;

[0025] In the encoder layer (203), the input vector first passes through the normalization layer and then is sent to the GCMHAM module. The detailed structure of the module is as follows: Figure 3 .

[0026] In GCMHAM, the split input tensor is subjected to maximum pooling (Max pooling) and average pooling (Average pooling) to obtain a tensor of shape (301); these two vectors are multiplied with the Values vector obtained in step 303 after passing through the SoftMax activation function.

[0027] The result vector obtained in step 301 is restored to its original length through linear transformation, and then normalized by Sigmoid operation and converted into a global channel coefficient matrix (302), as shown in formula (3): y=linear(Concat(V max ,V Avg )).sigmoid (3)

[0028] At the same time as step 301, the original tensor is split into three tensors (303): Query, Keys, and Values. The obtained Query and Keys vectors are multiplied by the scale. The result is multiplied by the Values tensor after the Softmax and Dropout operations to obtain the output of the position. The calculation process is shown in formula (4): z'=dropout(softmax((Query'·Keys'.transpose(-2,-1))*scale))·Values' (4)

[0029] The result obtained in 303 is linearly transformed to restore the original shape for subsequent operations (304).

[0030] Finally, the Hadamard product of the results obtained in 302 and 304 is calculated and passed through the Dropout layer to obtain the final output (305). The calculation process is shown in formula (5):

[0031] Continuing with step 202, the output of the GCMHAM is residually connected to the original input and then fed into the network's Early Candidate Elimination module. This module allows each block area in each search area image to independently calculate a similarity with the sample image as its score. The candidate areas with the highest scores are retained, and the remaining candidate areas are discarded. The output is then fed into a normalization layer, and the result is fed into a multilayer perceptron (MLP) for residual connection to obtain the final output of the encoder.

[0032] At the head of the network (204), the network reinterprets the zero-padded search region label sequence after candidate elimination into a 2D spatial feature map, which is then input into a fully convolutional network consisting of several stacked Conv-BN-ReLU layers. The output contains the target classification score map and a local offset to compensate for the discretization error caused by the resolution reduction and the normalized bounding box size.

[0033] The position with the highest classification score in step 204 is the target position (205).

[0034] To demonstrate the tracking effect of the improved model and verify the effectiveness of the specific improvement measures, this application uses the proposed algorithm to compare with other mainstream tracking algorithms on the OTB-100 dataset. The overall results of the experiment are shown in Table 1: Table 1: OTB-100 dataset test results The experimental results are shown in Table 1. The results show that the proposed algorithm outperforms current mainstream algorithms on the OTB-100 dataset in terms of the following metrics: AUC (Area under the curve): the area under the curve of the intersection-of-union (IoU) ratio of the pixels within the predicted box and the target's true bounding box as the threshold representing successful tracking changes; OP (Overlap precision): the overlap ratio, i.e., the IoU ratio of the pixels within the predicted box and the target's true bounding box, where OP50 and OP75 represent the success rates when the threshold representing successful tracking is set to 0.5 and 0.75, respectively (i.e., the predicted box overlaps with the target's true bounding box by more than 50% and by more than 75%, respectively); Precision: the average Euclidean distance between the predicted box and the center of the target's true bounding box, with a threshold of 20 (i.e., a center distance within 20 is considered successful); and Norm Precision: the normalized precision, which calculates the ratio of the Euclidean distance between the predicted box and the center of the target's true bounding box to the hypotenuse of the target's true bounding box, taking into account the scale of the target box. Compared to the OSTrack algorithm, the proposed algorithm has improved in all metrics.

[0035] In order to evaluate the generalization ability of the algorithm, this application also uses the VOT2018 target tracking dataset for experiments. The results are shown in Table 2. The algorithm of this application is generally better than other mainstream algorithms on the VOT2018 dataset. Table 2: VOT2018 dataset test results

[0036] To prove the detection effect of the patent model, the experimental results of the patent algorithm and the classic target tracking algorithm OSTrack are shown. Figure 5 、 Figure 6 As shown in Figure 2, the two algorithms are used for target tracking in the OTB-100 dataset and the VOT2018 dataset. Figure 5 、 Figure 6 As shown, during the tracking process, when faced with some occlusion phenomena and interference from similar objects, the algorithm of the present application can better avoid these interferences and maintain normal tracking.

[0037] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A target tracking algorithm, characterized by: It can achieve efficient and accurate anti-interference target tracking; First, real-time video images are collected through various shooting devices; the target area is selected and constructed based on the collected video information; the network model is initialized based on the target area samples; the target is continuously tracked and the training network model is updated; and it is determined whether the tracking process is disturbed; Based on the type of interference, an interference response strategy is used to eliminate the interference. If the target cannot be tracked normally, motion estimation is used to predict the target position. The tracking results are returned and displayed on the computer. The specific steps of this method are as follows: S1: Acquire video images; S2: Obtain sample data and select tracking targets based on video images; S3: Build and initialize the target tracking network; S4: target tracking and real-time updating of network model; S5: Determine whether the current tracking is interfered with according to the algorithm, and use the corresponding anti-interference strategy according to the interference situation. Eliminate interference effects; S6: Motion estimation of targets that cannot be tracked normally; S7: The system returns the target tracking result after eliminating the interference in real time.

2. A target tracking algorithm according to claim 2, characterized in that: The step S3 comprises: 1) Inputting a sample of the target area of the video image into the target tracking network to extract the features of the target area; 2) Build a network model based on the initialization weights; 3) Use the above target area features as template features, save and adjust the model parameters.

3. A target tracking algorithm according to claim 2, characterized in that: Wherein said step S5 comprises: 1) Generate corresponding likelihood scores for candidate regions within the current search area; 2) Enumerate candidate regions based on the above scores, build and initialize the target tracking network; 3) Establishing the association between the candidate regions in the candidate region set and the regions in the historical information to determine the current interference type; 4) Based on the above judgment, select candidate areas and eliminate the corresponding interference effects.

4. A target tracking algorithm according to claim 2, characterized in that: Wherein said step S6 comprises: 1) Record the location information of the prediction results in the historical information; 2) Calculate the movement direction and speed trend of the tracked target based on the above historical information; 3) Estimate the possible motion state of the currently lost target and predict the current target position based on the motion direction and motion speed trend of the above-mentioned tracked target.

5. A target tracking algorithm according to claim 3, characterized in that: The target tracking network is constructed and initialized in step S3, including: This application extracts target features: 1) Image segmentation: By dividing the image into small blocks, the attention module is able to focus on each area of the image, improving the model's global vision; 2) Combining sample images and search area images: By combining sample images and search area images, a single-stream Transformer model is implemented to enhance model computation speed and efficiency. 3) Write position encoding: By adding position encoding to the combined input image tensor, the model is provided with position information of different blocks for subsequent operations; The target tracking network coding layer constructed in this application: 1) Normalization layer: By standardizing the input or output of each layer, it stabilizes training, accelerates convergence, and alleviates gradient problems; 2) Global Channel Multi-Head Attention Module: This module splits the input tensor into query, key, and value multi-head vectors and performs global maximum pooling, global average pooling, linear transformation, dropout, softmax, sigmoid, and Hadamard score to extract comprehensive, multi-angle, and detailed features. 3) Residual connections: Residual connections add the input of a layer directly to its output, avoiding the vanishing gradient problem in deep networks and enabling the model to train deeper architectures more efficiently. 4) Early Candidate Elimination Module: This module can remove useless feature information, effectively shortening the time for model training and forward prediction; 5) Multilayer Perceptron: Performs nonlinear transformation and dimensionality adjustment on the features output by self-attention, enhancing the model's expressiveness and stabilizing training together with residual connections. The target tracking network head constructed in this application: 1) Split the feature vector: Split the sample image and search area image features in the feature vector for subsequent operations; 2) Token padding and reshaping: Filling in the parts deleted in the early candidate elimination module to ensure the same length of input and output tensors; 3) Target prediction: The target is predicted based on the above tensor information to obtain the plane coordinate information of the target area.

6. A target tracking algorithm according to claim 3, characterized in that: The anti-interference strategy includes: 1) Generate scores for prediction results: Make predictions based on sample templates and generate similarity scores; 2) Generate scores for other candidate regions and summarize them into a candidate region set: Use the adaptive candidate threshold to filter candidate regions with higher scores to form a candidate region set; 3) Eliminate similar objects: Combine historical information to establish an association between each candidate region set and the regions in the historical information. Candidate regions that match non-target regions are judged as interferers and deleted; 4) Return the candidate area after the above process.

7. A target tracking algorithm according to claim 5, characterized in that: The motion estimation S6 includes: 1) Get historical target location information: Based on the saved historical target location information, get the past target including coordinates and and area size and other information for subsequent use; 2) Calculate the motion state based on historical information; 3) Get and return the motion estimation result.

Images