Uniflow Transform tracking method based on direction attention

By introducing directional information and designing a directional attention mechanism in the single-stream Transformer tracking framework, the problem of target tracking accuracy being disturbed in complex scenarios is solved, achieving higher tracking accuracy and robustness.

CN121883531APending Publication Date: 2026-04-17YUNNAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
YUNNAN UNIV
Filing Date
2025-04-03
Publication Date
2026-04-17

AI Technical Summary

Technical Problem

Existing single-stream Transformer trackers fail to distinguish the target's motion direction in complex scenes, resulting in tracking accuracy being affected by low-probability similar information in the motion direction and cluttered backgrounds, thus impacting target position determination.

Method used

Introducing directional information into the single-stream Transformer tracking framework and designing a directional attention mechanism, the search region features are weighted saliency by calculating the correlation between search region features and directional information to focus on information in the direction of target motion and suppress interference from other directions.

Benefits of technology

It improves the accuracy and robustness of target tracking, especially in long-term, large-scale tracking datasets and various complex environments, with both success rate and accuracy improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121883531A_ABST
    Figure CN121883531A_ABST
Patent Text Reader

Abstract

The invention discloses a single flow Transform tracking method based on direction attention. According to the method, direction information in a search area is introduced and a direction attention mechanism is designed based on a single-flow Transform tracking framework. Compared with a method for uniformly treating all direction information, the direction attention mechanism enables the tracker to pay more attention to the information in the target movement direction in the search area when searching the target, and the accuracy of target tracking and positioning can be improved while information interference in other directions, namely the low-probability movement direction, is inhibited.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to a video target tracking method, and more particularly to a single-stream Transformer tracking method based on directional attention. This method introduces directional information and designs directional attention on the basis of the single-stream Transformer tracking framework, thereby improving tracking accuracy. Background Technology

[0002] Video object tracking (VOT) is a core technology in computer vision, tasked with predicting the position of a target in a subsequent video sequence, given its initial position. This technology has broad application prospects in video surveillance, autonomous driving, and robotics. However, in practical applications, object tracking faces numerous challenges, including background interference, partial occlusion, interference from similar objects, and changes in the target's appearance. These factors make it difficult for existing trackers to meet practical requirements in terms of accuracy and robustness.

[0003] Single-stream Transformer tracking is currently the most mainstream target tracking framework. It achieves unified feature extraction and fusion of the target template and the search region based on the VisionTransformer network. The search region is typically generated by expanding the target's scale by a certain factor and centering it on the target's position in the previous frame. Currently, existing single-stream Transformer trackers typically treat all possible motion directions within the search region indiscriminately when searching for a target. However, in complex real-world scenes, low-probability motion direction similarities and cluttered backgrounds within the search region can interfere with target location determination.

[0004] The problem and drawback of single-stream Transformer trackers, such as OSTrack, is that when searching for a target in a search area, they typically process information from all directions within that area indiscriminately. However, in complex real-world scenarios, low-probability similarity in motion directions and cluttered backgrounds can interfere with target location determination. By incorporating target motion direction information and paying more attention to this information within the search area, we can suppress interference from low-probability similarity or cluttered backgrounds, thereby improving tracking accuracy.

[0005] Taking the OSTrack tracker as an example, we conducted experiments to verify the unified processing of information from all directions within the search area. For example... Figure 1As shown, the first row represents the target's location within the search area, and the second row is a heatmap of the OSTrack tracker's tracking results. The bright yellow areas represent the target's localization. For the Cheetah sequence, the presence of a cluttered background within the search area causes interference from low-probability motion direction information below the search area, resulting in the highest-scoring position in the heatmap shifting to the background area below, leading to tracking errors. For the Skydiving sequence, interference from multiple similar targets in the scene causes the highest-scoring position in the heatmap to shift to a similar target on the right. For the Parkour sequence, the tracked target moves rapidly; the lack of target motion direction information results in the true target position having similar scores to adjacent positions, making accurate target locking difficult.

[0006] The above analysis shows that in real-world target tracking systems in complex scenarios, because both the target and the background are constantly changing, uniformly processing information from all directions within the search area will inevitably be affected by interference from information in low-probability motion directions, thus impacting the tracker's accuracy. Therefore, this invention, based on a single-stream Transformer tracking framework, introduces directional information from the search area and designs a directional attention mechanism. This directional attention mechanism allows the tracker to focus more on information in the target's motion direction within the search area when searching for a target, thereby suppressing interference from information in other directions (i.e., low-probability motion directions) and improving the accuracy of target tracking and localization. Summary of the Invention

[0007] To overcome the aforementioned shortcomings of existing technologies, this invention proposes a single-stream Transformer tracking method based on directional attention. This method, based on the single-stream Transformer tracking framework, introduces directional information within the search region and designs a directional attention mechanism. Compared to methods that uniformly treat all directional information, this directional attention mechanism allows the tracker to focus more on information related to the target's motion direction within the search region when searching for the target. This suppresses interference from information in other directions, i.e., low-probability motion directions, while improving the accuracy of target tracking and localization.

[0008] A single-stream Transformer tracking method based on directional attention consists of the following steps:

[0009] Step 1: Input the first frame of the video

[0010] Step 2: Generate the target template token sequence

[0011] Step 3: Input the next frame and generate the search region image.

[0012] Input a new video frame as the current frame image, and use the tracking result of the previous frame. Enlarge the rectangular region b 2The magnified region is used as the search region image. ,in and These represent the height and width of the search region image, respectively. , This indicates the number of channels in the image.

[0013] Step 4: Generate the search area token sequence

[0014] right The image is divided into blocks, each with a pixel resolution of [missing value]. It can form a sequence of image patches. ,in Indicates the number of image patches in the search area. This indicates the token embedding dimension. After linear projection and add position embedding It can generate search area token sequences. ,Right now .

[0015] Step 5: Generate a direction information token sequence

[0016] (1) Extracting the direction information of the search area

[0017] The target within the current frame's search area may move in eight directions: up, down, left, right, upper left, upper right, lower left, and lower right, or remain stationary, resulting in a total of nine possible directions. (The value is...) The child window, in Slide from left to right and from top to bottom, with horizontal and vertical sliding step sizes of respectively. and Each sliding position covers information from one direction for the image patch, thus allowing the extraction of information from nine directions within the search area. The pixel resolution of the image patch corresponding to each direction is... It can form a sequence of directional information. Where 9 represents the number of directional information. This indicates the dimension of directional information.

[0018] (2) Generate token sequence

[0019] Using two-dimensional convolution operations Can Information dimension transformation to search area token embedding dimension Standardize the scale. Then perform linear projection. and add position embedding It can generate a sequence of direction information tokens. ,Right now .

[0020] Step Six: Feature Extraction and Fusion

[0021] target template token sequence Search area token sequence Directional information token sequence Connect into a token sequence .Will The data is fed into the Visual Transformer (ViT) backbone network for feature extraction and fusion. The ViT network uses a conventional construction and processing method, comprising 12 encoder layers. Each encoder layer mainly consists of multi-head self-attention and a multilayer perceptron. After ViT processing, the target template, search region, and orientation information tokens undergo full attention interaction, and its output is... .in , , These represent the output target template feature sequence, search region feature sequence, and direction information feature sequence, respectively.

[0022] Step 7: Directional Attention Weighting Processing

[0023] The ViT full-attention interaction process treats search region features uniformly across all directions, meaning that search region features lack directional discrimination. To make the search region features more focused on features along the target's motion direction, two steps are required: calculating directional attention and weighting for saliency.

[0024] (1) Calculate directional attention

[0025] Calculate the feature sequence of the search region and directional information feature sequence The correlation between them can be used to obtain a correlation matrix. The calculation method is as follows: ,in This indicates the transpose operation. Each element Indicates the first The features of the first search region and the first The degree of correlation between information features in each direction.

[0026] Since each directional information feature learns target template information during the ViT processing, the correlation between search region features and all directional information features can be analyzed. Learn the correlation between the features of the search area and the direction of target movement The calculation method is as follows: ,in This indicates a global average pooling operation. Each original Indicates the first The degree of correlation between the features of each search region and the target's motion direction. Then, it is processed through a sigmoid activation function. Directional attention can be obtained The overall calculation process can be expressed as follows: .

[0027] (2) Significance weighting

[0028] Use directional attention The feature sequence of the search region can be saliency-weighted, making it focus more on features along the target's motion direction while suppressing features in non-motion directions. The saliency-weighting method is as follows: ,in This is the weighted feature sequence of the search region.

[0029] Step 8: Determine the tracking results

[0030] Based on the weighted search region feature sequence Target localization can be achieved. This localization process is a standard method in the target localization stage of single-stream tracking, mainly including reinterpreting the feature sequence of the search region into a two-dimensional feature map, passing it through a multi-layered fully convolutional network with a Conv-BN-ReLU structure, and determining the target position based on information such as the classification score map and local offsets. The total training loss of the tracking model is... Among them, classification loss Weighed Focal Loss was used for classification, and L1 loss was used for regression. and Generalized IoU loss Perform bounding box regression. The target localization result is used as the tracking result for the current frame. .

[0031] Step 9: Determine the last frame

[0032] If the current frame is the last frame, the tracking ends; otherwise, proceed to step three.

[0033] Furthermore, step one specifically involves: inputting the first frame of the video.

[0034] A video sequence consists of several frames, each containing at least one target. The target's position in the first frame is known, while the target's position in each of the remaining frames is unknown. The number of video frames is a positive integer greater than 1. The top-left corner of each frame in the video frame sequence is the origin. The width and height are respectively and Input the first frame image and automatically or manually mark the rectangular area of ​​the tracked target. That is, the selected tracking target. This represents the coordinates of the top-left corner of the rectangular region. These represent the width and height of the rectangular region, respectively. The target selected in the first frame is also called the tracking result of the current frame. = The subscript indicates the current frame number.

[0035] Furthermore, step two specifically involves: generating a target template token sequence.

[0036] Zoom in on the rectangular area of ​​the selected target in the first frame. 2 The magnified region is used as the target template image. ,in and These represent the height and width of the target template image, respectively. , This indicates the number of channels in the image.

[0037] right The image is divided into blocks, each with a pixel resolution of [missing value]. It can form a sequence of image patches. ,in Indicates the number of target template image blocks. This indicates the token embedding dimension. After linear projection and add position embedding It can generate target template token sequence ,Right now .

[0038] The technical effects and advantages of this invention are as follows:

[0039] In this invention, the directional information of the search region image is introduced into the input information of the backbone network. After the backbone network outputs features, in order to make the search region features pay more attention to the features in the direction of target motion, directional attention is designed based on the search region feature sequence and the directional information feature sequence. The search region feature sequence is weighted significantly, and the target localization can be achieved based on the weighted search region feature sequence. (1) Tracking effect under the long-term large-scale tracking dataset LaSOT

[0040] The success rate (left) and accuracy (right) curves on the LaSOT dataset demonstrate that our invention performs better. In particular, compared to OSTrack, the success rate is improved by 0.2% and the accuracy by 0.3%. This finding indicates that our invention (Ours) can better handle changes in the target state in long-term tracking scenarios.

[0041] (2) Tracking performance under various complex environments

[0042] In the UAV123 dataset, under 12 challenging environments, this invention ranked first in success rate (left) in 10 challenging environments and second in 2. In terms of tracking accuracy (right), it ranked first in tracking accuracy in 9 challenging environments and second in 3. Specifically, this invention achieved an accuracy of 91.3% in the Camera Motion environment and 89.7% in the Viewpoint Change environment. Furthermore, its tracking accuracy exceeded 85% in challenging environments such as Partial Occlusion and Scale Variation. In contrast, OSTrack only ranked first in success rate in two challenging environments and first in accuracy in only three. This demonstrates that this invention can better handle a variety of complex environments. Attached Figure Description

[0043] Figure 1 OSTrack tracker tracing example demonstration;

[0044] Figure 2 : Schematic diagram of the tracking method of the present invention;

[0045] Figure 3 Tracking performance on the long-term, large-scale tracking dataset LaSOT;

[0046] Figure 4 Tracking performance under 12 challenging environments in the UAV123 dataset;

[0047] Figure 5 : A schematic diagram of the network model of this invention;

[0048] Figure 6 : Schematic diagram of directional attention weighting in this invention;

[0049] Figure 7 The selection of the tracking region in the first frame of the parkour sequence;

[0050] Figure 8 Partial tracking results; Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] like Figure 1-8As shown, according to the technical solution of this invention, the parkour video sequence in the NFS dataset is selected for tracking, which has three challenging attributes: fast movement, cluttered background, and interference from similar objects.

[0053] Step 1: Input the first frame of the video

[0054] like Figure 7 As shown, the parkour video sequence contains 57 frames; the top left corner of each frame is the origin. The width and height are respectively and Input the first frame image and mark the rectangular region of the tracked target. .in, This represents the coordinates of the top-left corner of the rectangular region. These represent the width and height of the rectangular region, respectively. The target selected in the first frame is also called the tracking result of the current frame. = The subscript 1 indicates that the current frame number is 1.

[0055] Step 2: Generate the target template token sequence

[0056] Enlarge the rectangular area of ​​the selected target in the first frame by 2. 2 The magnified region corresponds to an image size deformation of 128×128, which serves as the target template image. The height and width 3 indicates the number of channels in the image.

[0057] right The image is divided into blocks, each with a pixel resolution of [missing value]. It can form a sequence of image patches. ,in Indicates the number of target template image blocks. This indicates the token embedding dimension. After linear projection And add learnable one-dimensional position embeddings. It can generate target template token sequence ,Right now .

[0058] Step 3: Input the next frame and generate the search region image.

[0059] Input the second video frame as the current frame image, and use the tracking result of the previous frame, i.e., the first frame. Enlarge the rectangular area by 4 2 The magnified region corresponds to an image size deformation of 256×256, which is used as the search region image. The height and width 3 indicates the number of channels in the image.

[0060] Step 4: Generate the search area token sequence

[0061] right The image is divided into blocks, each with a pixel resolution of [missing value]. It can form a sequence of image patches. ,in Indicates the number of image patches in the search area. This indicates the token embedding dimension. After linear projection And add learnable one-dimensional position embeddings. It can generate search area token sequences. ,Right now .

[0062] Step 5: Generate a direction information token sequence

[0063] (1) Extracting the direction information of the search area

[0064] The target within the current frame's search area may move in eight directions: up, down, left, right, upper left, upper right, lower left, and lower right, or remain stationary, resulting in a total of nine possible directions. (The value is...) The child window, in The slider moves from left to right and top to bottom, with horizontal and vertical step sizes of 64 and 64 respectively. Each sliding position covers information from one direction for the image patch, thus extracting information from nine directions within the search area. The pixel resolution of the image patch corresponding to each direction is... It can form a sequence of directional information. Where 9 represents the number of directional information. This indicates the dimension of directional information.

[0065] (2) Generate token sequence

[0066] Using two-dimensional convolution operations Can Information dimension transformation to search area token embedding dimension Standardize the scale. Then perform linear projection. And add learnable one-dimensional position embeddings. It can generate a sequence of direction information tokens. ,Right now .

[0067] Step Six: Feature Extraction and Fusion

[0068] target template token sequence Search area token sequence Directional information token sequence Connect into a token sequence .Will The data is fed into the Visual Transformer (ViT) backbone network for feature extraction and fusion. The ViT network uses a conventional construction and processing method, comprising 12 encoder layers. Each encoder layer mainly consists of multi-head self-attention and a multilayer perceptron. After ViT processing, the target template, search region, and orientation information tokens undergo full attention interaction, and its output is... .in , , These represent the output target template feature sequence, search region feature sequence, and direction information feature sequence, respectively.

[0069] Step 7: Directional Attention Weighting Processing

[0070] The ViT full-attention interaction process treats search region features uniformly across all directions, meaning that search region features lack directional discrimination. To make the search region features more focused on features along the target's motion direction, two steps are required: calculating directional attention and weighting for saliency.

[0071] (1) Calculate directional attention

[0072] Calculate the feature sequence of the search region and directional information feature sequence The correlation between them can be used to obtain a correlation matrix. The calculation method is as follows: ,in This indicates the transpose operation. Each element Indicates the first The features of the first search region and the first The degree of correlation between information features in each direction.

[0073] Since each directional information feature learns target template information during the ViT processing, the correlation between search region features and all directional information features can be analyzed. Learn the correlation between the features of the search area and the direction of target movement The calculation method is as follows: ,in This indicates a global average pooling operation. Each original Indicates the first The degree of correlation between the features of each search region and the target's motion direction. Then, it is processed through a sigmoid activation function. Directional attention can be obtained The overall calculation process can be expressed as follows: .

[0074] (2) Significance weighting

[0075] Use directional attention The feature sequence of the search region can be saliency-weighted, making it focus more on features along the target's motion direction while suppressing features in non-motion directions. The saliency-weighting method is as follows: ,in This is the weighted feature sequence of the search region.

[0076] Step 8: Determine the tracking results

[0077] Based on the weighted search region feature sequence Target localization can be achieved. This localization process is a standard method in the target localization stage of single-stream tracking, mainly including reinterpreting the feature sequence of the search region into a two-dimensional feature map, passing it through a multi-layered fully convolutional network with a Conv-BN-ReLU structure, and determining the target position based on information such as the classification score map and local offsets. The total training loss of the tracking model is... Among them, classification loss Weighed Focal Loss was used for classification, and L1 loss was used for regression. and Generalized IoU loss Perform bounding box regression. The target localization result is used as the tracking result for the current second frame. .

[0078] Step 9: Determine the last frame

[0079] The current frame is frame 2, so proceed to step 3.

[0080] Finally, the examples of this invention were trained and tracked on an NVIDIA RTX 4090 GPU, and some tracking results are shown below. Figure 8 As shown in the figure, the green box indicates the actual location of the video marker, the red box indicates the tracking result of this invention, and the yellow label in the upper left corner is the video frame number.

[0081] The most similar prior art solution to this invention is the OSTrack tracking method (Ye, Botao, et al. "Joint feature learning and relation modeling for tracking: A one-stream framework." In ECCV, 2022.), which proposes a single-stream Transformer tracking framework. Moreover, since 2022, most target tracking methods in recent years have followed the framework structure proposed by OSTrack. Existing single-stream trackers, represented by OSTrack, generally consist of three parts: image pair input, backbone network, and prediction head. The image pair includes a target template and a search region image. The backbone network performs unified feature extraction and fusion on the image pair, and the search region features output by the backbone network are fed into the prediction head for target localization.

[0082] This invention also implements tracking based on a single-stream Transformer tracking framework. It differs from existing methods in two ways: first, it introduces directional information from the search region image during the image input stage; second, after the backbone network outputs features, to give more attention to the feature information related to the target's motion direction in the search region, it designs a directional attention mechanism based on the tokens corresponding to the directional information and the tokens corresponding to the search region. The weighted search region tokens are then fed into the prediction head for target localization.

[0083] (1) Tracking performance on the long-term, large-scale tracking dataset LaSOT

[0084] like Figure 3 As shown in the success rate (left) and accuracy (right) curves on the LaSOT dataset, our invention demonstrates superior performance. Specifically, compared to OSTrack, the success rate is improved by 0.2%, and the accuracy by 0.3%. This finding indicates that our invention (Ours) is better able to handle changes in target state during long-term tracking scenarios.

[0085] (2) Tracking performance under various complex environments

[0086] like Figure 4As shown, in the UAV123 dataset, under 12 challenging environments, this invention ranks first in success rate (left) in 10 challenging environments and second in 2. In terms of tracking accuracy (right), this invention ranks first in tracking accuracy in 9 challenging environments and second in 3. Specifically, this invention achieves 91.3% accuracy in the CameraMotion environment and 89.7% accuracy in the Viewpoint Change environment. Furthermore, its tracking accuracy exceeds 85% in challenging environments such as Partial Occlusion and Scale Variation. In contrast, OSTrack only ranks first in success rate in two challenging environments and first in accuracy in only three. This demonstrates that this invention can better handle various complex environments.

[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A single-stream Transformer tracking method based on directional attention, comprising the following steps: Step 1: Input the first frame of the video; Step 2: Generate the target template token sequence; Enlarging a rectangular region of the first frame selected target 2 The enlarged region corresponds to an image as a target template image wherein and respectively represent the height and width of the target template image, and , represents the number of channels of the image; To perform image block division, the pixel resolution of each image block is , and the image block sequence can be formed , where represents the number of target template image blocks, represents the token embedding dimension; After linear projection , the position embedding is added , and the target template token sequence , that is ; Step 3: Input the next frame and generate the search region image. Input a new video frame as the current frame image, and use the tracking result of the previous frame. Enlarge the rectangular region b 2 The magnified region is used as the search region image. ,in and These represent the height and width of the search region image, respectively. , Indicates the number of channels in an image; Step 4: Generate the search area token sequence right The image is divided into blocks, each with a pixel resolution of [missing value]. It can form a sequence of image patches. ,in Indicates the number of image patches in the search area. Indicates the token embedding dimension; After linear projection and add position embedding It can generate search area token sequences. ,Right now ; Step 5: Generate a direction information token sequence (1) Extracting the direction information of the search area The target within the current frame's search area may move in eight directions: up, down, left, right, upper left, upper right, lower left, and lower right, or remain stationary, resulting in a total of nine possible directions; (The last part, "sized," appears to be a typo and can be omitted.) The child window, in Slide from left to right and from top to bottom, with horizontal and vertical sliding step sizes of respectively. and Each sliding position covers information from one direction for the image patch, thus allowing the extraction of information from nine directions within the search area; the pixel resolution of each direction information corresponds to an image patch of [resolution value missing]. It can form a sequence of directional information. Where 9 represents the number of directional information. Indicates the dimension of directional information; (2) Generate token sequence Using two-dimensional convolution operations Can Information dimension transformation to search area token embedding dimension Standardize the scale; then perform linear projection. and add position embedding It can generate a sequence of direction information tokens. ,Right now ; Step Six: Feature Extraction and Fusion target template token sequence Search area token sequence Directional information token sequence Connect into a token sequence ;Will The data is fed into the Visual Transformer (ViT) backbone network for feature extraction and fusion. The ViT network uses a conventional construction and processing method, comprising 12 encoder layers. Each encoder layer mainly consists of multi-head self-attention and a multilayer perceptron. After ViT processing, the target template, search region, and orientation information tokens undergo full attention interaction, and the output is... ;in , , These represent the output target template feature sequence, search region feature sequence, and orientation information feature sequence, respectively. Step 7: Directional Attention Weighting Processing The ViT full attention interaction process treats search region features uniformly across all directions, meaning that search region features have no directional discrimination. To make the search region features pay more attention to features in the direction of target motion, two steps are required: calculating directional attention and saliency weighting. (1) Calculate directional attention Calculate the feature sequence of the search region and directional information feature sequence The correlation between them can be used to obtain a correlation matrix. The calculation method is as follows: ,in Indicates the transpose operation; Each element Indicates the first The features of the first search region and the first The degree of correlation between information features in each direction; Since each directional information feature learns target template information during the ViT processing, the correlation between search region features and all directional information features can be analyzed. Learn the correlation between the features of the search area and the direction of target movement The calculation method is as follows: ,in This indicates a global average pooling operation; Each original Indicates the first The degree of correlation between the features of each search region and the direction of target motion; then, after passing through the sigmoid activation function. Directional attention can be obtained The overall calculation process can be expressed as follows: ; (2) Significance weighting Use directional attention The feature sequence of the search region can be saliency-weighted to focus more on features along the target's motion direction while suppressing features in non-motion directions; the saliency-weighting method is as follows: ,in The weighted feature sequence of the search region; Step 8: Determine the tracking results Based on the weighted search region feature sequence Target localization can be achieved; this localization process is a standard method for target localization in single-stream tracking, mainly including reinterpreting the feature sequence of the search region into a two-dimensional feature map, passing it through a multi-layered fully convolutional network with a Conv-BN-ReLU structure, and determining the target position based on information such as the classification score map and local offset; the total training loss of the tracking model is... Among them, classification loss Weighed Focal Loss was used for classification, and L1 loss was used for regression. and Generalized IoU loss Perform bounding box regression; the target localization result is used as the tracking result for the current frame. ; Step 9: Determine the last frame If the current frame is the last frame, the tracking ends; otherwise, proceed to step three.

2. The method according to claim 1, characterized in that, Step one specifically involves: The first frame of the input video: A video sequence contains several frames, each containing at least one target. The target's position in the first frame is known, while the target's position in each of the remaining frames is unknown. The number of video frames is a positive integer greater than 1. The top-left corner of each frame in the video frame sequence is the origin. The width and height are respectively and Input the first frame image and automatically or manually mark the rectangular area of ​​the tracked target. That is, the selected tracking target; among which, This represents the coordinates of the top-left corner of the rectangular region. These represent the width and height of the rectangular region, respectively; the target selected in the first frame is also called the tracking result of the current frame. = The subscript indicates the current frame number.

3. The method according to claim 1, characterized in that, Step two specifically involves: generating the target template token sequence. The rectangular region of the selected target in the first frame is enlarged by a2 times, and the image corresponding to the enlarged region is used as the target template image, where and represent the height and width of the target template image, respectively, and represents the number of channels of the image; The image is divided into blocks, each with a pixel resolution of , which forms an image block sequence, where represents the number of target template image blocks and represents the token embedding dimension. After linear projection and the addition of positional embedding, the target template token sequence can be generated, i.e.