Single target tracking method based on aggregation token perception characteristics

Abstract

The invention provides a single target tracking method based on aggregation token perception features, and particularly relates to the technical field of vision and image processing. A single-target tracking method based on aggregation token sensing features comprises the following steps: step 1, under single-target tracking based on the aggregation token sensing features, constructing an overall model composed of a ViT-B backbone network, an aggregation token sensing feature module, a multi-scale feature fusion module and a positioning head; 2, initialization: giving a data set X including N video sequences, selecting a first frame of image and an initial bounding box position of a target, using a target detection model of a pre-trained ViT-B backbone network or a manually labeled bounding box to initialize a tracking model, feature extraction and enhancement process, obtaining template key information through an aggregation token perception feature module, and obtaining template key information; and utilizing a deep neural network to accurately extract the feature representation of the current frame image and the target area, and inputting the cut target area into a feature extraction network.

Description

Technical Field

[0001] The present invention mainly relates to the field of vision and image processing technology, and specifically to a single target tracking method based on aggregated token perception features. Background Art

[0002] Single target tracking is an important task in the field of computer vision. It is widely used in video surveillance, autonomous driving, intelligent transportation and other fields. Researching efficient single target tracking networks has important theoretical and practical significance for promoting social production and daily life. The tracking performance is affected by many factors, such as illumination, occlusion, size change and complex background interference. Traditional methods use convolutional neural networks to extract and match features, but there is a problem that global and local features cannot be well integrated, resulting in the weakening of subtle features. Vision Transformer has gradually become the mainstream hotspot for single target tracking tasks due to its advantages in capturing long-range dependencies and self-attention mechanisms. However, this model usually contains a large number of parameters and computational overhead, and simple pruning will cause performance loss, which poses a challenge to single target tracking tasks with high real-time requirements. Therefore, the research on lightweight single target tracking networks has become an important direction.

[0003] At present, there are still some problems in the performance of lightweight single target tracking networks, mainly in the following two aspects:

[0004] (1) Balance between accuracy and speed in lightweight networks: In lightweight single-target tracking networks, there are some deficiencies in the balance between accuracy and speed. This is because in the process of designing lightweight models, it is often necessary to make a trade-off between model running speed and tracking accuracy. Some lightweight models may sacrifice a certain degree of accuracy in pursuit of faster speed, resulting in a decrease in accuracy when tracking the target.

[0005] (2) Robustness in complex scenes: In lightweight single-target tracking networks, facing the challenges of complex scenes, such as drastic changes in illumination and complex and changing backgrounds, their robustness has obvious defects. This deficiency may cause the tracking network to be unstable or even fail when dealing with complex environments.

[0006] In summary, the present invention provides a single target tracking method based on aggregated token perception features. Summary of the invention

[0007] In order to solve the problems in the prior art, the present invention provides a single target tracking method based on aggregated token perception features.

[0008] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0009] A single target tracking method based on aggregate token perception features, characterized in that the method comprises the following steps:

[0010] Step 1: Under single target tracking based on aggregated token-aware features, an overall model consisting of the ViT-B backbone network, aggregated token-aware feature module, multi-scale feature fusion module and positioning head is constructed.

[0011] Step 2: Initialization: Given a dataset X containing N video sequences, select the first frame image and the initial bounding box position of the target, and use the target detection model of the pre-trained ViT-B backbone network or the manually annotated bounding box to initialize the tracking model.

[0012] Step 3: Feature extraction and enhancement process, obtain the key information of the template through the aggregate token perception feature module, use the deep neural network to accurately extract the feature representation of the current frame image and the target area, and input the cropped target area into the feature extraction network. The network will pass through multiple attention layers in sequence, extracting from the bottom-level texture features to the high-level semantic features, and finally output a fixed-length feature vector as the feature representation of the target.

[0013] Step 4: Target positioning and tracking: Input the current frame features and the initial target features into the tracking network, repeat the above feature extraction steps, continue to track the target through the trained model processing video sequence in subsequent frame images, and finally output the video sequence with the target position marked.

[0014] Step 5: Model training and performance evaluation: Model training requires training data sets and test data sets, and evaluating model performance requires multiple evaluation indicators.

[0015] In step 3, the aggregated token perception feature module can effectively aggregate the original representation information of the image for perception features, where the process of aggregating the image can be summarized as follows:

[0016] Row avg =avg(Z 11 ,Z 12 , Z 13 ...Z 1n )

[0017] Column_avg=avg(Z 11 , Z 21 , Z 31 ...Z n1 )

[0018] Z′=Z-Row_avg+Z-Column_avg

[0019] Where Z ijrepresents the value of the i-th row and j-th column of the two-dimensional feature map of the original image, Row_avg represents the average value of each row of the two-dimensional feature map, column_avg represents the average value of each column of the two-dimensional feature map, avg(·) represents the average value, and Z′ represents the value finally input into VIT after the above operations, that is,

[0020]

[0021] By subtracting the average value of the template feature map, the information in the feature map can be aggregated and weighted, so that the model pays more attention to the target area.

[0022] In step 1, the multi-scale feature fusion module is used to make up for the shortcomings of insufficient feature extraction and capturing small objects or subtle changes in images. The multi-scale feature fusion module has four different layers, each of which contains convolutions of different numbers and sizes, so as to obtain richer receptive fields and deeper semantic information. The output features are passed into the multi-scale feature fusion module, and the output of each layer is fused through convolutions of different layers and sizes.

[0023] The above process can be expressed as:

[0024] D0=Z output

[0025]

[0026]

[0027]

[0028]

[0029] Where Z output Denotes the output of the template image after VIT, D i represents the output of the i-th layer in the module, and cat represents the concatenation operation.

[0030] After the above operation, the template features have deeper semantic information. In order to retain this information, we finally perform residual connection between the results obtained in each layer and the original feature D0, and perform element-by-element addition to obtain the final result D output :

[0031] D output =D0+D1+D2+D3+D4

[0032] When the entire network model performs attention calculation, the feature extraction method of the template image Z remains unchanged;

[0033] When performing attention calculation for the search image X, we continue the asynchronous feature extraction method and perform separate feature extraction on the search image X for the first 5 layers. The formula is as follows:

[0034]

[0035]

[0036] Where Q, K and V are the query matrix, key matrix and value matrix respectively. Subscripts Z and X denote the matrix entries belonging to the template and search region respectively. Z' denotes the output of the template image Z after passing through the aggregate token-aware module.

[0037] In the subsequent layers, the output D of the template image features after multi-scale feature fusion module is performed output After concatenation, attention is calculated, as shown below:

[0038]

[0039] Where Q, K and V are the query matrix, key matrix and value matrix respectively. The subscripts X and D represent the matrix items belonging to the search region and the output of the template image after passing through the multi-scale feature fusion module.

[0040] In step 3, corner point localization and center head localization in target tracking are two common prediction methods, which are used to determine the position and posture of the target in the image or video. Since the bounding box of corner point localization is much larger than the target, it generates too much redundancy and erroneous information. Therefore, we use the center head method for prediction here, that is, three convolution branches are used to represent center classification, offset regression and size regression respectively, and then the corresponding regression coordinates are used to calculate the bounding box, and the target position is determined as the final prediction result.

[0041] For the loss function, weighted focal loss is used for classification, and l1 loss and ordinary GIoU loss are combined for training. The final total loss function is expressed as follows:

[0042]

[0043] Among them, λ G =2 and It is a trade-off weight.

[0044] In the step 5, the training datasets: GOT-10K, a dataset containing 10,000 video clips, covering more than 560 categories of moving objects and 87 motion modes, with no overlap between most training and testing categories; TrackingNet: a large-scale target tracking dataset, the training set data includes 30,132 sequences, with an average of 471.4 frames per sequence; LaSOT: a large-scale, long sequence target tracking dataset, containing 1,140 training sequences, 280 test sequences, 1,400 sequences, totaling more than 3.5 million frames, with an average video length of 2,500 frames, and all sequences are long-term; COCO: the dataset has 328,000 images, containing 2.5 million annotated instances, created by using advanced user interfaces such as category detection, instance annotation, and instance segmentation;

[0045] When training the model, for the benchmark test of the GOT-10k dataset, we followed the one-time protocol and only used the training set of GOT-10k as the training set of the model, and trained the model for 100 epochs. For other benchmarks, we used TrackingNet, Lasot, GOT-10K and COCO training splits to train the model for 300 epochs.

[0046] The optimizer is AdamW optimizer with weight decay of 1e-4, the initial learning rates of the encoder and decoder are 4e-5 and 4e-4 respectively, and the batch size is set to 32.

[0047] This was implemented using PyTorch 2.2 in Python 3.11 on a 48G NVIDIA A40 graphics card, with each epoch training taking about 5 minutes.

[0048] The evaluation indicator area under the curve (AUC) is used to measure the overall performance of the tracking algorithm at different overlap thresholds. The calculation formula is as follows:

[0049]

[0050] Where SR is the success rate under different overlap thresholds, and α is the overlap threshold (IoU). A higher AUC value indicates that the algorithm performs better under various conditions.

[0051] Precision (P) measures the average deviation between the predicted position and the true target position, especially the deviation of the center position. The calculation formula is as follows:

[0052]

[0053] Where N is the total number of frames, C i is the center position of the prediction box of the i-th frame, is the center position of the ground truth frame of the i-th frame, is the Euclidean distance between two center points, and I is an indicator function that takes 1 when the distance is less than or equal to the threshold δ, otherwise it takes 0. High precision means that the prediction of the center position of the tracking target is accurate.

[0054] Normalized precision (P Norm ) is used to measure the tracking accuracy at different scales, and the standardization process eliminates the impact of scale differences between different videos;

[0055]

[0056] Among them, diag(G i ) is the diagonal length of the ground truth box of the i-th frame;

[0057] The average overlap (AO) measures the average overlap between the predicted box and the real box, reflecting the matching degree between the tracking result and the real target;

[0058]

[0059] Among them, B i is the predicted bounding box of the i-th frame, G i is the ground-truth bounding box of the i-th frame, IoU(B i , G i ) is the intersection-over-union ratio of the predicted box and the true box.

[0060] SR 0.5 and SR 0.75 They represent the success rates at IOU thresholds of 0.5 and 0.75, respectively, and are used to evaluate the performance of the tracking algorithm under different levels of strictness;

[0061]

[0062] Among them, α is the IoU threshold (0.5 or 0.75), I is the indicator function, which takes 1 when IoU is greater than or equal to the threshold, otherwise it is 0.

[0063] FPS measures the processing speed of the tracking algorithm, which is especially important in real-time applications;

[0064]

[0065] A high FPS value means the algorithm is able to process more frames in real time.

[0066] Compared with the prior art, the beneficial effects of the present invention are:

[0067] The present invention proposes a single target tracking method based on aggregated token perception features. Without increasing the network burden, the information in the image is highlighted by aggregated tokens. By calculating the key information of the template image, the aggregated tokens are obtained to replace the original template tokens, thereby enhancing the key information of the template. The multi-scale feature fusion module integrates deep and shallow semantic information, compensates for the loss caused by pruning operations, and enriches the image information.

[0068] A large number of experiments were carried out on three data sets, and the results proved the effectiveness and robustness of the single target tracking method based on aggregated token-aware features, and its advantages in various scenarios. The method of the present invention can still achieve fast and accurate tracking under conditions of target occlusion, target appearance change, etc. The single target tracking method provided by the present invention brings an efficient and lightweight solution to the field of target tracking through its innovative aggregated token-aware feature method and network structure, and has broad application prospects, especially in modern applications that require real-time processing of large amounts of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a network framework diagram of the present invention;

[0070] Figure 2 This is a framework diagram of the aggregated token sensing feature module of the present invention;

[0071] Figure 3 This is a framework diagram of the multi-scale feature fusion module of the present invention. DETAILED DESCRIPTION

[0072] The present invention will be further described with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art may make various changes or modifications to the present invention, and these equivalent forms also fall within the scope limited by the application.

[0073] A single target tracking method based on aggregated token perception features, the method comprising the following steps:

[0074] Step 1: Under single target tracking based on aggregated token-aware features, an overall model consisting of the ViT-B backbone network, aggregated token-aware feature module, multi-scale feature fusion module and positioning head is constructed.

[0075] Step 2: Initialization: Given a dataset X containing N video sequences, select the first frame image and the initial bounding box position of the target, and use the target detection model of the pre-trained ViT-B backbone network or the manually annotated bounding box to initialize the tracking model.

[0076] Step 3: Feature extraction and enhancement process, obtain the key information of the template through the aggregate token perception feature module, use the deep neural network to accurately extract the feature representation of the current frame image and the target area, and input the cropped target area into the feature extraction network. The network will pass through multiple attention layers in sequence, extracting from the bottom-level texture features to the high-level semantic features, and finally output a fixed-length feature vector as the feature representation of the target.

[0077] Step 4: Target positioning and tracking: Input the current frame features and the initial target features into the tracking network, repeat the above feature extraction steps, continue to track the target through the trained model processing video sequence in subsequent frame images, and finally output the video sequence with the target position marked.

[0078] Step 5: Model training and performance evaluation: Model training requires training data sets and test data sets, and evaluating model performance requires multiple evaluation indicators.

[0079] In step 3, the aggregated token perception feature module can effectively aggregate the original representation information of the image for perception features, where the process of aggregating the image can be summarized as follows:

[0080] Row avg =avg(Z 11 , Z 12 , Z 13 ...Z 1n )

[0081] Column_avg=avg(Z 11 , Z 21 , Z 31 ...Z n1 )

[0082] Z′=Z-Row_avg+Z-Column_avg

[0083] Where Z ij represents the value of the i-th row and j-th column of the two-dimensional feature map of the original image, Row_avg represents the average value of each row of the two-dimensional feature map, Column_avg represents the average value of each column of the two-dimensional feature map, avg(·) represents the average value, and Z′ represents the value finally input into VIT after the above operations, that is,

[0084]

[0085] Through the above operation of subtracting the average value of the template feature map, the information in the feature map can be aggregated and weighted, so that the model can pay more attention to the target area. The aggregation token perception feature module and the Cls_Token carrying the global features in VIT are spliced ​​to enable the model to better understand the graphics and better process images under different conditions.

[0086] In step 1, the multi-scale feature fusion module is used to make up for the shortcomings of insufficient feature extraction and capturing small targets or subtle changes in images. The multi-scale feature fusion module has four different layers, each of which contains convolutions of different numbers and sizes, so as to obtain richer receptive fields and deeper semantic information. The output features are passed into the multi-scale feature fusion module, and the output of each layer is fused through convolutions of different layers and sizes.

[0087] The above process can be expressed as:

[0088] D0=Z output

[0089]

[0090]

[0091]

[0092]

[0093] Where Z output Denotes the output of the template image after VIT, D i represents the output of the i-th layer in the module, and cat represents the concatenation operation.

[0094] After the above operation, the template features have deeper semantic information. In order to retain this information, we finally perform residual connection between the results obtained in each layer and the original feature D0, and perform element-by-element addition to obtain the final result D output :

[0095] D output =D0+D1+D2+D3+D4

[0096] When the entire network model performs attention calculations, the feature extraction method for the template image Z remains unchanged.

[0097] When performing attention calculation for the search image X, we continue the asynchronous feature extraction method and perform separate feature extraction on the search image X for the first 5 layers. The formula is as follows:

[0098]

[0099]

[0100] Where Q, K and V are the query matrix, key matrix and value matrix respectively. Subscripts Z and X denote the matrix entries belonging to the template and search region respectively. Z' denotes the output of the template image Z after passing through the aggregate token-aware module.

[0101] In the subsequent layers, the output D of the template image features after multi-scale feature fusion module is performed output After concatenation, attention is calculated, as shown below:

[0102]

[0103] Where Q, K and V are the query matrix, key matrix and value matrix respectively. The subscripts X and D represent the matrix items belonging to the search region and the output of the template image after passing through the multi-scale feature fusion module.

[0104] On the basis of the above two modules, in order to ensure the performance and practicality of the network, the two network branches have the characteristic of shared weights during the process of template image and search image feature extraction.

[0105] In step 3, corner point localization and center head localization in target tracking are two common prediction methods, which are used to determine the position and posture of the target in the image or video. Since the bounding box of corner point localization is much larger than the target, it generates too much redundancy and error information. Therefore, we use the center head method for prediction here, that is, three convolution branches are used to represent center classification, offset regression and size regression respectively, and then the corresponding regression coordinates are used to calculate the bounding box, and the target position is determined as the final prediction result.

[0106] For the loss function, weighted focal loss is used for classification, and l1 loss and ordinary GIoU loss are combined for training. The final total loss function is expressed as follows:

[0107]

[0108] Among them, λ G =2 and It is a trade-off weight.

[0109] In step 5, the training datasets are: GOT-10K, a dataset containing 10,000 video clips covering more than 560 categories of moving objects and 87 motion patterns. There is no overlap between most training and test categories. TrackingNet: a large-scale target tracking dataset. The training set data includes 30,132 sequences, with an average of 471.4 frames per sequence. LaSOT: a large-scale, long sequence target tracking dataset, containing 1,140 training sequences, 280 test sequences, 1,400 sequences, totaling more than 3.5 million frames, with an average video length of 2,500 frames. All sequences are long-term. COCO: the dataset has 328,000 images and contains 2.5 million annotated instances, created using advanced user interfaces such as category detection, instance annotation, and instance segmentation.

[0110] When training the model, for the benchmark test of the GOT-10k dataset, we followed the one-time protocol and only used the training set of GOT-10k as the training set of the model, and trained the model for 100 epochs. For other benchmarks, we used TrackingNet, Lasot, GOT-10K and COCO training splits to train the model for 300 epochs.

[0111] The optimizer is AdamW optimizer with weight decay of 1e-4, the initial learning rates of the encoder and decoder are 4e-5 and 4e-4 respectively, and the batch size is set to 32.

[0112] This was implemented using PyTorch 2.2 in Python 3.11 on a 48G NVIDIA A40 graphics card, with each epoch training taking about 5 minutes.

[0113] The evaluation indicator area under the curve (AUC) is used to measure the overall performance of the tracking algorithm at different overlap thresholds. The calculation formula is as follows:

[0114]

[0115] Where SR is the success rate under different overlap thresholds, and α is the overlap threshold (IoU). A higher AUC value indicates that the algorithm performs better under various conditions.

[0116] Precision (P) measures the average deviation between the predicted position and the true target position, especially the deviation of the center position. The calculation formula is as follows:

[0117]

[0118] Where N is the total number of frames, C i is the center position of the prediction box of the i-th frame, is the center position of the ground truth frame of the i-th frame, is the Euclidean distance between two center points, and I is an indicator function that takes 1 when the distance is less than or equal to the threshold δ, otherwise it takes 0. High precision means that the prediction of the center position of the tracking target is accurate.

[0119] Normalized precision (P Norm ) is used to measure the tracking accuracy at different scales, and the standardization process eliminates the impact of scale differences between different videos.

[0120]

[0121] Among them, diag(G i ) is the diagonal length of the ground truth box of the i-th frame;

[0122] The average overlap (AO) measures the average overlap between the predicted box and the true box, reflecting the matching degree between the tracking result and the true target.

[0123]

[0124] Among them, B i is the predicted bounding box of the i-th frame, G i is the ground-truth bounding box of the i-th frame, IoU(B i , G i ) is the intersection-over-union ratio of the predicted box and the true box.

[0125] SR 0.5 and SR 0.75 They represent the success rates at IOU thresholds of 0.5 and 0.75, respectively, and are used to evaluate the performance of tracking algorithms under different levels of strictness.

[0126]

[0127] Among them, α is the IoU threshold (0.5 or 0.75), I is the indicator function, which takes 1 when IoU is greater than or equal to the threshold, otherwise it is 0.

[0128] FPS measures the processing speed of the tracking algorithm and is especially important in real-time applications.

[0129]

[0130] A high FPS value means the algorithm is able to process more frames in real time.

[0131] Params is used to measure the number of parameters of the model, mainly including weights and bias parameters. The number of parameters reflects the storage requirements and training complexity of the model.

[0132] MACs is used to measure the amount of computation required by the model during a forward propagation process, especially the number of multiplication and accumulation operations. It reflects the computational complexity and operating efficiency of the model and is an important indicator for evaluating the inference speed of the model on different hardware devices.

[0133] Some classic and latest real-time and non-real-time algorithms in the field of single target tracking are experimentally compared on three datasets: Lasot, TrackingNet and Got10k. The results are shown in Table 1.

[0134] Table 1. Comparison of this method with other methods on Lasot, TrackingNet and GOT-10k datasets.

[0135]

[0136] The lightweight model is analyzed, tested and evaluated in terms of computational complexity (MACs), floating point operations (FLOPs), number of parameters (Params) and frames per second (FPS) on an Nvidia Tesla T4-16GB GPU compared with the most advanced non-real-time tracker. The detailed results are shown in Table 2.

[0137] Table 2. Comparison of the computational complexity (MACs), number of parameters (Params), and frames per second (FPS) of the model with existing non-real-time trackers.

[0138]

Claims

1. A single target tracking method based on aggregated token perception features, characterized in that: The method comprises the following steps: Step 1: Under single target tracking based on aggregated token-aware features, an overall model consisting of the ViT-B backbone network, aggregated token-aware feature module, multi-scale feature fusion module and localization head is constructed; Step 2: Initialization: Given a dataset X containing N video sequences, select the first frame image and the initial bounding box position of the target, and use the target detection model of the pre-trained ViT-B backbone network or the manually annotated bounding box to initialize the tracking model; Step 3: Feature extraction and enhancement process, the key information of the template is obtained through the aggregated token perception feature module, and the feature representation of the current frame image and the target area is accurately extracted using a deep neural network. The cropped target area is input into the feature extraction network. The network will pass through multiple attention layers in sequence to extract texture features from the bottom layer to semantic features at the high level, and finally output a feature vector of fixed length as the feature representation of the target; Step 4: Target positioning and tracking: Input the current frame features and the initial target features into the tracking network, repeat the above feature extraction steps, continue to track the target through the trained model processing video sequence in subsequent frame images, and finally output the video sequence with the target position annotated; Step 5: Model training and performance evaluation: Model training requires training data sets and test data sets, and evaluating model performance requires multiple evaluation indicators.

2. A single target tracking method based on aggregated token perception features according to claim 1, characterized in that: In step 3, the aggregated token perception feature module can effectively aggregate the original representation information of the image for perception features, wherein the process of aggregating the image can be summarized as follows: Row avg =Aug(Z 11 ,With 12 ,With 13 ...With 1n ) Column_avg=avg(Z 11 ,From 21 ,From 31 ...From n1 ) Z′=Z-Row_avg+Z-column_avg Where Z ij represents the value of the i-th row and j-th column of the two-dimensional feature map of the original image, Row_avg represents the average value of each row of the two-dimensional feature map, Column_avg represents the average value of each column of the two-dimensional feature map, avg(·) represents the average value, and Z′ represents the value finally input into VIT after the above operations, that is, By subtracting the average value of the template feature map, the information in the feature map can be aggregated and weighted, so that the model pays more attention to the target area.

3. A single target tracking method based on aggregated token perception features according to claim 1, characterized in that: In step 1, the multi-scale feature fusion module is used to make up for the shortcomings of insufficient feature extraction and capturing small targets or subtle changes in images. The multi-scale feature fusion module has four different layers, each of which contains convolutions of different numbers and sizes, so as to obtain richer receptive fields and deeper semantic information. The output features are passed into the multi-scale feature fusion module, and the output of each layer is fused through convolutions of different layers and sizes; The above process can be expressed as: D0=Z output Where Z output Denotes the output of the template image after VIT, D i represents the output of the i-th layer in the module, and cat represents the concatenation operation; After the above operation, the template features have deeper semantic information. In order to retain this information, we finally perform residual connection between the results obtained in each layer and the original feature D0, and perform element-by-element addition to obtain the final result D output : D output ND0+D1+D2+D3+D4 When the entire network model performs attention calculation, the feature extraction method of the template image Z remains unchanged; When performing attention calculation for the search image X, the asynchronous feature extraction method is continued, and separate feature extraction is performed on the search image X for the first 5 layers. The formula is as follows: Among them, Q, K and V are the query matrix, key matrix and value matrix respectively, subscripts Z and X represent the matrix items belonging to the template and search area respectively, and Z' represents the output of the template image Z after the aggregate token perception module; In the subsequent layers, the output D of the template image features after multi-scale feature fusion module is performed output After concatenation, attention is calculated, as shown below: Where Q, K and V are the query matrix, key matrix and value matrix respectively. The subscripts X and D represent the matrix items belonging to the search region and the output of the template image after passing through the multi-scale feature fusion module.

4. The single target tracking method based on aggregated token perception feature according to claim 1 is characterized in that: In step 3, corner point positioning and center head positioning in target tracking are two common prediction methods, which are used to determine the position and posture of the target in the image or video. Since the corner point positioning has a situation where the bounding box is much larger than the target, it generates too much redundancy and error information. Therefore, we use the center head method for prediction here, that is, three convolution branches are used to represent center classification, offset regression and size regression respectively, and then the corresponding regression coordinates are used to calculate the bounding box, and the target position is determined as the final prediction result; For the loss function, weighted focal loss is used for classification, and l1 loss and ordinary GIoU loss are combined for training. The final total loss function is expressed as follows: Among them, λ G =2 and It is a trade-off weight.

5. The single target tracking method based on aggregated token perception feature according to claim 1 is characterized in that: In the step 5, the training datasets: GOT-10K, a dataset containing 10,000 video clips, covering more than 560 categories of moving objects and 87 motion modes, with no overlap between most training and testing categories; TrackingNet: a large-scale target tracking dataset, the training set data includes 30,132 sequences, with an average of 471.4 frames per sequence; LaSOT: a large-scale, long sequence target tracking dataset, containing 1,140 training sequences, 280 test sequences, 1,400 sequences, totaling more than 3.5 million frames, with an average video length of 2,500 frames, and all sequences are long-term; COCO: the dataset has 328,000 images, containing 2.5 million annotated instances, created by using advanced user interfaces such as category detection, instance annotation, and instance segmentation; When training the model, for the benchmark test of the GOT-10k dataset, we followed the one-time protocol and only used the training set of GOT-10k as the training set of the model, and trained the model for 100 epochs. For other benchmarks, we used TrackingNet, Lasot, GOT-10K and COCO training splits to train the model for 300 epochs. The optimizer is AdamW optimizer, the weight decay is 1e-4, the initial learning rates of the encoder and decoder are 4e-5 and 4e-4 respectively, and the batch size is set to 32; Use PyTorch2.2 in Python3.11 on a 48G NVIDIAA40 graphics card, and each epoch training time is about 5 minutes; The evaluation indicator area under the curve (AUC) is used to measure the overall performance of the tracking algorithm at different overlap thresholds. The calculation formula is as follows: Where SR is the success rate under different overlap thresholds, and α is the overlap threshold (IoU). A higher AUC value indicates that the algorithm performs better under various conditions; Precision (P) measures the average deviation between the predicted position and the true target position, especially the deviation of the center position. The calculation formula is as follows: Where N is the total number of frames, c i is the center position of the prediction box of the i-th frame, is the center position of the ground truth frame of the i-th frame, is the Euclidean distance between two center points, I is the indicator function, which takes 1 when the distance is less than or equal to the threshold δ, otherwise it takes 0. High precision means that the prediction of the center position of the tracking target is accurate; Normalized precision (P Norm ) is used to measure the tracking accuracy at different scales, and the standardization process eliminates the impact of scale differences between different videos; Among them, diag(G i ) is the diagonal length of the ground truth box of the i-th frame; The average overlap (AO) measures the average overlap between the predicted box and the real box, reflecting the matching degree between the tracking result and the real target; Among them, B i is the predicted bounding box of the i-th frame, G i is the ground-truth bounding box of the i-th frame, IoU(B i , G i ) is the intersection-over-union ratio of the predicted box and the true box; SR 0.5 and SR 0.75 They represent the success rates at IOU thresholds of 0.5 and 0.75, respectively, and are used to evaluate the performance of the tracking algorithm under different levels of strictness; Where α is the IoU threshold (0.5 or 0.75), I is the indicator function, which takes 1 when IoU is greater than or equal to the threshold, otherwise it takes 0; FPS measures the processing speed of the tracking algorithm, which is especially important in real-time applications; A high FPS value means the algorithm is able to process more frames in real time.

Images