A single-target long-time tracking method

Abstract

The present application belongs to the field of visual target tracking, and provides a single-target long-time tracking method. In the process of target tracking, the present application first distinguishes different tracking states according to the difference between the target features in the search area represented by the confidence degree and the actual target features; in the case of good tracking effect, only the template area representing the target position needs to be updated at regular intervals; for the case of poor tracking effect, if the confidence degree is only low, it is determined that the target may be occluded, and only the search area needs to be enlarged in the tracking process of the next frame so that the target is completely contained in the search area; only when the confidence degree is too low, the target is determined to be lost, and the template area and the search area are updated, i.e. the complete re-detection process is performed. Therefore, the present application sets corresponding tracking strategies for different tracking effects, avoids unnecessary re-detection steps as much as possible, thereby improving the overall re-detection efficiency and ensuring the real-time performance of target tracking.

Description

Technical Field

[0001] This invention belongs to the field of visual target tracking, specifically relating to a long-term single-target tracking method. Background Technology

[0002] Visual target tracking refers to the technique of continuously tracking a specified target in a video. The tracking process is usually visualized by using a bounding box to lock onto the target. Typical target tracking methods are based on Siamese networks, defining the tracking task as a problem of measuring the similarity between two images. Initialized template information is input, and the image region with the highest degree of matching with the template is found in subsequent frames.

[0003] In long-term tracking scenarios, the aforementioned tracking methods accumulate erroneous information learned by the tracker, eventually leading to tracking failure. Furthermore, since the target is not always present in the scene, a target re-detection mechanism is required after tracking loss. Existing target re-detection mechanisms often predict the target's position in the next video frame through tracking detection, determine the reliability of the predicted position, and if the predicted position is found to be unreliable, perform a global re-detection, recalculate and determine the predicted position, and continue tracking the target based on the new predicted position.

[0004] For example, Chinese invention patent application CN111242977A discloses a target tracking method for panoramic video. In this method, if the confidence level of the predicted tracking position is less than a confidence threshold, target detection is performed again. Based on the detection results, a new predicted tracking bounding box is determined, and tracking is then performed on the next video frame using the new box. If it is determined that the target is not occluded, the tracker needs to be restarted after the new box is determined. This re-detection method avoids the accumulation of errors from using a fixed template and allows for timely updates to the tracking status. However, since the entire re-detection process takes time, frequent re-detection may affect the real-time performance of target tracking. Summary of the Invention

[0005] The purpose of this invention is to provide a long-term single-target tracking method to solve the problems of low re-detection efficiency and poor real-time performance caused by frequent re-detection process triggering in the target tracking process of existing technologies.

[0006] To achieve the above objectives, the present invention provides a long-term single-target tracking method, comprising the following steps:

[0007] 1. A long-term single-target tracking method, characterized by comprising the following steps:

[0008] 1) Obtain a video frame sequence, and determine a first template region and a first search region from the video frame sequence based on the tracking target; the first template region is a region of a first predetermined size cropped from the area near the target in the video frame where the target is first identified; the first search region is a region of a second predetermined size cropped from the area containing the target in the video frame where the target is first identified.

[0009] 2) Input the template region and the search region of the current video frame into the convolutional neural network to extract features from the two images and obtain two feature maps;

[0010] 3) Reduce the dimensionality of the two feature maps to one-dimensional feature vectors, perform feature fusion, and obtain a set of fused feature vectors;

[0011] 4) The fused feature vector is mapped to a high dimension through a convolutional kernel, and then the location prediction network is used to locate the target's position in the current frame using the mapped features to obtain the prediction result. At the same time, the fused features are also input into the scoring network to perform confidence scoring on the prediction result.

[0012] 5) Set corresponding tracking strategies based on different confidence scores, as follows:

[0013] If the confidence score output by the scoring network Greater than the first set threshold And the interval between the current time and the last time the template region was updated is greater than 100%. If the prediction result is not found, it will be added to the template pool as a reference when updating the template region of the next frame of the video. After calculating and updating the template region, the updated template region and the search region of the next frame of the video will be input into the convolutional neural network. The search region is determined based on the prediction result.

[0014] like Greater than the first set threshold The interval between the current time and the last time the template region was updated is less than 10 ... If the template region is not updated, the tracking continues into the next frame.

[0015] like Less than the first set threshold And greater than the second set threshold If not, no template region update will be performed, and the search region for the next frame will be expanded to the area of ​​the original search region, centered on the prediction result of the current frame. times;

[0016] like Less than the second set threshold If the template region is not updated, the entire image of the current frame of the video is input and the regression network is used to coarsely locate the target position. Then, the classification head in the regression network is used to determine whether the coarsely located region contains the target. If the target is contained, the region near the target is cropped out to a second set size as the new search region for the next frame of the video. Otherwise, the search region is not updated and the tracking of the next frame is performed directly.

[0017] The second set threshold Less than the first set threshold ;

[0018] 6) Repeat steps 2)-5) above until the video frame sequence ends.

[0019] The beneficial effects are: this tracking method can infer the actual target tracking status based on the tracking effect when the tracking effect is poor, and distinguish different tracking states; if it is determined that the target is only partially occluded, the search area is expanded in the next frame's tracking process to fully display the target, thus achieving accurate tracking; only when it is determined that the target is basically lost is the target re-located in the entire image, i.e., a complete re-detection process is performed to relocate and find the target. Because this method can set corresponding tracking strategies for different tracking effects, it can avoid unnecessary re-detection steps as much as possible while ensuring target tracking accuracy, thereby improving the overall re-detection efficiency and ensuring the real-time performance of target tracking.

[0020] Furthermore, to obtain more reliable target location results, in step 4), the purpose of the location prediction network is to learn a regression function. In the regression function, Indicates the prediction result. For the input image, These are the weights of the regression function; the location prediction network uses the RPN function to generate candidate bounding boxes, and the regression function is trained using the following formula:

[0021]

[0022] in This represents the feature extraction function of Siamese networks. For RPN functions, For input The corresponding tags This is a template image.

[0023] Furthermore, to accurately calculate the confidence level of the prediction results and make the judgment of the tracking status more reliable, in step 4), the input of the scoring network is the feature fused from the template region features and the search region features. The purpose of this network is to learn a function. Thus maximizing the confidence function Finally, through normalization control, the scoring network outputs a continuous value between 0 and 1.

[0024] Wherein, the confidence function is , Represents the feature extraction function. Represents the feature fusion function. Represents the template region image. Represents the image of the search area. This is a hyperparameter.

[0025] The confidence function is trained using the following objective function:

[0026]

[0027] in For loss function, These are the parameters learned by the network through training. For label functions, Follows a Gaussian normal distribution. Indicates the first Frame search region image, For input The corresponding tags.

[0028] Further, in step 4), the loss during the training of the scoring network is the GFocal loss:

[0029]

[0030] For FQ loss, For DF loss, For GIoU loss, As a positive sample, and They are and The weights, Represents the center coordinates of the sample location. exist The value is 1 if it is true, and 0 otherwise.

[0031] DF loss aims to amplify the probability of values ​​around the target; GIoU loss considers the degree of overlap between the predicted result and the true target shape; FQ loss smoothly adjusts weights when the predicted result deviates from the label, focusing on learning information from hard negative samples, thereby automatically reducing the contribution of easy samples and balancing the imbalance between positive and negative samples. GFocal loss uses FQ loss to balance the positive and negative sample set, and learns information from samples near the target center at the center of the positive samples through DF loss and GIoU loss, allowing the confidence function to learn a better data distribution from both positive and hard negative samples.

[0032] Furthermore, in order to optimize the feature fusion effect and thus improve the reliability of the prediction results, in step 3), feature fusion is performed through a Transformer encoder and decoder structure; wherein, the Transformer encoder structure is a multi-head self-attention module with a feedforward network; the decoder consists of multi-head self-attention, multi-head cross-attention and a feedforward network.

[0033] Furthermore, to accurately distinguish different target tracking situations and avoid updating the template region too frequently, thus reducing unnecessary update computation costs, in step 5), a first threshold is set. Select within the range, and set the second threshold. exist Select from the range. ,in This represents the total number of frames in the video sequence.

[0034] The first threshold setting distinguishes between good and poor tracking. The second threshold setting further categorizes poor tracking into two degrees: one is insufficient tracking leading to target occlusion (the target in the search area is not complete, a milder degree); the other is that the predicted search area almost entirely excludes the target (complete loss of the target, a more severe degree). Different tracking strategies are applied to the next video frame for these different tracking conditions, making the tracking strategy more closely match the actual tracking situation and improving efficiency. Furthermore, by setting thresholds to control the number of template region updates, frequent updates to the template region without significant changes can be avoided when the frame rate is too high, thus improving the efficiency of template region updates.

[0035] Furthermore, in step 5), when strategy ① is adopted, we have:

[0036]

[0037] For the updated template region features, Features of the first frame template region. ;

[0038] To fuse the features of all template images within the template pool, the concat function is used as the fusion method. For hyperparameters, This represents the element-wise product.

[0039] Using this fusion template information can effectively preserve the initial template information, through control. The value can prevent template information from being contaminated.

[0040] Furthermore, in order to accurately determine the target's presence status in the current frame and recapture the target in the event of tracking failure or target loss, in step 5), the strategy... The regression network described uses a backbone network and a feature enhancement network to extract and output features. The system trains a regression head and a classification head in the head network. The input to the regression network is the target template and the global image. After passing through the feature extraction network, the classification head determines the classification score of the target and the background in the global image. If the highest classification score of the output target is greater than a set threshold, the regression head coarsely locates the target position. The global image refers to the entire image of the current frame of the video. The regression head uses the RPN function to select the regression box through non-maximum suppression. A region of a second set size is cropped from the center coordinates of the box as a new search region for the next frame of the video, completing the current tracking. The classification head trains a discriminant function to match the features of the search region with the features of the template region to obtain the classification score.

[0041]

[0042] For template features, As a linear cross-correlation operator, FFN is a feedforward network containing convolutional layers, normalization layers, and softmax layers, outputting continuous values ​​between 0 and 1 as classification scores.

[0043] Furthermore, the set score threshold is 0.55.

[0044] Furthermore, to ensure that the template area better reflects the target features and the search area fully encompasses the target, the first set size is [size to be specified]. The second set size is . Attached Figure Description

[0045] Figure 1 This is a flowchart of a single-target long-term tracking method in an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the tracking network in an embodiment of the method of the present invention;

[0047] Figure 3 This is a schematic diagram illustrating different situations of determining the tracking state and the corresponding tracking strategies in an embodiment of the method of the present invention;

[0048] Figure 4 This is a schematic diagram of the re-detection network in an embodiment of the method of the present invention;

[0049] Figure 5 This is a comparison chart of the test results of the present invention and several different algorithms on the long-term tracking dataset LaSOT. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0051] Method Implementation Examples:

[0052] This embodiment provides a long-term single-target tracking method, referring to... Figure 1 The specific steps are as follows:

[0053] 1) Obtain a video frame sequence as a template region, and determine a first template region and a first search region from the video frame sequence according to the tracking target; the first template region is a region of a first set size cropped from the area near the target in the video frame where the target is first identified; the first search region is a region of a second set size cropped from the area containing the target in the video frame where the target is first identified.

[0054] When performing target tracking, the initial target location and features are first obtained as the basis for identification. Since the target may not appear in the first frame of the video, it is necessary to identify the target frame by frame according to the video sequence until the target is first identified in the video frame. Then, the target features in that frame are used as the first template region, which serves as the basis for comparison when identifying the target in subsequent frames. The area near the target is used as the first search area for target identification in the next frame. That is, target identification in the next frame is only performed within the search area, without the need to identify and match the entire image, thus improving the identification efficiency and ensuring real-time tracking.

[0055] 2) Input the template region and the search region of the current video frame into a convolutional neural network to extract features from the two images, resulting in two feature maps. The convolutional neural network used here can be a feature extraction network such as ResNet50.

[0056] 3) Reduce the dimensionality of the two feature maps to one-dimensional feature vectors, perform feature fusion, and obtain a set of fused feature vectors.

[0057] Feature fusion is achieved through a Transformer encoder and decoder structure. The Transformer encoder is a multi-head self-attention module with a feedforward network. Its input is flattened convolutional features. The multi-head self-attention module captures the feature dependencies of all elements in the input sequence and outputs a one-dimensional feature containing the positional relationships of each element. The decoder consists of multi-head self-attention, multi-head cross-attention, and a feedforward network. Its input is the encoder output and a target feature code. Through the interaction of the template and feature information on the search region by the self-attention and cross-attention modules, the predicted target location code is finally output.

[0058] 4) The fused feature vector is mapped to a high dimension using a convolutional kernel. A location prediction network then uses the mapped features to locate the target's position in the current frame and obtain the prediction result. Simultaneously, the fused features are also input into a scoring network to assign a confidence score to the prediction result. In this embodiment, when mapping the feature vector to a high dimension using a convolutional kernel, the following method is used: The convolution kernel, by controlling the depth of the convolution kernel, ultimately adjusts the feature dimension to 1024.

[0059] The purpose of the location prediction network is to learn a regression function. In this function, Represents the prediction result. For the input image, These are the weights of the regression function. The location prediction network uses the RPN function to generate candidate bounding boxes, and the regression function is trained using the following formula:

[0060]

[0061] in This represents the feature extraction function of Siamese networks. For RPN functions, For input The corresponding tags This is a template image.

[0062] The scoring network takes as input features a fused from template region features and search region features, and its goal is to learn a function. Thus maximizing the confidence function Finally, through normalization control, the scoring network outputs a continuous value between 0 and 1.

[0063] Wherein, the confidence function is , Represents the feature extraction function. Represents the feature fusion function. Represents the template region image. Represents the image of the search area. This is a hyperparameter.

[0064] The confidence function is trained using the following objective function:

[0065]

[0066] in For loss function, These are the parameters learned by the network through training. For label functions, Follows a Gaussian normal distribution. Indicates the first Frame search region image, For input The corresponding tags.

[0067] A higher confidence score indicates that more target features are identified within the search area, meaning better target tracking. A lower confidence score indicates that some or even most of the target's features are not included in the search area, thus indicating poor target tracking.

[0068] The individual networks in steps 2)-4) above constitute the overall tracking network, such as... Figure 2 As shown.

[0069] 5) Set corresponding tracking strategies based on different confidence scores, referring to... Figure 3 The details are as follows:

[0070] If the confidence score output by the scoring network Greater than the first set threshold And the interval between the current time and the last time the template region was updated is greater than 100%. The prediction result is then used as a new template region for the next frame of the video, and input into the convolutional neural network together with the search region for the next frame of the video. The search region is determined based on the prediction result.

[0071] like Greater than the first set threshold The interval between the current time and the last time the template region was updated is less than 10 ... If the template region is not updated, the tracking of the next frame can proceed directly; this avoids the need to extract template features for each frame during normal tracking, thus reducing the number of parameters.

[0072] like Less than the first set threshold And greater than the second set threshold If not, no template region update will be performed, and the search region for the next frame will be expanded to the area of ​​the original search region, centered on the prediction result of the current frame. times;

[0073] like Less than the second set threshold If the template region is not updated, the entire image of the current frame of the video is input and the regression network is used to coarsely locate the target position. Then, the classification head in the regression network is used to determine whether the coarsely located region contains the target. If the target is contained, the region near the target is cropped out to a second set size as the new search region for the next frame of the video. Otherwise, the search region is not updated and the tracking of the next frame is performed directly.

[0074] The second set threshold Less than the first set threshold And the first set threshold Second set threshold The value can be adaptively set according to actual needs, such as: first setting a threshold. Select within the range, and set the second threshold. exist Select within the range. The above thresholds are empirical constants, which usually reflect the degree of overlap between the tracker's prediction and the target's true location. The target deformation caused by local occlusion or rapid movement is usually below 50%.

[0075] In this step, two thresholds are compared with the confidence score to classify the tracking status into three categories and match different tracking strategies accordingly. A confidence score greater than the first set threshold... In the first case, most of the target's features are contained within the search area, indicating a reliable prediction. The prediction is used as a reference for updating the template region, and tracking continues in the next frame. If the confidence score is less than a first set threshold... And greater than the second set threshold In the second case, considering that the target is partially occluded by the search area, the search area is expanded in the next frame to continue tracking; when the confidence score is less than the second set threshold... When the target is considered to have largely disappeared from the image, it is necessary to recapture the target through a regression network. This involves sending the entire image of the current frame into the regression network for detection. If the regression network detects the target, it performs coarse localization, then crops the image area obtained from the coarse localization, and re-sends it into the tracking network to further refine the target location and obtain new template and search areas.

[0076] The first type of case is further divided into two different tracking strategies. The template region is only updated when the time interval since the last update is sufficiently long. This avoids overly frequent template region updates when tracking video sequences with high frame rates, thus improving the efficiency of template updates. In one embodiment, the period for updating the template region is... ,in This represents the total number of frames in the video sequence.

[0077] In one embodiment, when updating the template region, it is necessary to refer to the template region features before the update in the current frame to calculate the new template region features; in another embodiment, updating the template region also requires referring to the first template region features cropped from the area near the target in the video frame where the target was first identified, and the specific calculation formula is as follows:

[0078]

[0079] For the updated template region features, Features of the first frame template region. To fuse the features of all template images within the template pool, the fusion method used is concat. For hyperparameters, This represents element-wise product. Using this formula to fuse template information can effectively preserve the initial template information, and can be controlled... This prevents template information from being contaminated. Among these measures is the template pool. The capacity is 15. When the capacity is exceeded, the earliest template image that entered the template pool is discarded.

[0080] Strategy The regression network in the image is a re-detection network; after the target is lost, it needs to search for the target across the entire image. Figure 4 The entire image of the current frame of the video, i.e., the global image, and the target template are input into the backbone network to extract features, and the feature enhancement network (FPN) is used to optimize the feature representation of the input global image. Then, a regression head and a classification head are trained in the head network;

[0081] The regression network takes a target template (i.e., the template region corresponding to the target) and a global image as input. Both images are extracted through a backbone network to obtain corresponding features. For global image features, the feature representation of the input global image is optimized using FPN. For target template features, average pooling is used to preserve background information and max pooling is used to preserve details. Then, the target template features, the average pooled features, and the features after average pooling and further max pooling are input into three classification heads. The classification heads determine the classification scores of the target and background in the global image. If the classification score of one of the classification heads is higher than a set threshold, the regression head coarsely locates the target position. If the score is lower than the threshold, the tracking of the next frame is directly performed. The input of the search region for the next frame is an image region of a second set size, centered on the center coordinates of the "inaccurate" output box output by the local tracker. In a preferred embodiment, refer to Figure 4 A single classification head can be set up, with its input consisting of target template features obtained through average pooling and max pooling, thus avoiding data redundancy. The regression head similarly uses the target template and global image features, selecting regression boxes through non-maximum suppression. A region of a second predetermined size is then cropped from the center coordinates of this bounding box as the new search region for the next frame of the video, completing the tracking of the current frame. This classification head trains a discriminant function that matches the search region features with the target template features to obtain a classification score.

[0082]

[0083] For template features, As a linear cross-correlation operator, FFN is a feedforward network containing convolutional layers, normalization layers, and softmax layers. It outputs continuous values ​​between 0 and 1 as classification scores. If the highest output score is greater than 0.55, it outputs the prediction result of the regression head, i.e., the regression box output by the regression head.

[0084] 6) Repeat steps 2)-5) above until the video sequence ends.

[0085] The cropping dimensions of the template area and the search area, namely the first set dimension and the second set dimension, can be adaptively set according to different target objects. In one embodiment, the first set dimension is... The second set size is The template needs to contain the entire area where the target is located, and setting the search area to an integer multiple of the template area is beneficial for the two areas to perform convolution. On this basis, the smaller the search area, the faster the tracker runs, and the larger the search area, the slower the tracker runs.

[0086] The template region is a cropped area from the entire image where the main object in the scene is the target. The search region is another "more likely" target-containing area cropped from the tracker's prediction result in the previous frame. In this embodiment, a tracking bounding box is also provided, which is a bounding box generated when a target is present within the region. The tracking bounding box has two parts: the tracker's predicted bounding box and the labeled ground truth location. The tracking performance of the tracker using this method is demonstrated in tests on several listed video sequences, which include challenging properties such as motion blur, partial or complete target occlusion, target partially out of view, significant scale changes, and interference from similar objects. In the case of a completely occluded target, the tracking bounding box fails to locate the target, but it can still be recaptured after the object reappears.

[0087] The following section compares and analyzes the target tracking process and results of a target with those of other target tracking methods and the target tracking method described in this embodiment:

[0088] like Figure 5 As shown, our algorithm (Ours) achieved second place in success rate, accuracy, and normalized precision curves on the long-term LaSOT dataset test set. This dataset extensively includes scenarios where targets are occluded or disappear from view. Figure 5 As can be seen, the success rate of this method on the LaSOT test set is 3% lower than LTMU, but 14% higher than the ATOM algorithm. Notably, the tracking speed of this method reached 34 FPS on the NVIDIA RTX 3060 GPU platform, while LTMU, ATOM, and the relatively efficient SPLT algorithm achieved tracking speeds of 13 FPS, 33 FPS, and 25.7 FPS, respectively. Therefore, this method achieves a good balance between tracking efficiency and accuracy.

[0089] In target tracking, this invention addresses situations where tracking performance is poor by first differentiating the tracking state based on the degree of difference between the target features in the search area (represented by confidence level) and the actual target features. If the confidence level is low, indicating the target may simply be occluded, the search area only needs to be expanded in the next frame. Only when the confidence level is extremely low and the target is determined to be lost is re-localization performed, i.e., a complete re-detection process is initiated. Therefore, this invention sets corresponding tracking strategies for different tracking performances, minimizing unnecessary re-detection steps, thereby improving overall re-detection efficiency and ensuring real-time target tracking.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A long-term single-target tracking method, characterized in that, Includes the following steps: 1) Obtain a video frame sequence, and determine a first template region and a first search region from the video frame sequence based on the tracking target; the first template region is a region of a first predetermined size cropped from the area near the target in the video frame where the target is first identified; the first search region is a region of a second predetermined size cropped from the area containing the target in the video frame where the target is first identified. 2) Input the template region and the search region of the current video frame into the convolutional neural network to extract features from the two images and obtain two feature maps; 3) Reduce the dimensionality of the two feature maps to one-dimensional feature vectors, perform feature fusion, and obtain a set of fused feature vectors; 4) The fused feature vector is mapped to a high dimension through a convolutional kernel, and then the location prediction network is used to locate the target's position in the current frame using the mapped features to obtain the prediction result. At the same time, the fused features are also input into the scoring network to perform confidence scoring on the prediction result. 5) Set corresponding tracking strategies based on different confidence scores, as follows: If the confidence score output by the scoring network Greater than the first set threshold And the interval between the current time and the last time the template region was updated is greater than 100%. If the prediction result is not found, it will be added to the template pool as a reference when updating the template region of the next frame of the video. After calculating and updating the template region, the updated template region and the search region of the next frame of the video will be input into the convolutional neural network. The search region is determined based on the prediction result. like Greater than the first set threshold The interval between the current time and the last time the template region was updated is less than 10 ... If the template region is not updated, the tracking continues into the next frame. like Less than the first set threshold And greater than the second set threshold If not, no template region update will be performed, and the search region for the next frame will be expanded to the area of ​​the original search region, centered on the prediction result of the current frame. times; like Less than the second set threshold If the template region is not updated, the entire image of the current frame of the video is input and the regression network is used to coarsely locate the target position. Then, the classification head in the regression network is used to determine whether the coarsely located region contains the target. If the target is contained, the region near the target is cropped out to a second set size as the new search region for the next frame of the video based on the target position. Otherwise, the search region is not updated and the tracking of the next frame is performed directly. The second set threshold Less than the first set threshold ; 6) Repeat steps 2)-5) above until the video frame sequence ends.

2. The single-target long-term tracking method according to claim 1, characterized in that, In step 4), the goal of the location prediction network is to learn a regression function. In the regression function, Indicates the prediction result. For the input image, These are the weights of the regression function; the location prediction network uses the RPN function to generate candidate bounding boxes, and the regression function is trained using the following formula: ； in This represents the feature extraction function of Siamese networks. For RPN functions, For input The corresponding tags The template image is denoted by ; i is the ordinal number of the input image, and n represents the total number of input images. This represents the L2 loss function.

3. The single-target long-term tracking method according to claim 1, characterized in that, In step 4), the input to the scoring network is the fused feature of the template region and the search region. The purpose of this network is to learn a function. Thus maximizing the confidence function Finally, through normalization control, the scoring network outputs a continuous value between 0 and 1. Wherein, the confidence function is , Represents the feature extraction function. Represents the feature fusion function. Represents the template region image. Represents the image of the search area. For hyperparameters; The confidence function is trained using the following objective function: ； in For loss function, These are the parameters learned by the network through training. For label functions, Represents the template region image. Follows a Gaussian normal distribution. Indicates the first Frame search region image, For input The corresponding label indicates the first Frame template region image; y is the template pool to which the template region image y belongs.

4. The single-target long-term tracking method according to claim 1, characterized in that, In step 4), the loss during the training of the scoring network is the GFocal loss: ； For FQ loss, For DF loss, For GIoU loss, As a positive sample, and They are and The weights, Represents the center coordinates of the sample location. exist The value is 1 if it is true, and 0 otherwise. The label function is determined based on z.

5. The single-target long-term tracking method according to claim 1, characterized in that, In step 3), feature fusion is performed through a Transformer encoder and decoder structure; wherein, the Transformer encoder structure is a multi-head self-attention module with a feedforward network; the decoder consists of multi-head self-attention, multi-head cross-attention and a feedforward network.

6. The single-target long-term tracking method according to claim 1, characterized in that, In step 5), the first set threshold Select within the range, and set the second threshold. exist Select from the range. ,in This represents the total number of frames in the video sequence.

7. The single-target long-term tracking method according to claim 1, characterized in that, In step 5), when strategy ① is adopted, we have: ； For the updated template region features, Features of the first frame template region. ; To fuse the features of all template images within the template pool, the concat function is used as the fusion method. For hyperparameters, Represents element-wise product; D is the template pool of the template region before the current frame update, f i Let i be the i-th template image in template pool D.

8. The single-target long-term tracking method according to claim 1, characterized in that, In step 5), the strategy The regression network described uses a backbone network and a feature enhancement network to extract and output features. The system trains a regression head and a classification head in the head network. The input to the regression network is the target template and the global image. After passing through the feature extraction network, the classification head determines the classification scores of the target and the background in the global image. If the highest classification score of the output target is greater than a set score threshold, the regression head coarsely locates the target position. The global image refers to the entire image of the current frame of the video. The regression head uses the RPN function to select the regression box through non-maximum suppression. A region of a second set size is cropped from the center coordinates of the box as a new search region for the next frame of the video, completing the current tracking. The classification head trains a discriminant function to match the features of the search region with the features of the template region to obtain the classification score. ； For template features, As a linear cross-correlation operator, FFN is a feedforward network containing convolutional layers, normalization layers, and softmax layers, outputting continuous values ​​between 0 and 1 as classification scores.

9. The single-target long-term tracking method according to claim 8, characterized in that, The set score threshold is 0.

55.

10. The single-target long-term tracking method according to claim 1, characterized in that, The first set size is The second set size is .

Images