Robust object tracker optimized with combined trajectory loss and ranking loss

Abstract

The application provides a robust target tracker optimized by combining trajectory loss and ranking loss, and comprises the following steps: S1, extracting features by using a non-weight sharing twin backbone network to obtain a template feature map and a history feature map; S2, obtaining a target similarity response map by using the template feature map and the history feature map; S3, learning the motion information of the target by using trajectory loss learning; S4, optimizing a classification subnetwork and a regression subnetwork of the tracker by using IOU guided ranking loss; and S5, estimating the state of the target by combining the trajectory loss and the IOU guided ranking loss. The application can improve the accuracy and robustness of the twin tracker.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, and particularly relates to a robust object tracker optimized by combining trajectory loss and ranking loss. BACKGROUND

[0002] The purpose of visual object tracking is to estimate the location information of an arbitrary target in each frame of a video sequence. At present, many trackers are built on the basis of a twin network structure, which can achieve the most advanced performance. Although the twin tracking has achieved good performance, there are still two limitations: (1) It is difficult for the twin tracking to distinguish background interference. In particular, during the training stage, a large number of training samples are used to optimize the classification sub-network, which breaks the video sequence information and lacks the learning of motion information, so it is difficult for the tracker to distinguish the background interference. This training method has little effect on the optimization of the network. During the test, although most simple samples can be distinguished by the tracker, complex interference will seriously mislead the tracker, resulting in tracking failure.(2) The mismatch between classification and positioning. Most trackers decouple the classification sub-network and the regression sub-network. More specifically, the classification branch enables the model to learn to distinguish the target from the background, while the regression branch estimates the target bounding box of all positive samples without considering the target class. Therefore, a good regression result may have a relatively low foreground confidence, and a target with a high foreground confidence may produce a lower regression accuracy.

[0003] At present, many different sample weight redistribution strategies are used to suppress interference in target detection. However, in the target tracking task, most difficult negative samples have the same semantic category as the tracking target, and it is difficult to distinguish them in the classification embedding space. SUMMARY

[0004] The present application proposes a robust object tracker optimized by combining trajectory loss and ranking loss, which can improve the accuracy and robustness of the twin tracking.

[0005] The present application is implemented by the following technical solutions:

[0006] A robust object tracker optimized by combining trajectory loss and ranking loss, comprising the following steps:

[0007] Step S1, extracting features using a non-weight sharing twin backbone network to obtain a template feature map and a historical feature map;

[0008] Step S2, obtaining a target similarity response map using the template feature map and the historical feature map;

[0009] Step S3, learning the motion information of the target by trajectory loss, including: sequentially optimizing the minimum angle between the predicted bounding box and the center point of the ground truth bounding box by angle loss, optimizing the normalized distance between the predicted bounding box and the center point of the ground truth bounding box by distance loss, optimizing the aspect ratio between the predicted bounding box and the ground truth bounding box by shape loss, and optimizing the overlap rate between the predicted bounding box and the ground truth bounding box by IOU loss;

[0010] Step S4, optimizing the classification subnetwork and the regression subnetwork of the tracker by IOU guided ranking loss;

[0011] Step S5, realizing the estimation of the target state by combining the trajectory loss and the IOU guided ranking loss.

[0012] Further, in the step S1, the features of the input template image z and the search image x are extracted by using a non-weight sharing twin backbone network to generate a template feature map F z and a history feature map F x , in the step S2, the target similarity response map S is calculated according to the formula , wherein ω represents a feature similarity matrix, the elements of which are i and j respectively represent the indexes of each pixel point on F z and F x , C represents the channel dimension of the template feature map F z and the history feature map F x , concat(·) represents matrix connection, represents matrix multiplication, ⊙ represents dot product operation, and m represents the number of pixel points on F z .

[0013] Further, in the step S3, the minimum angle between the predicted bounding box and the center point of the ground truth bounding box is optimized by angle loss , wherein C h represents the difference between the vertical coordinates of the predicted bounding box and the center point of the ground truth bounding box, C w represents the difference between the horizontal coordinates of the predicted bounding box and the center point of the ground truth bounding box, represents the coordinates of the center point of the ground truth bounding box, represents the coordinates of the center point of the predicted bounding box, α represents the minimum angle between the predicted bounding box and the center point of the ground truth bounding box, x represents the sine value of the minimum angle between the predicted bounding box and the center point of the ground truth bounding box, and Δ represents the angle loss between the predicted bounding box and the ground truth bounding box.

[0014] Further, in the step S3, the distance loss optimizing the normalized distance between the predicted bounding box and the ground truth bounding box center point, wherein, γ = 2 - Δ, b w denotes the width of the predicted bounding box and the ground truth bounding box circumscribed rectangle, b h denotes the height of the predicted bounding box and the ground truth bounding box circumscribed rectangle, γ denotes a weight parameter, p x denotes the normalized distance between the predicted bounding box and the ground truth bounding box center point in the X-axis, p y denotes the normalized distance between the predicted bounding box and the ground truth bounding box center point in the Y-axis, Γ denotes the distance loss between the predicted bounding box and the ground truth bounding box.

[0015] Further, in the step S3, a shape loss optimizing the aspect ratio between the predicted bounding box and the ground truth bounding box, wherein, w and h respectively denote the width and height of the predicted bounding box, w gt and h gt respectively denote the width and height of the ground truth bounding box, ω w denotes the difference between the width of the predicted bounding box and the ground truth bounding box and the maximum width ratio between them, ω h denotes the difference between the length of the predicted bounding box and the ground truth bounding box and the maximum length ratio between them, θ is a hyperparameter for controlling the optimization speed of the bounding box shape, Ω denotes the shape loss between the predicted bounding box and the ground truth bounding box.

[0016] Further, in the step S3, an IOU loss optimizing the overlap rate between the predicted bounding box and the ground truth bounding box, wherein, ∩ denotes intersection operation, ∪ denotes union operation, B denotes the predicted bounding box, B GT denotes the ground truth bounding box, IOU denotes the IOU loss between the predicted bounding box and the ground truth bounding box, Ω denotes the shape loss between the predicted bounding box and the ground truth bounding box, Δ denotes the angle loss between the predicted bounding box and the ground truth bounding box, l traj,i denotes the trajectory loss between the i-th sample pair, and respectively denote the coordinates of the i-th sample pair at the predicted bounding box and the ground truth bounding box center position point, and respectively denote the coordinates of the i+1-th sample pair at the predicted bounding box and the ground truth bounding box center position point, ||x i || denotes the regularization term between the i-th sample and the i+1-th sample, L traj denotes the trajectory loss, T denotes the number of sample pairs.

[0017] Further, in the step S4, the IOU guided ranking loss is utilized The classification subnetwork and the regression subnetwork of the tracker are optimized, where σ is a hyperparameter for weighting the loss value, p i represents the confidence score of the positive sample i, p j represents the confidence score of the positive sample j, IOU i represents the predicted IOU value of the positive sample i, IOU j represents the predicted IOU value of the positive sample j, N pos represents the number of positive samples, L rank represents the IOU guided ranking loss; in the backpropagation optimization process, if the IOU i > IOU j , p i and p j will be optimized so that the ranking p i is higher than p j , if p i > p j , the IOU j will be frozen and only the IOU i will be optimized to achieve the expected ranking.

[0018] Further, the step S5 comprises the following steps:

[0019] Step S51, the combined trajectory loss and ranking loss are represented by the formula L traj-rank = L traj + λL cls + βL rank , where λ and β are hyperparameters for weighting the loss value, L traj represents the trajectory loss, L rank represents the IOU guided ranking loss, L cls represents the cross-entropy loss, L traj-rank represents the combined trajectory loss and ranking loss.

[0020] Step S52, the target position is predicted by the classification subnetwork of the tracker The target bounding box shape is predicted by the regression subnetwork of the tracker where, represents the classification subnetwork, represents the regression subnetwork, θ cls represents the classification response map, θ reg represents the regression response map, S represents the target similarity response map.

[0021] Further, the twin backbone network is the feature extraction network of the classic twin tracker.

[0022] Furthermore, the classic twin tracker can be a SiamRPN++ tracker, a SiamFC++ tracker, or an STMTrack tracker.

[0023] The present invention has the following beneficial effects:

[0024] In summary, the advantages of this invention are as follows:

[0025] This invention first combines the extracted template feature map and historical feature map to obtain the target similarity response map. Second, it improves the stability of the tracker by combining trajectory loss and IOU-guided ranking loss. Finally, it generates the tracker of this invention, which effectively improves the accuracy and robustness of mainstream twin trackers. Attached Figure Description

[0026] The present invention will now be described in further detail with reference to the accompanying drawings.

[0027] Figure 1 This is a flowchart of the present invention.

[0028] Figure 2-1 For the search frame.

[0029] Figure 2-2 This provides a visualization of the tracking results for the search frame using the existing tracker SiamRPN++.

[0030] Figure 2-3 This is a visual tracking result of the tracker of the present invention on the search frame.

[0031] Figure 3-1 This is a schematic diagram illustrating the angle optimization of trajectory loss according to the present invention.

[0032] Figure 3-2 This is a schematic diagram illustrating the distance optimization of trajectory loss according to the present invention.

[0033] Figure 3-3 This is a schematic diagram illustrating the shape optimization of the trajectory loss according to the present invention.

[0034] Figure 4-1 This is the accuracy curve obtained from the evaluation of the present invention and the existing tracker LaSOT.

[0035] Figure 4-2 This is a graph showing the success rate of the present invention compared to that of the existing tracker LaSOT. Detailed Implementation

[0036] like Figure 1 As shown, the robust target tracker optimized by joint trajectory loss and ranking loss includes the following steps:

[0037] Step S1: Extract features using a non-weight-sharing Siamese backbone network to obtain template feature maps and historical feature maps. Specifically:

[0038] extract features from the input template image z and the search image x using a non-weight-shared twin backbone network, to generate a template feature map and a history feature map wherein the twin backbone network is a feature extraction network of a classic twin tracker, which can be a SiamRPN++ tracker based on anchor boxes and deep cross-correlation, a SiamFC++ tracker without anchor boxes and deep cross-correlation, or an STMTrack tracker without anchor boxes and pixel-level cross-correlation;

[0039] Step S2, obtaining a target similarity response map using the template feature map and the history feature map, specifically:

[0040] According to the formula , the target similarity response map S is calculated, wherein ω represents a feature similarity matrix, and the elements of ω are i and j represent the pixel indices of F z and F x , respectively, and C represents the channel dimension of the template feature map F z and the history feature map F x , concat(·) represents matrix concatenation, represents matrix multiplication, ⊙ represents dot product operation, and m represents the number of pixel points on the template feature map F z .

[0041] Step S3, learning the motion information of the target using trajectory loss learning, including: sequentially using angle loss to optimize the minimum angle between the predicted bounding box and the center point of the ground truth bounding box, using distance loss to optimize the normalized distance between the predicted bounding box and the center point of the ground truth bounding box, using shape loss to optimize the aspect ratio between the predicted bounding box and the ground truth bounding box, and using IOU loss to optimize the overlap rate between the predicted bounding box and the ground truth bounding box, as shown in Figures 3-1 to 3-3 , specifically:

[0042] using angle loss to optimize the minimum angle between the predicted bounding box and the center point of the ground truth bounding box, wherein C h represents the difference in vertical coordinates between the center position point of the predicted bounding box and the center position point of the ground truth bounding box, C w represents the difference in horizontal coordinates between the center position point of the predicted bounding box and the center position point of the ground truth bounding box, represents the coordinates of the center position point of the ground truth bounding box, denotes the coordinate of the center position point of the predicted bounding box, a denotes the minimum included angle between the predicted bounding box and the center position point of the ground truth bounding box, x denotes the sine value of the minimum included angle between the predicted bounding box and the center position point of the ground truth bounding box, d denotes the Euclidean distance between the center position point of the predicted bounding box and the center position point of the ground truth bounding box, and D denotes the angle loss between the predicted bounding box and the ground truth bounding box; if the minimum included angle between the predicted bounding box and the center position point of the ground truth bounding box is then a is optimized; otherwise,

[0043] the distance loss is utilized the normalized distance between the predicted bounding box and the center position point of the ground truth bounding box is optimized, wherein, y = 2 - D, b w denotes the width of the circumscribed rectangle of the predicted bounding box and the ground truth bounding box, b h denotes the height of the circumscribed rectangle of the predicted bounding box and the ground truth bounding box, y denotes a weight parameter, and p x denotes the normalized distance of the distance between the predicted bounding box and the center position point of the ground truth bounding box on the X-axis, p y denotes the normalized distance of the distance between the predicted bounding box and the center position point of the ground truth bounding box on the Y-axis, and G denotes the distance loss between the predicted bounding box and the ground truth bounding box; when a approaches 0, that is, the predicted bounding box and the ground truth bounding box are nearly parallel, at this time, y approaches 2, the angle loss G approaches 0, and the contribution of the normalized distance to the overall loss is low; when a approaches , y approaches 1, the calculated angle loss G approaches 1, and the normalized distance should be valued, and the distance loss has a greater contribution degree;

[0044] the shape loss is utilized the aspect ratio between the predicted bounding box and the ground truth bounding box is optimized, wherein, w and h respectively denote the width and height of the predicted bounding box, w gt and h gt respectively denote the width and height of the ground truth bounding box, w w denotes the difference between the width of the predicted bounding box and the ground truth bounding box and the maximum width ratio therebetween, w h denotes the difference between the length of the predicted bounding box and the ground truth bounding box and the maximum length ratio therebetween, and Q is a hyperparameter for controlling the optimization speed of the bounding box shape, and Q denotes the shape loss between the predicted bounding box and the ground truth bounding box;

[0045] the IOU loss is utilized the overlap rate between the predicted bounding box and the ground truth bounding box is optimized, wherein, denotes the intersection operation, denotes the union operation, B denotes the predicted bounding box, and B GTdenotes the true value bounding box, IOU denotes the IOU loss between the predicted bounding box and the true value bounding box, Ω denotes the shape loss between the predicted bounding box and the true value bounding box, Δ denotes the angle loss between the predicted bounding box and the true value bounding box, l traj,i denotes the trajectory loss between the i-th sample pair, and denote the coordinates of the i-th sample pair at the center position points of the predicted bounding box and the true value bounding box, respectively, and denote the coordinates of the i+1-th sample pair at the center position points of the predicted bounding box and the true value bounding box, respectively, ||x i ||denotes the regularization term between the i-th sample and the i+1-th sample, L traj denotes the trajectory loss, and T denotes the number of sample pairs;

[0046] Step S4, the classification subnetwork and the regression subnetwork of the tracker are optimized by using the IOU-guided ranking loss, specifically:

[0047] IOU-guided ranking loss optimizes the classification subnetwork and the regression subnetwork of the tracker, where σ is a hyperparameter for weighting loss values, p i denotes the confidence score of the positive sample i, p j denotes the confidence score of the positive sample j, IOU i denotes the predicted IOU value of the positive sample i, IOU j denotes the predicted IOU value of the positive sample j, N pos denotes the number of positive samples, L rank denotes the IOU-guided ranking loss; during the backpropagation optimization process, if IOU i > IOU j , p i and p j are optimized so that the ranking p i is higher than p j , if p i > p j , the IOU j is frozen, and only the IOU i is optimized to achieve the expected ranking;

[0048] Step S5, the estimation of the target state is realized by combining the trajectory loss and the IOU-guided ranking loss, specifically including:

[0049] Step S51, the combined trajectory loss and ranking loss are denoted by the formula L traj-rank = L traj + λL cls + βL rank , where λ and β are hyperparameters for weighting loss values, Ltraj represents the trajectory loss, L rank represents the IOU guided ranking loss, L cls represents the cross-entropy loss, L traj-rank represents the joint trajectory loss and ranking loss;

[0050] Step S52, predicting the target position using the classification subnetwork of the tracker predicting the target bounding box shape using the regression subnetwork of the tracker wherein, represents the classification subnetwork, represents the regression subnetwork, cls represents the classification response map, reg represents the regression response map, S represents the target similarity response map.

[0051] The loss function of the classic Siamese tracker is replaced by the joint trajectory loss and ranking loss from the formula L traj-rank = L traj + λL cls + βL rank , that is, a robust target tracker optimized for joint trajectory loss and ranking loss can be generated, such as:

[0052] The loss function of the SiamRPN++ tracker is replaced by the joint trajectory loss and ranking loss from the formula L traj-rank = L traj + λL cls + βL rank , that is, a robust target tracker optimized for joint trajectory loss and ranking loss (TRO-SiamRPN++) can be generated;

[0053] The loss function of the SiamFC++ tracker is replaced by the joint trajectory loss and ranking loss from the formula L traj-rank = L traj + λL cls + βL rank , that is, a robust target tracker optimized for joint trajectory loss and ranking loss (TRO-SiamFC++) can be generated;

[0054] The loss function of the STMTrack tracker is replaced by the joint trajectory loss and ranking loss from the formula L traj-rank = L traj + λL cls + βL rank , that is, a robust target tracker optimized for joint trajectory loss and ranking loss (TRO-STMTrack) can be generated.

[0055] As Figure 4-1 and 4-2As shown, the performance of the tracker of the present application is verified through simulation experiments, and specifically, the effectiveness of the proposed tracker is verified through comparison with three mainstream trackers, namely STMTrack (line 2), SiamFC++ (line 4), and SiamRPN++ (line 6) as a comparison, and the public data sets OTB-50, TC128, UAV123 and LaSOT are selected for testing, Figure 4-1 and Figure 4-2 Ours (line 1), TRO-SiamFC++ (line 3), and TRO-SiamRPN++ (line 5) in the table and TRO-STMTrack, TRO-SiamFC++, and TRO-SiamRPN++ in the table represent the test results of the present application.

[0056] The parameters required in the experiment are set as follows: the size of the template image is 127x127, and the size of the search image is 255x255; in the track loss, the weight parameter λ = 0.75, and the weight parameter β = 0.5.

[0057] Figure 4-1 In the table, the abscissa represents the positioning error threshold, and the ordinate represents the accuracy, Figure 4-2 In the table, the abscissa represents the overlap threshold, and the ordinate represents the success rate, and from the two figures, it can be seen that the tracker of the present application exceeds the existing tracker in accuracy and success rate, and overall, the accuracy of the tracker of the present application is better than that of other trackers, and reaches the first in accuracy and success rate. Table 1 shows the AUC values of 15 trackers on OTB-50, TC128, and UAV123, and from Table 1, it can be seen that the TRO-SiamRPN++, TRO-SiamFC++, and TRO-STMTrack trackers of the present application have improved AUC values compared with the baseline tracker; compared with the AUC values of other trackers, the TRO-STMTrack tracker proposed in the present application reaches the highest on the TC128 and UAV123 data sets; and on the OTB100 data set, it is only slightly lower than SiamBAN (ACM). Similarly, TRO-SiamFC++ and TRO-STMTrack also effectively improve the performance of SiamFC++ and STMTrack, respectively.

[0058] Table 1

[0059]

[0060]

[0061] The above merely describes preferred embodiments of the present application, and therefore cannot limit the scope of the present application, i.e. equivalent changes and modifications made according to the patent application scope and specification content of the present application should still be within the scope of the present application.

Claims

1. A robust object tracker optimized with joint trajectory loss and ranking loss, characterized in that: The method comprises the following steps: Step S1, extracting features by using a non-weight sharing twin backbone network to obtain a template feature map and a history feature map; Step S2, obtaining a target similarity response map by using the template feature map and the history feature map; Step S3, learning motion information of the target by using a trajectory loss, comprising: sequentially optimizing a minimum included angle between a predicted bounding box and a true value bounding box center point by using an angle loss, optimizing a normalized distance between the predicted bounding box and the true value bounding box center point by using a distance loss, optimizing an aspect ratio between the predicted bounding box and the true value bounding box by using a shape loss, and optimizing an overlap rate between the predicted bounding box and the true value bounding box by using an IOU loss; Step S4, optimizing a classification subnetwork and a regression subnetwork of the tracker by using an IOU guided ranking loss; Step S5, realizing estimation of a target state by jointly using the trajectory loss and the IOU guided ranking loss. In the step S1, features of the input template image z and the search image x are extracted by using a non-weight shared twin backbone network to generate a template feature map F z and a history feature map F x In the step S2, a target similarity response map S is calculated according to the formula where ω represents a feature similarity matrix, elements of which are i and j respectively represent indexes of each pixel point on F z and F x C represents a channel dimension of the template feature map F z and the history feature map F x concat(·) represents matrix connection, represents matrix multiplication, ⊙ represents dot product operation, and m represents a number of pixel points on F z . the angle loss is calculated in step S3 the minimum angle between the predicted bounding box and the ground truth bounding box is optimized, wherein, C h represents the difference between the longitudinal coordinate of the predicted bounding box and the ground truth bounding box center position point, C w represents the difference between the transverse coordinate of the predicted bounding box and the ground truth bounding box center position point, represents the coordinate of the ground truth bounding box center position point, represents the coordinate of the predicted bounding box center position point, α represents the minimum angle between the predicted bounding box and the ground truth bounding box center point, x represents the sine value of the minimum angle between the predicted bounding box and the ground truth bounding box center point, and Δ represents the angle loss between the predicted bounding box and the ground truth bounding box.

2. The robust object tracker optimized with combined trajectory loss and ranking loss according to claim 1, wherein: In the step S3, the distance loss optimizing a normalized distance between the predicted bounding box and the ground truth bounding box center point, wherein, γ = 2 - Δ, b w denotes the width of the predicted bounding box and the ground truth bounding box circumscribed rectangle, b h denotes the height of the predicted bounding box and the ground truth bounding box circumscribed rectangle, γ denotes a weight parameter, ρ x denotes the normalized distance of the distance between the predicted bounding box and the ground truth bounding box center point in the X-axis, ρ y denotes the normalized distance of the distance between the predicted bounding box and the ground truth bounding box center point in the Y-axis, Γ denotes the distance loss between the predicted bounding box and the ground truth bounding box.

3. The robust object tracker optimized with combined trajectory loss and ranking loss according to claim 2, wherein: In the step S3, the shape loss The aspect ratio between the predicted bounding box and the ground truth bounding box is optimized, wherein, w and h represent the width and height of the predicted bounding box, respectively, w gt and h gt represent the width and height of the ground truth bounding box, respectively, ω w represents the difference between the width of the predicted bounding box and the ground truth bounding box and the maximum width ratio between them, ω h represents the difference between the length of the predicted bounding box and the ground truth bounding box and the maximum length ratio between them, θ is a hyperparameter for controlling the optimization speed of the shape of the bounding box, and Ω represents the shape loss between the predicted bounding box and the ground truth bounding box.

4. The robust object tracker optimized with combined trajectory loss and ranking loss according to claim 3, wherein: In the step S3, the IOU loss is utilized The overlap rate between the optimized prediction bounding box and the ground truth bounding box is optimized, wherein, ∩ represents intersection operation, ∪ represents union operation, B represents the prediction bounding box, B GT represents the ground truth bounding box, IOU represents the IOU loss between the prediction bounding box and the ground truth bounding box, Ω represents the shape loss between the prediction bounding box and the ground truth bounding box, Δ represents the angle loss between the prediction bounding box and the ground truth bounding box, l traj,i represents the trajectory loss between the i-th sample pair, and respectively represent the coordinates of the i-th sample pair at the center position points of the prediction bounding box and the ground truth bounding box, and respectively represent the coordinates of the i+1-th sample pair at the center position points of the prediction bounding box and the ground truth bounding box, ||x i || represents the regularization term between the i-th sample and the i+1-th sample, L traj represents the trajectory loss, and T represents the number of sample pairs.

5. The robust object tracker optimized with combined trajectory loss and ranking loss according to claim 4, wherein: In step S4, the ranking loss guided by IOU is utilized. Optimize the classification and regression subnetworks of the tracker, where σ is a hyperparameter used to weigh the loss value, and p i p represents the confidence score of positive sample i. j IOU represents the confidence score of positive sample j. i This represents the predicted IOU value for positive sample i. j N represents the predicted IOU value of positive sample j. pos L represents the number of positive samples. rank This represents the ranking loss driven by IOU; during backpropagation optimization, if IOU... i >IOU j , will optimize p i and p j , making the ranking p i Higher than p j If p i >p j This will freeze the IOU. j Only optimize IOU i To achieve the desired ranking.

6. The robust object tracker optimized with combined trajectory loss and ranking loss according to claim 5, wherein: The step S5 comprises the following steps: Step S51, the combined trajectory loss and ranking loss is represented by the formula L traj-rank = L traj + λL cls + βL rank , where λ and β are hyperparameters for balancing the loss values, L traj represents the trajectory loss, L rank represents the IOU-guided ranking loss, L cls represents the cross-entropy loss, L traj-rank represents the combined trajectory loss and ranking loss; Step S52, predicting target position by classification subnetwork of tracker predicting target bounding box shape by regression subnetwork of tracker wherein, denotes classification subnetwork, denotes regression subnetwork, cls denotes classification response map, reg denotes regression response map, S denotes target similarity response map.

7. The robust object tracker optimized with combined trajectory loss and ranking loss of any of claims 1-6, wherein: The twin backbone network is a feature extraction network of a classic twin tracker.

8. The robust object tracker optimized with combined trajectory loss and ranking loss according to claim 7, wherein: The classic twin tracker can be a SiamRPN++ tracker, a SiamFC++ tracker or an STMTrack tracker.

Images