A single-target tracking method based on position uncertainty estimation

Abstract

A precise target tracking method based on a target transformation regression network includes the following steps: 1) generating training examples; 2) training the network itself; 3) offline training of the meta-classifier; and 4) online tracking. The position uncertainty estimation module designed in this invention can predict the confidence information of the network output position coordinates. In subsequent stages, a position voting mechanism is used to generate the final predicted bounding box, thus providing an accurate regression bounding box. Furthermore, this invention proposes an online update strategy based on meta-learning, enabling the tracker to adapt to changes in target shape and scale, thereby improving the tracker's robustness. Compared to existing single-target tracking methods, the tracking method of this invention has better adaptability to object deformation during the tracking process, effectively improving the accuracy of target regression.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer software, and relates to a single target tracking technology, in particular to a single target tracking method based on position uncertainty estimation. BACKGROUND

[0002] Target tracking is a basic task in the field of computer vision. Generally, the target tracking problem can be simply summarized as follows: given a video sequence and the position information of a target in the first frame, an algorithm needs to accurately track the position of the target in the subsequent video sequence, so as to obtain complete trajectory information of the target motion.

[0003] The target tracking problem can be regarded as a combination of a classification task and a state estimation task, wherein the former mainly provides a rough position information through classification, and the latter gives more accurate target state information on this basis. In order to make the tracking result more accurate, the design of the state estimation task is very important. Current target tracking algorithms can be classified into the following three categories according to state estimation. The first category mainly includes early correlation filter methods and twin network methods such as DCF, SiamFC, etc., which adopt a simple multi-scale test method in the state estimation stage. This method is neither accurate nor time-consuming. The second category is mainly SiamRPN algorithm series, which introduces the RPN module commonly used in the target detection field into SiamFC, so that the tracker can regress the position and shape, and also saves the multi-scale test. Although this way improves the accuracy of the algorithm and maintains the speed, there are still many shortcomings. The third category mainly includes ATOM and DiMP methods, which randomly generate a plurality of candidate boxes on the rough position given by the classification task, and then iteratively optimize these candidate boxes based on a specially designed IoU prediction network and using gradient descent, so as to obtain more accurate prediction boxes. Although this way makes great progress in accuracy, it has large amount of calculation and introduces too many hyperparameters which need to be carefully adjusted. In addition, some anchor-free methods recently appeared in the field of target detection have also been applied to the field of target tracking and have achieved good results, but these methods are still not accurate enough and the robustness needs to be improved. SUMMARY

[0004] The problem to be solved by the present application is that the position confidence is missing in the candidate frame screening stage of the target tracking process, which leads to low accuracy of the algorithm, and the tracker cannot adapt to the shape and size changes of the target, which leads to low robustness of the algorithm. In the candidate frame screening stage of the previous tracking method based on the twin network (such as SiamRPN, SiamFC++), the prediction frame corresponding to the highest class confidence is generally selected as the final prediction frame, but research in the target detection field has pointed out that this way is neither reasonable nor can the model only get a suboptimal solution. At the same time, most of these methods lack a fast and effective online updating mechanism to adapt to the shape and size changes of the target often occurring in the tracking process. Therefore, the design goal of the present application is to introduce a position uncertainty estimation module in the state estimation task to guide the screening of the candidate frame, and introduce a meta-learning-based classifier in the classification task for online updating, respectively used to improve the accuracy and robustness of the tracker.

[0005] The technical scheme of the present application is: a single target tracking method based on position uncertainty estimation, first, the network parameters of the target tracking network are trained offline, then in the tracking process, part of the previously predicted video frames are selected as online training samples of the classification branch of the target tracking network, the network parameter updating of the meta-classifier is carried out, the robustness of the tracking is improved, and the position voting mechanism is used in the candidate frame screening stage of the target tracking to improve the accuracy of the tracking.

[0006] Further, the method comprises generating training samples, offline training of the main network, offline training of the meta-classifier and online tracking:

[0007] 1) Generating training samples, first, the target region enhancement processing is performed on each frame image of each video in the offline training data set, then the target search region after the enhancement processing is cropped and scaled to a fixed size, two frames are extracted from each cropped video frame sequence according to a certain interval to generate a positive sample pair, one frame is randomly extracted from each of two different video sequences to generate a negative sample pair, one of each sample pair is used as a template frame, and the other is used as a search frame, for the positive sample pair, the classification branch label and the regression branch label are generated according to the search frame and the target annotation box, and for the negative sample pair, only the classification branch label is generated according to the search frame and the target annotation box;

[0008] 2) Offline training of the main network, including training of the network main part and training of the meta-classifier; for the training of the network main part, the template frame and the search frame picture are input into the twin network, the classification feature map and the regression feature map are extracted, the classification feature map of the template frame is used as the convolution kernel f cls of the classification branch, acts on the classification feature map of the search frame, and generates a class score confidence map M clsThe regression feature map of the template frame is taken as the convolution kernel f of the regression branch reg The regression feature map of the search frame is subjected to a convolution operation to generate a center point to target boundary distance regression map M reg and a distance confidence map M uncert respectively representing the distance of the center point of the target from the four boundaries of the object and the confidence value of the predicted distance; then the class confidence map M cls is found, and the offset distance corresponding to the point and the surrounding adjacent points on M reg is found, and the final predicted target frame is obtained by voting according to the confidence of the offset distance;

[0009] During training, the Focal Loss in RetinaNet is used as the loss function for the classification branch, the DIoU loss function is used for the regression branch, and the negative power log-likelihood loss (NPLL) loss function is used for the uncertainty estimation module. Combined with the labels obtained from the search frame, the SGD optimizer is used to update the entire network parameters through the backpropagation algorithm, and the above process is repeated by randomly extracting positive and negative samples until the iteration number is reached.

[0010] 3) Offline training of the meta-classifier, and the input of the meta-classifier in the inference stage is the classification feature map of the search frame, and the output is the class confidence map M' cls At this time, the class confidence map is weighted and summed with the class confidence map M cls in step 2) to obtain the final class confidence map M cls ←α·M cls +(1-α)·M' cls During the training stage, the MAML algorithm is used to train the meta-classifier to find a set of initialization parameters, so that the classifier only needs to use a small number of samples and undergo several gradient updates to quickly learn the information of the target.

[0011] 4) Online tracking, first, the target frame search area in the first frame image of the video to be tracked is cropped as a template, and then the template frame is expanded into an online training data set containing 5 frames of images. After 5 times of gradient descent, the network parameters are updated so that the meta-classifier can perform the classification task on the current tracking target. During tracking, every 10 frames from the already tracked frame sequence are selected as a frame with the highest classification score and the target frame that has been tracked to add to the online training data set for updating the meta-classifier.

[0012] The object of the present application is to construct a precise target tracker which can adapt to the deformation of the target, distinguish the background interference, etc., thereby improving the robustness of the tracker. As the previous analysis indicates, the SiamRPN series mainly depends on the category confidence of the corresponding candidate frame in the candidate frame screening stage, which is neither reasonable nor can lead to a suboptimal solution. Although the ATOM algorithm introduces an IoU prediction network and uses the IoU value predicted by the network to replace the category confidence value, the method has large calculation amount and limited accuracy improvement. The present application adopts a full convolutional Siamese network structure and invents a single target tracking method based on position uncertainty estimation, named FCST (Fully Convolutional Siamese Tracker). The position uncertainty estimation module designed in the present application can predict the confidence information of the position coordinates output by the network, and generate the final prediction frame using the position voting mechanism in the subsequent stage, so as to give an accurate regression bounding box. In addition, the present application proposes an online updating strategy based on meta-learning, so that the tracker can adapt to the shape and scale changes of the target, thereby improving the robustness of the tracker.

[0013] Compared with the prior art, the present application has the following advantages.

[0014] The present application proposes a single target tracking method based on position uncertainty estimation (Fully Convolutional Siamese Tracker, FCST). This method adopts a full convolutional Siamese network structure, introduces a position uncertainty estimation module in the target state estimation task, and generates the final prediction frame through a position voting method, thereby improving the tracking accuracy while ensuring the object tracking efficiency.

[0015] The present application introduces an online updating classifier based on meta-learning in the classification task, which can adapt to the shape and scale changes of the target after only a few iterations with a small amount of training samples in the tracking process. Compared with the existing tracking method based on Siamese network, the FCST tracking method proposed in the present application can better adapt to the deformation of the object in the tracking process, and effectively improve the robustness of target classification.

[0016] The present application has achieved good results in the single object tracking task, and improved the accuracy of target regression and the robustness of target classification. Compared with the existing method, the FCST tracking method proposed in the present application has good tracking success rate and positioning accuracy in multiple visual tracking test benchmark data sets. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is the system framework diagram used in the present application.

[0018] Figure 2 is the meta-classifier structure schematic diagram.

[0019] Figure 3 is a schematic diagram of extracting template branch features in the network subject of the application.

[0020] Figure 4 is a schematic diagram of extracting search branch features in the network subject of the application. DETAILED DESCRIPTION

[0021] The application proposes an accurate single target tracking method based on a target transformation regression network. After offline training on four training data sets of TrackingNet-Train, LaSOT-Train, COCO-Train and GOT-10k-Train, and testing on several test sets of OTB100, VOT2018, LaSOT-Test and Got10k-Test, a high accuracy and tracking success rate are achieved. The specific implementation uses Python 3.6 programming language and Pytorch 1.4 deep learning framework.

[0022] Figure 1 is a system framework diagram used in the application. Through a full convolutional twin network, target classification and target regression templates are generated to guide the classification and regression tasks and the strategy of online updating the classification and regression templates, so as to realize the target tracking task. The entire method includes a training sample generation stage, a network subject training stage, a meta-classifier offline training stage and an online tracking stage, and the specific implementation steps are as follows:

[0023] 1) The data preparation stage, i.e. the training sample generation stage. In the offline training process, training samples are generated. First, the target region enhancement processing is performed on each frame image of each video in the offline training data set. Then, the enhanced target search region is cropped and scaled to a fixed size. Then, two frames are extracted from each cropped video frame sequence at a certain interval to generate a positive sample pair, and one frame is randomly extracted from each of two different video sequences to generate a negative sample pair. One of each sample pair is used as a template frame, and the other is used as a search frame. For the positive sample pair, the classification branch label and the regression branch label are generated according to the search frame and its target annotation box. For the negative sample pair, only the classification branch label is generated according to the search frame and its target annotation box. If it is a positive sample pair, when the position corresponding to a certain coordinate point on the class label image falls within the center region of the annotation box, it is marked as 1, and if it falls outside the annotation box region, it is marked as 0, and other positions on the image are marked as -1; if it is a negative sample pair, when the position corresponding to a certain coordinate point on the class label image falls within the center region of the annotation box, it is marked as 0, and other regions are marked as -1. For the regression branch label, the coordinate value of the center of the annotation box is marked as 1, and other positions are marked as 0.

[0024] 2) Network subject training stage, specifically as follows.

[0025] 2.1) Extracting template branch feature: First, the modified GoogLeNet is used as the backbone of the twin network to extract features, and the template frame Z i ∈R B×3×127×127 The extracted features are F temp ∈R B×256×5×5 , where the superscript temp means the feature extracted from the template frame, and B represents the size of the batch size. The GoogLeNet uses the parameters pre-trained on ImageNet.

[0026] 2.2) Template branch feature adjustment: In order to adapt the obtained features to different tasks (classification task, regression task), the features need to be adjusted. The template branch feature obtained in step 2.1) is input into a network containing a single convolutional layer, which uses a 3*3 convolutional kernel, a step size of 1, an input channel number of 256, and an output channel number of 256. The size of the adjusted template branch feature is changed from F temp ∈R B×256×5×5 to Similarly, in order to obtain features suitable for regression tasks, the template branch feature is input into another single-layer convolutional network, which uses the same size, step size, and channel number as above. The size of the adjusted template branch feature is changed from F temp ∈R B×256×5×5 to The subscript cls represents that the feature is the feature used by the template branch for classification tasks, and the subscript reg represents that the feature is the feature used by the template branch for regression tasks.

[0027] 2.3) Search frame feature extraction: The search frame is input into the other branch of the twin network, which is also the adjusted GoogLeNet, and the network parameters are also pre-trained on ImageNet. Unlike the template frame, the search frame has a size of X i ∈R B×3×255×255 After extracting the features from the backbone network, the search branch feature F search ∈R B×256×27×27 is obtained, where 256 represents the channel number.

[0028] 2.4) Search frame feature adjustment: Similarly, in order to adapt the search branch feature to different tasks, it needs to be adjusted. The search branch feature obtained in step 2.3) is input into a network containing a single convolutional layer, which uses a 3*3 convolutional kernel, a step size of 1, an input channel number of 256, and an output channel number of 256. The size of the adjusted search branch feature is changed from F search ∈R B×256×27×27 to Similarly, to get the feature adapted to the regression task, the search branch feature is input to another single-layer convolutional network with the same size, stride and number of channels as above. The adjusted search branch feature size becomes F search ∈R B×256×27×27 becomes The subscript cls indicates that the feature is the feature used by the search branch for the classification task, and the subscript reg indicates that the feature is the feature used by the search branch for the regression task.

[0029] 2.5) Get the classification confidence map: the classification feature of the template branch is taken as a convolution kernel and the classification feature of the search branch is convolved (i.e., cross-correlated) to obtain a feature with a size of F cls ∈R B×256×23×23 , and then passes through a three-layer convolutional network to output the final class confidence map M cls ∈R B ×1×19×19 The convolution kernel size of the first two layers of the three-layer convolutional network is 3*3, the stride is 1, the input channel is 256, and the output channel is 256. The convolution kernel size of the last layer is 1*1, the stride is 1, the input channel number is 256, and the output channel number is 1, which mainly serves to fuse information of different dimensions.

[0030] 2.6) Get the distance regression map M reg and the corresponding position uncertainty map M uncert : the regression feature of the template branch is taken as a convolution kernel and convolved with the regression feature of the search branch to obtain a feature with a size of F reg ∈R B×256×23×23 . Then this feature is input to a two-layer convolutional network with a convolution kernel size of 3*3, a stride of 1, and an input channel and an output channel of 256. At this time, the output feature size is F reg ∈R B×256×19×19 . Finally, it is input into two parallel single-layer convolutional layers, respectively. The convolution kernel size of these two convolutional layers is 1*1, the stride is 1, the input channel number is 256, and the output channel number is 4. At this time, the output features are M reg ∈R B×4×19×19 and M uncert ∈R B×4×19×19 , respectively, representing the distance of the center point of the target from the four boundaries of the object and the confidence value of the predicted distance, respectively.

[0031] 2.7) For the offline training of the classification branch, the Focal Loss proposed by RetinaNet is used as the loss function, the deviation distance prediction module in the regression (state prediction) branch uses the DIoU Loss as the loss function, and the position confidence estimation module in the regression branch uses the negative power log-likelihood loss function NPLL (negative power log-likelihood loss) loss function. In the experiment, the SGD optimizer is used, the BatchSize is set to 16, the total number of training rounds is 20, the initial learning rate is 0.0001, which is divided by 10 after 15 rounds, the decay rate is set to 0.1, and the whole network parameters are updated through the back propagation algorithm. Repeat steps 2.1) to 2.7) until the number of iterations is reached.

[0032] 3) The offline training phase of the meta-classifier needs to be carried out after the training of the main network, wherein the meta-classifier is a network containing several convolutional layers, and the features F cls ∈R B×256×23×23 As the input of the meta-classifier, the output is still a class confidence map, whose size is the same as the class confidence map output in step 2.5) Figure 1 Therefore, it is denoted as M' cls ∈R B×1×19×19 The present application adopts the MAML algorithm (Model-Agnostic Meta Learning) to train it, mainly including inner-level optimization and outer-level optimization. Specifically,

[0033] Given a video sequence V i , first collect a training sample set In the field of meta-learning, it is also called a support set, and the classifier is defined as f(x; θ0), wherein x is the input picture, and θ0 is the network initialization parameter. The network is updated using the k-step stochastic gradient descent algorithm on the training set:

[0034]

[0035] Wherein α is a weight factor, α = 0.6 in the experiment, L is a loss function, and (x, y) is a sample pair in the training set. In the MAML algorithm, the above formula is called inner-level optimization.

[0036] In order to evaluate the generalization performance of the classifier, a sample set i needs to be collected from the same video sequence V In the field of meta-learning, it is also called Target Set, using the model optimized by inner-level, to calculate its loss on

[0037]

[0038] where denotes the union of Support Set and Target Set. The training goal of the whole network is to find the network initialization parameter θ0 which can satisfy all video sequences as well as possible, which can be expressed as,

[0039]

[0040] This formula is called outer-level optimization, which is optimized and updated using Adam algorithm.

[0041] First, train the main network according to step 2), and then train the meta-classifier offline based on it. In the offline training process, 8 video sequences are randomly selected in each batch (i.e. batch size = 8), 600 iterations are performed in each round, and a total of 100 rounds are trained. In the training process, the inner-level optimization uses the stochastic gradient descent algorithm for 5 gradient updates, the learning rate is 0.01, and the outer-level uses the Adam algorithm for optimization, the learning rate is 0.001.

[0042] 4) In the online tracking phase, the meta-classifier needs to be updated online. Specifically, given a video sequence and the label of the first frame, the algorithm first uses the first frame picture and its label as training positive samples to fine-tune the meta-classifier so that it can perform classification tasks on the current target in the subsequent process. Since there is only one sample in the first frame, in order to make the meta-classifier have stronger generalization ability, data augmentation is used to expand the training sample set to form the Support Set.

[0043] In the subsequent tracking process, the invention continuously collects the previous tracking results, uses the update operation in the subsequent process, and uses the position voting mechanism to improve the accuracy of tracking in the subsequent candidate frame screening stage. From the already tracked frame sequence, every 10 frames are selected as the frame with the highest classification score and the target box that has been tracked to add to the online training data set as labels for updating the meta-classifier. Since the tracking result is not as reliable as the first frame label data, there may be inaccurate or even incorrect results, therefore, only when the position confidence of the predicted box is greater than a certain threshold θ loc and the class confidence is also greater than a certain threshold θ cls ​, the Support Set will be selected. At most 15 samples are cached in the Support Set in the experiment, and considering the time-consuming of online update, the update is performed every 10 frames. Only one gradient descent is performed to save time.

[0044] At the beginning of tracking, the target bounding box in the first frame of the video to be tracked is cropped and scaled to 127*127 as the input of the template branch. For the subsequent frame sequence, the search range size of the current frame is calculated according to the prediction result of the previous frame, and then the search region of the corresponding size is cropped and scaled to 255*255, and input into the other branch of the twin network. When the main network outputs the class confidence map M cls , the bias distance regression map M reg , the position uncertainty map M uncert and the class confidence map M' cls output by the meta-classifier, the two class confidence maps are first weighted and fused to obtain the final class response map:

[0045] M cls ← α·M cls +(1-α)·M' cls

[0046] α is a weight factor, and α = 0.6 in the experiment.

[0047] The highest scoring point on the class response map is found, and the N+1 prediction boxes corresponding to the position and its surrounding N positions are selected as the candidate set, Each element in the set contains the position bias and the corresponding confidence, where l i represents the value of the left side boundary of the prediction box i, and its corresponding uncertainty; t represents the value of the upper side boundary of the prediction box i, and its corresponding uncertainty; r i represents the value of the right side boundary of the prediction box i, and its corresponding uncertainty, b i represents the value of the lower side boundary of the prediction box i, and its corresponding uncertainty, and i is the serial number of the predicted target box. Then the four boundaries are divided into four sub-sets:

[0048]

[0049]

[0050] Then the K items with the highest confidence are selected from each sub-set in turn to form a new sub-set:

[0051]

[0052] The final prediction box is denoted as B pred = {l pred , t pred , r pred , b pred}, where l pred is the left boundary value of the final prediction box, and the specific calculation formula is as follows:

[0053]

[0054] Similarly, the upper boundary value t pred , the right boundary value r pred , and the lower boundary value b pred of the final prediction box are calculated.

[0055] On the test data set, the tracking efficiency is 30fps, the tracking accuracy Auc reaches 70.1% on the OTB100 data set, and Pre reaches 91.5%. On the VOT2018 data set, EAO reaches 0.474, Robustness reaches 0.164, and Accuracy reaches 0.609; on the LaSOT data set, Suc reaches 56.2%; on the GOT10 data set, SR .5 is 0.723, SR .75 is 0.530, and AR is 0.614.

Claims

1. A single-target tracking method based on position uncertainty estimation, characterized by: First, the network parameters of the target tracking network are trained offline. Then, during the tracking process, some video frames with previously predicted results are selected as online training samples for the classification branch of the target tracking network. The network parameters of the meta-learning-based classifier are updated to improve the robustness of tracking. At the same time, a position voting mechanism is used in the candidate box selection stage of target tracking to improve the accuracy of tracking. The target tracking network employs a fully convolutional Siamese network to generate target classification and regression templates to guide classification and regression tasks and to implement strategies for online updating of classification and regression templates, thereby achieving the target tracking task. The method includes generating training samples, offline training of the main network, offline training of the meta-classifier, and online tracking. 1) Generate training examples. First, perform target region enhancement processing on each frame of each video in the offline training dataset. Then, crop out the enhanced target search region and scale it to a fixed size. Extract two frames from each cropped video frame sequence at certain intervals to generate positive sample pairs. Randomly extract one frame from each of the two different video sequences to generate negative sample pairs. One of each sample pair is used as the template frame and the other is used as the search frame. For positive sample pairs, generate classification branch labels and regression branch labels based on the search frame and its target bounding box. For negative sample pairs, generate only classification branch labels based on the search frame and its target bounding box. 2) Offline training of the main network, including training of the main network component and training of the meta-classifier; for training the main network component, template frames and search frame images are first input into the Siamese network to extract their respective classification feature maps and regression feature maps, and the classification feature map of the template frame is used as the convolution kernel f of the classification branch. cls The classification feature map applied to the search frame is used to generate a class score confidence map M after convolution. cls The regression feature map of the template frame is used as the convolution kernel f of the regression branch. reg The regression feature map applied to the search frame, after convolution, generates a regression map M representing the distance from the center point to the target boundary. reg And its corresponding distance confidence map M uncert , representing the distances from the center point of the target to the four boundaries of the object, and the confidence values ​​of the predicted distances, respectively; then, based on the category confidence map M cls Find the highest score point, at M reg Find the offset distances of the point and its neighboring points, and then vote based on the confidence levels of these offset distances to obtain the final predicted bounding box. During training, Focal Loss from RetinaNet is used as the loss function for the classification branch, DIoU loss function is used for the regression branch, negative log-likelihood loss function NPLL is used for the uncertainty estimation module, and the SGD optimizer is used in combination with the labels obtained from the search frame. The entire network parameters are updated through the backpropagation algorithm. The above process is repeated by randomly selecting positive and negative samples until the number of iterations is reached. 3) The meta-classifier is trained offline. During the inference phase, the input to the meta-classifier is the classification feature map of the search frame, and the output is the class confidence map M'. cls At this point, compare this category confidence map with the category confidence map M from step 2). cls The weighted summation yields the final category confidence map M. cls ←α·M cls +(1-α)·M' cls , where α is the weighting factor; During the training phase, the MAML algorithm is used to train the meta-classifier and find a set of initialization parameters so that the classifier can quickly learn the target information using only a small number of samples and several gradient updates in this state. 4) Online tracking: First, the target bounding box search region in the first frame of the video to be tracked is cropped as a template. Then, the template frame is expanded into an online training dataset containing 5 frames. The network parameters are updated through 5 gradient descent iterations, enabling the meta-classifier to perform classification tasks on the currently tracked target. During the tracking process, every 10 frames from the already tracked frame sequence are selected as the frame with the highest classification score and the already tracked target bounding box as labels and added to the online training dataset to update the meta-classifier.

2. The single-target tracking method based on position uncertainty estimation according to claim 1, characterized in that: The training of the main network component is as follows: 2.1) Extract template branch features: For template frame Z i ∈R B×3×127×127 Extracting features yields template branch features F temp ∈R B ×256×5×5 ; 2.2) Template Branch Feature Adjustment: The template branch features obtained in step 2.1) are input into two networks containing a single convolutional layer, respectively. The adjusted template branch feature sizes are: F temp ∈R B×256×5×5 Become and F temp ∈R B×256×5×5 Become The subscript cls indicates that the feature is used by the template branch for classification tasks, and the subscript reg indicates that the feature is used by the template branch for regression tasks. 2.3) Search frame feature extraction: The search frame size is X i ∈R B×3×255×255 After feature extraction by the backbone network, the search branch feature is obtained as F. search ∈R B×256×27×27 ; 2.4) Search Frame Feature Adjustment: The search branch features obtained in step 2.3) are input into two networks containing a single convolutional layer, respectively. The adjusted search branch feature sizes are: F search ∈R B×256×27×27 Become and F search ∈R B×256×27×27 Become The subscript cls indicates that the feature is used by the search branch for classification tasks, and the subscript reg indicates that the feature is used by the search branch for regression tasks. 2.5) Obtain the classification confidence map: Treat the classification features of the template branch as the convolution kernel. Classification features of search branches After performing the convolution operation, we obtain F. cls ∈R B×256×23×23 The features are then processed through a three-layer convolutional network to output the final class confidence map M. cls ∈R B×1×19×19 ; 2.6) Obtain the distance regression map M reg And the corresponding location uncertainty diagram M uncert : Regression features of template branches As a convolution kernel, it is used in conjunction with the regression features of the search branch. Perform a convolution operation to obtain a result of size F. reg ∈R B×256×23×23 The feature is then input into a two-layer convolutional network, where convolution yields feature F. reg ∈R B×256×19×19 Finally, these features are fed into two parallel single-layer convolutional layers, and the output features are M and M respectively. reg ∈R B×4×19×19 and M uncert ∈R B×4×19×19 , representing the distances from the center point of the target to the four boundaries of the object and the confidence value of the predicted distance, respectively; 2.7) For offline training of the classification branch, the Focal Loss proposed by RetinaNet is used as the loss function. The bias distance prediction module in the regression branch uses the DIoU Loss as the loss function, and the location confidence estimation module in the regression branch uses the NPLL (negative power log-likelihood loss) loss function. The SGD optimizer is used, the BatchSize is set to 16, the total number of training rounds is 20, the initial learning rate is 0.0001, and it is divided by 10 after 15 rounds. The decay rate is set to 0.

1. The parameters of the entire network are updated through the backpropagation algorithm. Steps 2.1) to 2.7) are repeated continuously until the number of iterations is reached.

Images