A method for anti-occlusion target tracking based on attention mask

Abstract

The present invention relates to an anti-occlusion target tracking method based on attention mask, and belongs to the field of computer vision. In view of the existing tracking method, when the target is occluded, the problem of tracking drift is prone to occur. The method first extracts features of the candidate area image by adding a feature extraction network with an attention mechanism; then, using the AMGN network, according to the extracted candidate area features and the spatial attention matrix, occlusion feature samples are obtained to simulate various target occlusion situations in the feature space and enhance the occlusion positive samples; finally, using the discriminant network, according to the fusion of the occlusion feature samples of the second and third convolutional layers, the feature is distinguished as belonging to the target or the background, and finally the target tracking under occlusion is realized. The method is suitable for video surveillance, human-computer interaction, unmanned driving and other fields, and can accurately track moving targets.

Description

Technical Field

[0001] The invention relates to a single target tracking method, in particular to an anti-occlusion target tracking method based on an attention mask, and belongs to the field of computer vision. Background Art

[0002] Moving target tracking is an important research topic in the computer field and has a wide range of applications in many fields, such as video surveillance, human-computer interaction, and unmanned driving. Existing moving target tracking methods can be roughly divided into generative methods and discriminative methods. With the application of deep learning networks, the performance of these two tracking methods has been greatly improved.

[0003] Generative methods usually locate targets through CNN (Convolutional Neural Network) feature learning related filtering. Specifically, a model describing the appearance of the target is first learned, and then the model is used to perform pattern matching in the entire image area. The area with the smallest error is the location of the tracked object. However, generative methods only focus on the tracked target itself and ignore background information. When the target is occluded or changes dramatically, tracking drift is very likely to occur.

[0004] Discriminative methods, also known as tracking-by-detection methods. This type of method first generates multiple proposal boxes, and then classifies each proposal box as a target or background. Through offline pre-training or online learning, a detector that can distinguish between background objects and foreground targets is obtained, and the position of the tracked object is further obtained. Due to the introduction of various online update detector models, this type of method is more adaptable to the complex changes of target objects during tracking, and has gradually become the mainstream in the field of target tracking research. However, when the target is occluded and sample data is lacking, the positive samples on each frame are highly overlapped in space, and the deep learning model is difficult to capture the target features in large-scale occlusion situations, resulting in tracking failure. Summary of the invention

[0005] In view of the problem that existing tracking methods are prone to tracking drift when the target is occluded, the present invention proposes an anti-occlusion target tracking method based on attention mask, so as to enable the tracker to better distinguish occluded targets and achieve higher accuracy and robustness.

[0006] The objective of the present invention is achieved through the following technical solutions:

[0007] The present invention discloses an anti-occlusion target tracking method based on attention mask. First, feature extraction is performed on the candidate region image by using a feature extraction network with attention mechanism added. Then, an AMGN (Attention-Based Mask Generative Net) network is used to obtain occlusion feature samples according to the extracted candidate region features and the spatial attention matrix, so as to simulate various target occlusion situations in the feature space and enhance the occlusion positive samples. Finally, a discriminant network is used to distinguish whether the feature belongs to the target or the background according to the fusion of the occlusion feature samples of the second and third convolutional layers, so as to finally realize target tracking under occlusion.

[0008] A method for anti-occlusion target tracking based on attention mask specifically comprises the following steps:

[0009] Step 1: Build a target tracking network for offline training and preprocess the training data.

[0010] The network structure used for offline training mainly includes a feature extraction network and a discriminant network. The feature extraction network consists of three pre-trained convolutional layers conv1, conv2, and conv3. The output of the CBAM module is added after conv3 as the output of conv3. Then, two fully connected layers FC4-1 and FC4-2 are used to cascade the feature maps obtained from conv2 and conv3 to the fully connected layers fc4 and fc5. The last layer is a multi-group binary classification layer, called fc6. fc4~6 are collectively referred to as the discriminant network.

[0011] The original training data contains continuous video frames captured from multiple videos. Each frame of the image is represented by a manually annotated Bounding Box (hereinafter referred to as bbox), that is, Ground-Truth bbox (hereinafter referred to as gt-bbox). In each frame of each video sequence, a positive and negative sample set is established based on the gt-bbox.

[0012] Step 2: Perform offline training on the constructed target tracking network.

[0013] In offline training, each video sequence corresponds to an fc6. In each round of training, one of the N videos is selected as the current sample source, and the fc6 corresponding to the sequence is set to trainable, and the remaining fc6 is set to untrainable. The video comes from a standard data set that has been manually annotated. Conv1~3, fc4, and fc5 share weights and are trainable. The input of the offline network is the image sample of the positive and negative sample bbox selected in step 1 that is scaled to a unified resolution of w×h. Through training with the current training data, the parameters of the conv1, conv2, and conv3 layers, as well as the initial parameters of the fc4 and fc5 layers are finally obtained. The training loss function is:

[0014]

[0015] Among them, L D Represents the training loss, posLoss is the positive sample score, and negLoss is the negative sample score.

[0016] The network is optimized using the SGD optimizer.

[0017] Step 3: Online Tracking

[0018] Step 3.1: In order to complete the online training process in the online tracking process, build the AMGN network and modify the network structure of the offline training network.

[0019] The AMGN network consists of two fully connected layers, with the feature map C extracted by conv3 3 and the spatial attention weight matrix M a As input. Placed after the feature extraction network and before the discriminant network.

[0020] Keep one fc6 and randomly reset its parameters. Use the output of conv3 to build a fully connected layer as the bounding box regression layer fc_bbox.

[0021] Step 3.2 Start tracking and bounding box regression

[0022] The bbox manually annotated in the first frame of tracking, i.e. gt-bbox, is used to represent the current tracking target. The input sample is established according to the position of gt-bbox, and the parameters of fc_bbox are obtained by linear regression. After the bounding box regression is completed, fc_bbox is no longer updated until the next tracking.

[0023] Step 3.3 First frame training

[0024] According to the position and IoU (Intersection over Union) of the gt-bbox, positive and negative input samples are established. Except for the AMGN network, the SGD optimizer is used for iterative training. In the feature extraction part, the multi-channel feature map C of the third layer convolution result is obtained. 3 , the second layer of convolution results in multi-channel feature map C 2 ; At the same time, obtain the spatial attention weight matrix M a ;

[0025] Step 3.4 AMGN pre-training

[0026] Step 3.4.1 Mask generation label

[0027] For all extracted positive sample feature maps, the result of the spatial attention module M a , which is similar to the single-channel C 3 The same. Select the position with the maximum value in the spatial attention weight matrix and set it to zero, and the rest to one, as the candidate label of the mask. Select the template with the lowest classification score as the final generated label M;

[0028] Step 3.4.2 Occlusion sample generation

[0029] According to M, use the AMGN network to generate a mask based on the attention mechanism G represents the generation operation of the AMGN network; the mask is combined with the multi-channel C 3 Perform dot multiplication to obtain feature samples under occlusion, which can be described by the formula:

[0030] Step 3.4.3 Feature fusion

[0031] Since the size of M is similar to the feature map C after the second convolution 2 The size does not match, and the mask needs to be processed. In the middle M th row and c M The weight value of the th column will be dot-multiplied by C 2 The corresponding position r S th row and c S th column, get C 2 _M, that is, C 2 M=M 2 ·C 2 , M 2 Reason The processed mask. Finally, the two layers of feature maps processed by the fully connected layer will be cascaded and sent to the final target classification branch. The process of mask adjustment is described by the following formula:

[0032]

[0033] r1=((r M -1)H S / H M )+(1 / 2)

[0034] r2=(r M H S / H M )+(1 / 2)

[0035] c1=((c M -1)W S / W M )+(1 / 2)

[0036] c2=(c M W S / W M )+(1 / 2)

[0037] Size H S ×W S The mask M 2 Directly from the size H M ×W M Mask Transformed to obtain. s The range of r is determined by r1 and r2 related to W, c s The value range of c is determined by c1 and c2 related to H.

[0038] Step 3.4.4 AMGN network pre-training

[0039] The pre-training loss function of the AMGN network is MSELoss, which is described by the following formula:

[0040]

[0041] Among them, x i ,y i is the value of the corresponding position of the mask template and the generated mask. n corresponds to the size of the mask template and the generated mask. (If the mask size is 3×3, then n=9). The network is optimized using the SGD optimizer.

[0042] Step 4: Real-time tracking

[0043] Based on the bbox output of the previous frame (the next frame after the first frame uses gt-bbox), generate candidate bboxes, calculate their positive sample scores through the network, and select the bbox with the largest value. If its positive sample score is greater than th, it is considered that the tracking is successful, and the following operations are performed:

[0044] The bbox is adjusted with the parameters of bounding box regression to obtain the bbox of the tracking result; the positive and negative sample sets are supplemented based on the bbox and IoU after bounding box regression.

[0045] If its positive sample score is less than th, tracking is considered to have failed.

[0046] Step 5: Real-time training

[0047] Real-time training is divided into long-term and short-term training. Training only updates the parameters of all fully connected layers. Long-term training is performed once per fixed frame, and short-term training is performed immediately in the frame where tracking fails. During long-term training, the AMGN network is added to the training, and the steps are the same as 3.4.1, 3.4.2, and 3.4.3. The loss function of the entire network is expressed as follows:

[0048]

[0049] f is the operation that adjusts the mask size and acts on the feature map. D is the discriminant operation of the discriminant network.

[0050] By alternately executing step 4 of the real-time tracking process and step 5 of the real-time training process, the target tracking result in the entire video sequence is finally obtained, that is, the position and scale information of the target.

[0051] Beneficial Effects

[0052] 1. The present invention discloses an anti-occlusion target tracking method based on attention mask, which effectively occludes the distinguishable features of the target and conducts adversarial training with a classification tracker as a discriminator, thereby enhancing the occlusion training samples and strengthening the target tracking capability when the target is occluded. After the target is occluded, it can be effectively tracked according to the features of the unoccluded area, thereby counteracting the problem of tracking drift when the target is occluded during target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of feature extraction network;

[0054] Figure 2 A schematic diagram of the AMGN network structure disclosed by the present invention;

[0055] Figure 3 A network structure diagram of the anti-occlusion target tracking method based on attention mask disclosed by the present invention;

[0056] Figure 4 Examples of candidate masks disclosed in the present invention;

[0057] Figure 5An example of tracking results of the anti-occlusion target tracking method based on attention mask disclosed in the present invention, where the left picture is the tracking result of the VITAL method, and the right picture is the tracking result of the target tracking method disclosed in this embodiment (the green box is the gt-bbox, and the red box is the tracking result). DETAILED DESCRIPTION

[0058] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. The technical problems solved by the technical solution of the present invention and the beneficial effects are also described. It should be noted that the described embodiments are only intended to facilitate the understanding of the present invention and do not have any limiting effect on it.

[0059] In the application scenario of video surveillance, pedestrians or vehicles as target objects are easily blocked by other pedestrians and vehicles, or easily blocked by other objects in the direction of the moving track. If it is impossible to continue tracking, the purpose of continuous tracking of suspicious persons or vehicles cannot be achieved. In order to ensure the tracking capability of the video surveillance system, this embodiment discloses an anti-occlusion target tracking method based on attention mask for a video surveillance and tracking system.

[0060] In this embodiment, the Pytorch deep learning framework is used for training and tracking in an environment with Intel(R) Core(TM) i7-8700 CPU 3.20GHz and NVIDIA GdForce GTX1660Ti GPU. The system block diagram is shown in the attached figure. Figure 3 As shown, the specific steps are as follows:

[0061] Step 1: Build a target tracking network for offline training and preprocess the training data.

[0062] The feature extraction network diagram is shown in the attached figure. Figure 1 . The network structure used for offline training mainly includes a feature extraction network and a discriminant network. The feature extraction network consists of three pre-trained convolutional layers conv1, conv2, and conv3. The output of the CBAM module is added after conv3 as the output of conv3. Then, two fully connected layers FC4-1 and FC4-2 are used to cascade the feature maps obtained from conv2 and conv3 to the fully connected layers fc4 and fc5. The last layer is a multi-group binary classification layer, called fc6. fc4~6 are collectively referred to as the discriminant network.

[0063] The original training data contains continuous video frames captured from multiple videos. Each frame of the image is manually annotated with a bounding box (hereinafter referred to as bbox), that is, a ground-truth bbox (hereinafter referred to as gt-bbox), which represents the location of the target in the image. In each frame of each video sequence, 50 bboxes with IoU ≥ 0.7 of gt-bbox are established as positive samples based on gt-bbox, and 200 bboxes with IoU ≤ 0.5 are established as negative samples.

[0064] Step 2: Perform offline training on the constructed target tracking network.

[0065] Mini batch is used in offline training. Each video sequence in offline training corresponds to an fc6. In each round of training, one of the N videos is selected as the current sample source, and the fc6 corresponding to the sequence is set to trainable, and the remaining fc6 is set to untrainable. The video comes from a standard data set that has been manually annotated. conv1~3, fc4, and fc5 share weights and are trainable. The samples in the mini batch come from 8 frames of images randomly selected from the current video sequence. The input of the offline network is the image sample of the positive and negative sample bbox selected in step 1, which is scaled to a uniform resolution of 107×107. Through training with the current training data, the parameters of the conv1, conv2, and conv3 layers, as well as the initial parameters of the fc4 and fc5 layers are finally obtained. The training loss function is:

[0066]

[0067] Among them, L D Represents the training loss, posLoss is the positive sample score, and negLoss is the negative sample score. The network is optimized using the SGD optimizer. The learning rate of conv1~3 is 0.0001, and the learning rate of fc4~6 is 0.001.

[0068] Step 3: Online Tracking

[0069] Step 3.1 In order to complete the online training process in the online tracking process, build the AMGN network. The schematic diagram of the AMGN network structure is shown in the attached figure. Figure 2 As shown in Figure 2, the network structure of the offline training network is modified.

[0070] The AMGN network consists of two fully connected layers, with the feature map C extracted by conv3 3 and the spatial attention weight matrix M a As input. Placed after the feature extraction network and before the discriminant network.

[0071] Keep one fc6 and randomly reset its parameters. Use the output of conv3 to build a fully connected layer as the bounding box regression layer fc_bbox.

[0072] Step 3.2 Start tracking and bounding box regression

[0073] Manually select the tracking target in the first frame of the surveillance video and obtain the first frame bbox, that is, gt-bbox represents the current tracking target. According to the position of gt-bbox, use the uniform random method to establish 1000 gt-box bboxes with IoU ≥ 0.6. The images within the range of 1000 bboxes are scaled to obtain 1000 107×107 resolution input samples, and linear regression is used to obtain the parameters of fc_bbox. After completing the border regression, fc_box is no longer updated until the next tracking.

[0074] Step 3.3 First frame training

[0075] According to the position and IoU (Intersection over Union) of the gt-bbox, a Gaussian distribution random method is used to establish 500 gt-box bboxes with IoU ≥ 0.7 as positive samples, and a uniform random method is used to establish 5000 gt-bbox bboxes with IoU ≤ 0.3 as negative samples. In addition to the AMGN network, the SGD optimizer is used for 50 iterations of training, and the learning rate of fc4~5 is set to 0.0001, and the learning rate of fc6 is set to 0.001. The minibatch size of each iteration is 128, using 32 randomly selected positive samples and 96 hard negative samples randomly selected from 1024 negative samples. In the feature extraction part, the multi-channel feature map C of the third layer convolution result is obtained. 3 , the second layer of convolution results in multi-channel feature map C 2 ; At the same time, obtain the spatial attention weight matrix M a ;

[0076] Step 3.4 AMGN pre-training

[0077] Step 3.4.1 Mask generation label

[0078] For all extracted positive sample feature maps, the result of the spatial attention module M a , which is similar to the single-channel C 3 The position with the maximum value in the spatial attention weight matrix is ​​selected and set to zero, and the rest are set to one, as the candidate label of the mask. Figure 4 Several possible mask templates are listed in . The template with the lowest classification score is selected as the final generated label Label_G;

[0079] Step 3.4.2 Occlusion sample generation

[0080] According to Label_G, the AMGN network is used to generate a mask M=G(C 3 ), G represents the generation operation of the AMGN network; the mask is combined with the multi-channel C 3 Perform dot multiplication to obtain the feature samples under occlusion, which can be described by the formula C 3 _M=C 3 M;

[0081] Step 3.4.3 Feature fusion

[0082] Since the size of M is similar to the feature map C after the second convolution 2 The size does not match, and the mask needs to be processed. M th row and c M The weight value of the th column will be dot-multiplied by C 2 The corresponding position r S th row and c S th column, get C 2 _M, that is, C 2 _M=M 2 ·C 2 , M 2 is the mask processed by M. Finally, the two layers of feature maps processed by the fully connected layer will be cascaded and sent to the final target classification branch. The process of mask adjustment is described by the following formula:

[0083]

[0084] r1=((r M -1)H S / H M )+(1 / 2)

[0085] r2=(r M H S / H M )+(1 / 2)

[0086] c1=((c M -1)W S / W M )+(1 / 2)

[0087] c2=(c M W S / W M )+(1 / 2)

[0088] Size H S ×W S The mask M 2 Directly from the size H M ×WM The mask M is transformed. Step 3.4.4 AMGN network pre-training

[0089] The pre-training loss function of the AMGN network is MSELoss, which is described by the following formula:

[0090]

[0091] Among them, x i ,y i The corresponding position value of the mask template and the generated mask. The network is optimized using the SGD optimizer.

[0092] Step 4: Real-time tracking

[0093] According to the bbox output of the previous frame (the next frame after the first frame uses gt-bbox), 256 candidate bboxes are randomly generated in a Gaussian distribution, and their positive sample scores are obtained after network calculation. The bbox with the largest value is selected. If its positive sample score is greater than 0.5, it is considered that the tracking is successful, and the following operations are performed:

[0094] The bbox is adjusted with the parameters of bounding box regression to obtain the bbox of the tracking result. Based on the bbox and IoU after bounding box regression, 50 positive samples with IoU ≥ 0.7 and 200 negative samples with IoU ≤ 0.3 are randomly generated in a Gaussian distribution to supplement the positive and negative sample sets.

[0095] If its positive sample score is less than 0.5, tracking is considered to have failed.

[0096] Step 5: Real-time training

[0097] Real-time training is divided into long-term and short-term training. Training only updates the parameters of all fully connected layers. Long-term training is performed every 10 frames, and short-term training is performed immediately in the frame where tracking fails. During long-term training, the AMGN network is added to the training, and the steps are the same as 3.4.1, 3.4.2, and 3.4.3. The loss function of the entire network is expressed as follows:

[0098]

[0099] f is an operation that resizes the mask and acts on the feature map.

[0100] By alternately executing step 4 of the real-time tracking process and step 5 of the real-time training process, the tracking result of the selected target in the entire monitoring video sequence is finally obtained, that is, the position and scale information of the target.

[0101] Attached Figure 5 This is a comparison chart of the tracking effect of the anti-occlusion target tracking method based on attention mask disclosed in this embodiment and the existing state-of-the-art method VITA method. It can be clearly seen that when there are other obstacles or similar objects blocking the target, the tracking method disclosed in this embodiment has a stronger anti-occlusion ability.

[0102] The specific description above further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for anti-occlusion target tracking based on attention mask, characterized in that: The steps include: Step 1: Build a target tracking network for offline training and preprocess the training data; The implementation method of step 1 is: The network structure used for offline training mainly includes feature extraction network and discriminant network; the feature extraction network consists of three pre-trained convolutional layers conv1, conv2, and conv3, and the output of the CBAM module is added after conv3 as the output of conv3; then the feature maps obtained by conv2 and conv3 are cascaded to the fully connected layers fc4 and fc5 using two fully connected layers FC4-1 and FC4-2; the last layer is a multi-group binary classification layer, called fc6; fc4~6 are collectively called the discriminant network; The original training data contains continuous video frames captured from multiple videos. Each frame is manually annotated with a bbox as gt-bbox, which indicates the location of the target in the image. In each frame of each video sequence, a positive and negative sample set is established based on the gt-bbox. Step 2: Offline training of the constructed target tracking network; The implementation method of step 2 is: In offline training, each video sequence corresponds to an fc6. In each round of training, one of the N videos is selected as the current sample source, and the fc6 corresponding to the sequence is set to trainable, and the remaining fc6 is set to untrainable. The video comes from a standard data set that has been manually annotated. Conv1~3, fc4, and fc5 share weights and are trainable. The input of the offline network is the image sample of the positive and negative sample bbox selected in step 1 that is scaled to a uniform resolution of w×h. Through training with the current training data, the parameters of the conv1, conv2, and conv3 layers, as well as the initial parameters of the fc4 and fc5 layers are finally obtained. The training loss function is: Among them, L D Represents training loss, posLoss is the positive sample score, and negLoss is the negative sample score; Use SGD optimizer to optimize the network; Step 3: Online tracking; The implementation method of step 3 is: Step 3.1: In order to complete the online training process in the online tracking process, build the AMGN network; modify the network structure of the offline training network; The AMGN network consists of two fully connected layers, with the feature map C extracted by conv3 3 and the spatial attention weight matrix M a As input; placed after the feature extraction network and before the discriminant network; Keep one fc6 and randomly reset its parameters; use the output of conv3 to establish a fully connected layer as the bounding box regression layer fc_bbox; Step 3.2 Start tracking and bounding box regression Use the bbox manually annotated in the first frame of tracking and gt-bbox to represent the current tracking target, establish input samples according to the position of gt-bbox, and use linear regression to get the parameters of fc_bbox; after completing the bounding box regression, fc_bbox will no longer be updated until the next tracking; Step 3.3 First frame training According to the position and intersection-over-union ratio (IoU) of gt-bbox, positive and negative input samples are established. Except for the AMGN network, the SGD optimizer is used for iterative training. In the feature extraction part, the multi-channel feature map C of the third-layer convolution result is obtained. 3 , the second layer of convolution results in multi-channel feature map C 2 ; At the same time, obtain the spatial attention weight matrix M a ; Step 3.4 AMGN pre-training Step 3.4.1 Mask generation label For all extracted positive sample feature maps, the result of the spatial attention module M a , which is similar to the single-channel C 3 The same; select the position where the maximum value in the spatial attention weight matrix is ​​set to zero, and the rest are set to one, as the candidate label of the mask; select the template with the lowest classification score as the final generated label M; Step 3.4.2 Occlusion sample generation According to M, use the AMGN network to generate a mask based on the attention mechanism G represents the generation operation of the AMGN network; the mask is combined with the multi-channel C 3 Perform dot multiplication to obtain feature samples under occlusion, which can be described by the formula: Step 3.4.3 Feature fusion because Size and feature map C after the second convolution 2 The size does not match, and the mask needs to be processed; In the r M Row and c M The column weight value will be dot-multiplied by C 2 The corresponding position r S Row and c S Column, get C 2 _M, that is, C 2 _M=M 2 ·C 2 , M 2 Reason The processed mask; the two layers of feature maps processed by the fully connected layer will be cascaded and sent to the final target classification branch; the process of mask adjustment is described by the following formula: r1=((r M -1)H S / H M )+(1 / 2) r2=(r M H S / H M )+(1 / 2) c1=((c M -1)W S / W M )+(1 / 2) c2=(c M IN S / IN M )+(1 / 2) r and c represent r S and c S The value range is H S ×W S The mask M 2 Directly from the size H M ×W M The mask M is transformed to obtain, where H S Represents mask M 2 Height, W S Represents mask M 2 Width; H M represents the height of the mask M, W M represents the width of the mask M; Step 3.4.4 AMGN network pre-training The pre-training loss function of the AMGN network is described by the following formula: Among them, x i ,y i The corresponding position values ​​of the mask template and the generated mask; the network is optimized using the SGD optimizer; Step 4: Real-time tracking; The implementation method of step 4 is: Generate candidate bboxes based on the bbox output of the previous frame. Use gt-bbox for the next frame after the first frame. Obtain their positive sample scores after calculation through the network. Select the bbox with the largest value. If its positive sample score is greater than th, the tracking is considered successful. Perform the following operations: Use the parameters of bounding box regression to adjust the bbox and obtain the bbox of the tracking result; use the bbox and IoU after bounding box regression as the benchmark to supplement the positive and negative sample sets; If its positive sample score is less than th, tracking is considered to have failed; Step 5: Real-time training; The implementation method of step 5 is: Real-time training is divided into long-term and short-term training. Training only updates the parameters of all fully connected layers. Long-term training is performed once per fixed frame, and short-term training is performed immediately in the frame where tracking fails. During long-term training, the AMGN network is added to the training, and the steps are the same as 3.4.1, 3.4.2, and 3.4.

3. The loss function of the entire network is expressed as follows: f is the operation of adjusting the mask size and acting on the feature map in step 3.4.3, and D is the discriminant operation of the discriminant network; By alternately executing step 4 of the real-time tracking process and step 5 of the real-time training process, the target tracking result in the entire video sequence is finally obtained, that is, the position and scale information of the target.

Images