Twin network target tracking method based on forgetting curve template update

Abstract

The application discloses a twin network target tracking method based on a forgetting curve template update, and specifically comprises the following steps: constructing an overall network, training the overall network structure; using the trained overall network to perform initialization tracking processing on a video image sequence to be tracked, obtaining an initial target template and initial position information of the target of the present tracking task; starting a tracking process, each frame of the video image sequence will calculate a position of a target in an image, and the target is displayed at the corresponding position in the image, obtaining a tracking result response map; using a double-layer template update threshold judgment mechanism to judge whether the present tracking result is reliable, if yes, updating the template, and if not, not updating the template; repeating the above process until all the video image sequences are tracked, and the tracking task is completed. The method solves the problems of poor robustness of object surface deformation in the target tracking process and template pollution caused by template update in the prior art.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of single target tracking based on video, and relates to a twin network target tracking method based on forgetting curve template updating. BACKGROUND

[0002] Single target tracking refers to a process of calculating the position of a target in subsequent frames according to a target tracking algorithm after manually specifying the position of the target in the first frame of a video image sequence. Single target tracking is an important research direction in the field of computer vision, which aims to enable computers to have the ability of human continuous capture of targets. The target tracking algorithm based on twin neural network is a mainstream research direction at present, and its achievements play an important role in scientific research and life application.

[0003] With the rapid development of intelligent monitoring, automatic driving and human-computer interaction, the demand for computer vision technology in various fields is also increasing. Video target tracking is a hot research branch in the field of computer vision, and its main purpose is to manually mark any target in the first frame of a video sequence, and predict the position of the target in each subsequent video frame through a tracking algorithm. In the target tracking process, many problems are faced, which can be mainly divided into two categories: external factors and internal factors. External factors include background clutter, light changes, similar interference and occlusion, etc. Internal factors include non-rigid deformation, motion blur, rotation and scale change, etc. How to cope with the above difficulties while tracking the target in real time and robustly is the goal of the target tracking algorithm.

[0004] With the rapid development of deep learning algorithm, the combination of deep learning and target tracking algorithm has also been paid more and more attention. Among them, the algorithm based on twin neural network structure is a mainstream direction. By using the target image given by the first frame to generate a template, the cross-correlation operation is performed on the subsequent images, and the position of the maximum value in the response map is mapped to the original image, which is the most possible position of the target. Under normal circumstances, the target template used by the target tracking algorithm based on twin neural network is kept unchanged, which leads to the fact that the tracker cannot adapt to the appearance change of the target. In addition, many methods involving template updating lack a good template updating mechanism, such as how to set a threshold judgment condition, and using unreliable template updating is easy to pollute the template. SUMMARY

[0005] The purpose of the present application is to provide a twin network target tracking method based on forgetting curve template updating, which solves the problems of poor robustness of object appearance deformation in the target tracking process and template pollution caused by template updating in the prior art.

[0006] The technical scheme adopted by the present application is a twin network target tracking method based on a forgetting curve template update, which is implemented according to the following steps:

[0007] Step 1, construct an overall network and perform end-to-end training on the overall network structure;

[0008] Step 2, use the overall network trained in step 1 to perform initialization tracking processing on a video image sequence to be tracked, and obtain an initial target template and initial position information of the target for the current tracking task;

[0009] Step 3, after obtaining the initial target template in step 2, start the tracking process, calculate the position of the target in the image for each frame of the video image sequence, and display the target at the corresponding position in the image to obtain a tracking result response map;

[0010] Step 4, after obtaining the tracking result response map in step 3, use a double-layer template update threshold judgment mechanism to judge whether the current tracking result is reliable, and if so, update the template, and if not, do not update the template;

[0011] Step 5, repeat steps 3 to 4 until all video image sequences are tracked, and the tracking task is completed.

[0012] The present application is characterized in that,

[0013] In step 1, the overall network structure is divided into four parts: the first part is a twin neural network for deep feature extraction, i.e., a feature extraction network, the second part is a template update module containing a forgetting curve and a template fusion, i.e., an FCM module, the third part is a cross-correlation network for accurately locating the target through a feature map, i.e., a target position prediction module; and the fourth part is a double-layer template update threshold judgment mechanism for judging whether the current result is reliable, i.e., a threshold judgment module.

[0014] In step 1, the structure of the whole network is trained end-to-end, specifically: using meta-learning one-shot learning, four video frames are randomly selected in each dataset video to form a picture pair, the first three are the target picture, and the last one is the search picture used to train the network, 10 picture pairs are selected from each video, each picture pair contains four video frames, the first video frame is the first frame of the video, the next three pictures are randomly selected in the video, the interval between the second and third pictures is not more than 15 frames, and the interval between the third and fourth pictures is not more than 10 frames, the first three video frames are used as the target picture to synthesize the tracking template, and the last video frame is used as the search picture, in the search picture processing, the three pictures imported into the template update module are the same, and are the last picture of the picture pair, the training is performed for 50 times, the batch_size is set to 8, the imported pictures are 32 pairs at a time, and the loss function adopts the Logistic loss function.

[0015] To generate a picture pair for training, data augmentation needs to be performed on the selected picture, which is implemented according to the following steps:

[0016] Step 1.1, first, the selected samples in the training set are subjected to a random stretching RandomStretch operation, the stretching size multiple is set to 0.095~1.005, and the enlarged part needs to be filled using a linear interpolation method; then, a center cropping CenterCrop operation is performed, that is, a region with a size of 263*263 is cropped from the center of the training picture pair, then a random cropping RandomCrop operation is performed, a region with a size of 255*255 is cropped from a random position in the training picture pair, and finally, a cropping conversion is performed. The BOX of the original GOT-10K dataset picture, that is, the target position frame, is given in the form of (left, top, weight, height), that is, the distance of the target frame from the left and top frame of the picture and the width and height of the target frame, and the cropping conversion operation converts the target frame coordinate form into (y, x, h, w), that is, the coordinates of the center point and the height and width of the target frame.

[0017] Step 1.2, LOSS calculation is performed:

[0018] The loss function formula used is:

[0019]

[0020] In the formula, v is the true score of the sample, g {+1, -1} is the label of the ground truth, and L is the loss of the sample.

[0021] Step 1.3, after calculating the loss, use gradient descent method to optimize the parameters, the calculation formula of the stochastic gradient descent method (SGD) is as follows:

[0022]

[0023] The optimal parameters can be obtained After 50 training, the final LOSS of the network is stable below 0.1, and the training process is completed.

[0024] The specific process of step 2 is as follows:

[0025] Step 2.1, specify the position of the target on the first frame of the video image sequence, cut the target from the image and scale it to get a target picture with a size of 127*127*3, then input the target picture into the feature extraction network in the whole network, and the output depth feature is the initial template. The size of the initial template is 6*6*256, and the unit is pixel. The calculation formula of the initial template is as follows:

[0026]

[0027] Wherein, z is the input target picture, the function represents the feature extraction network, represents the output of the feature extraction network, that is, the target template;

[0028] Step 2.2, parameter initialization:

[0029] In the first frame of the video image sequence, the target position information given by manual calibration is called BOX. The BOX has four information, which are the horizontal coordinate, vertical coordinate, width and height of the target. Therefore, the first frame does not need to perform tracking operation, only needs to set the initial center coordinate and initial width and height of the corresponding target in the BOX as the given value, that is, the initialization process of the target is completed.

[0030] The specific process of step 3 is as follows:

[0031] Step 3.1, extract the depth feature:

[0032] Take the target coordinates in the last frame of the image video sequence as the center, cut out the search area, and cut the search area into patch and input it into the feature extraction network to extract the depth feature of the search area, and get the search graph X.

[0033] Step 3.2, template update based on forgetting curve and template fusion:

[0034] Step 3.2.1, forgetting curve:

[0035] The initial template Z-initial and the current template Z-c are sent to the FCM module, and the retention value of the forgetting curve is calculated first ， The specific calculation formula is as follows:

[0036]

[0037] Wherein The retention rate is shown in percentage, t represents time in frames, and c and k are constants.

[0038] Step 3.2.2, template fusion:

[0039] After feature extraction, the initial template Z-initial, the historical template Z-h and the current template Z-c are transmitted to the FCM module for multi-template fusion operation;

[0040] In the first frame template initialization stage, the initial template is assigned to the current template Z-c, the historical template Z-h and the initial template Z-initial, and the three templates are given weights through the forgetting curve mechanism, and then the three templates are stacked together for template fusion to obtain the tracking template Z, the formula is as follows:

[0041] + +

[0042] In the formula, Z-initial is the initial template, Z-h is the historical template, Z-c is the current template.

[0043] Step 3.2.3, calculate the tracking result:

[0044] Finally, the response map is obtained by cross-correlation between the tracking template Z and the search map X, and the specific calculation formula is as follows:

[0045]

[0046] In the formula, represents the feature embedding function, bII represents the value corresponding to each position, represents the cross-correlation operation.

[0047] In step 4, the specific process of the double-layer template update threshold judgment mechanism is as follows:

[0048] First layer: using APCE and maximum response value F-max as the basis for judgment, whether the current tracking result is reliable, whether to update the template; The definition formula of APCE is as follows:

[0049] ​

[0050] wherein, , max and min represent the maximum and minimum values on the score map respectively, represents the value of the qth row and pth column of the score map;

[0051] If both the APCE and the F-max are lower than a preset threshold value, template updating is abandoned, and the next tracking is entered, and if both are higher than the threshold value, the current tracking result is sent to the FCM module for parameter matching and template fusion, so that template updating is performed;

[0052] The second layer: if the first layer threshold value judgment result is reliable, in the case of template updating, the cosine similarity of the current tracking result and the initial template is calculated and compared with a preset threshold value, to judge whether the tracking result is the tracking target and whether the forgetting curve value needs to be reset, and the calculation formula is as follows:

[0053]

[0054] wherein, the value ranges from [-1, 1], -1 is completely dissimilar, 1 is completely similar, A represents the initial template, represents the tracking result of the ith frame.

[0055] The beneficial effects of the present application are that the twin network target tracking method based on the forgetting curve template updating first proposes a multi-template fusion module (FCM module) based on the forgetting curve, the initial template features are propagated to each reliable template by giving the initial template a dynamic weight through the forgetting curve, and the background noise other than the target is inhibited; on this basis, a double-layer template updating threshold value judgment mechanism is also proposed, the APCE and the Pearson correlation coefficient are introduced to complement each other, reliable opportunities for updating the tracking template can be found in the tracking process, and reliable opportunities for resetting the forgetting curve value can be found when updating the template, which helps the tracker to continuously adapt to the target appearance change, and also greatly reduces the background noise introduced by template updating, prevents tracking drift, and at the same time, these modules are embeddable, convenient and fast, which can improve the performance of the tracker without affecting the real-time performance of the tracker. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 It is a whole framework schematic diagram of the template updating target tracking algorithm method based on the forgetting curve of the present application;

[0057] Figure 2 It is a network training schematic diagram of the template updating target tracking algorithm method based on the forgetting curve of the present application;

[0058] Figure 3A forgetting curve-based dynamic weight distribution module schematic diagram of the forgetting curve-based template updating target tracking algorithm method of the application;

[0059] Figure 4 A tracking initialization stage schematic diagram of the forgetting curve-based template updating target tracking algorithm method of the application;

[0060] Figure 5 A double-layer template updating threshold judgment mechanism schematic diagram of the forgetting curve-based template updating target tracking algorithm method of the application;

[0061] Figure 6 A template updating module schematic diagram of the forgetting curve-based template updating target tracking algorithm method of the application;

[0062] Figure 7 A tracking accuracy curve diagram of the forgetting curve-based template updating target tracking algorithm method of the application;

[0063] Figure 8 A tracking success rate curve diagram of the forgetting curve-based template updating target tracking algorithm method of the application;

[0064] Figure 9 An effect display diagram in a tracking process of the forgetting curve-based template updating target tracking algorithm method of the application. DETAILED DESCRIPTION

[0065] The application will be described in detail below in combination with the drawings and specific embodiments.

[0066] Embodiment 1

[0067] The application provides a twin network target tracking method based on forgetting curve template updating, as shown in Figure 1 The specific implementation is as follows:

[0068] Step 1, constructing an overall network and performing end-to-end training on the overall network structure;

[0069] The overall network structure is divided into four parts: the first part is a twin neural network for deep feature extraction, i.e., a feature extraction network, as shown in Figure 6 The second part is a template updating module containing forgetting curve and template fusion, i.e., an FCM module, the third part is a cross-correlation network for accurately locating a target through a feature map, i.e., a target position prediction module, and the fourth part is a double-layer template updating threshold judgment mechanism for judging whether a current result is reliable, i.e., a threshold judgment module;

[0070] As shown in Figure 2As shown, the end-to-end training of the overall network is specifically: using meta-learning one-shot learning, four video frames are randomly selected in each dataset video to form a picture pair, the first three are the target picture, and the last one is the search picture used to train the network, 10 picture pairs are selected from each video, each picture pair contains four video frames, the first video frame is the first frame of the video, the next three pictures are randomly selected in the video, the interval between the second and third pictures is not more than 15 frames, and the interval between the third and fourth pictures is not more than 10 frames, the first three video frames are used as the target picture to synthesize the tracking template, and the last video frame is used as the search picture, in the search picture processing, the three pictures imported into the template update module are the same, and are the last picture of the picture pair, the training is performed for 50 times, the batch_size is set to 8, the imported pictures are 32 pairs at a time, and the loss function adopts the Logistic loss function.

[0071] To generate the training picture pair, data enhancement needs to be performed on the selected picture, and the data enhancement is specifically implemented according to the following steps:

[0072] Step 1.1, first, the selected sample in the train training set in GOT-10k is subjected to a random stretching RandomStretch operation, the stretching size multiple is set to 0.095~1.005, and the part to be filled after enlargement is filled by using a linear interpolation method; then, a center cropping CenterCrop operation is performed, that is, a region with a size of 263*263 is cropped from the center of the training picture pair, then a random cropping RandomCrop operation is performed, a region with a size of 255*255 is cropped from a random position in the training picture pair, and finally, a cropping conversion is performed. The BOX of the original GOT-10K dataset picture, that is, the target position frame, is given in the form of (left, top, weight, height), that is, the distance of the target frame from the left and top frame and the width and height of the target frame, and the target frame coordinate form is converted into (y, x, h, w) by the cropping conversion operation, that is, the coordinates of the center point and the height and width of the target frame;

[0073] Step 1.2, LOSS calculation is performed:

[0074] The loss function formula used is:

[0075]

[0076] In the formula, v is the true score of the sample, g {+1, -1} is the label of the ground truth, and L is the loss of the sample.

[0077] Step 1.3, after calculating the loss, use gradient descent method to optimize the parameters, the calculation formula of stochastic gradient descent method (SGD) is as follows:

[0078]

[0079] The optimal parameters can be obtained in this way After 50 times of training, the final LOSS of the network is stable below 0.1, and the training process is ended.

[0080] Step 2, use the whole network trained in step 1 to initialize the tracking process of the video image sequence to be tracked, and obtain the initial target template and the initial position information of the target in this tracking task;

[0081] As shown in Figure 4 , the specific process of step 2 is as follows:

[0082] Step 2.1, specify the position of the target on the first frame of the video image sequence, scale the target after cutting it from the image to obtain a target picture with a size of 127*127*3, and then input the target picture into the feature extraction network in the whole network. The output depth feature is the initial template, the size of the initial template is 6*6*256, the unit is pixel, and the calculation formula of the initial template is as follows:

[0083]

[0084] Wherein, z is the input target picture, the function represents the feature extraction network, represents the output of the feature extraction network, that is, the target template;

[0085] Step 2.2, initialize the parameters:

[0086] In the first frame of the video image sequence, the target position information given by manual calibration is called BOX, which has four information, that is, the horizontal coordinate, vertical coordinate, width and height of the target. Therefore, the first frame does not need to perform tracking operation, and only needs to set the initial center coordinate and initial width and height of the corresponding target in the BOX to the values in the BOX, that is, the initialization process of the target is completed.

[0087] Step 3, after obtaining the initial target template in step 2, start the tracking process, and calculate the position of the target in each frame of the video image sequence in the image, and display it in the corresponding position in the image to obtain the tracking result response map.

[0088] The specific process of step 3 is as follows:

[0089] Step 3.1, extract the depth feature:

[0090] Using the target coordinates in the previous frame of the image / video sequence as the center, the search region is extracted and cropped into a patch, which is then fed into the feature extraction network to extract the depth features of the search region, thus obtaining the search map X.

[0091] Step 3.2, Template update based on forgetting curve and template fusion:

[0092] Step 3.2.1, Forgetting Curve:

[0093] The initial template Z-initial and the current template Zc are fed into the FCM module, and the retention value of the forgetting curve is calculated first. ， The specific calculation formula is as follows:

[0094]

[0095] in The percentage retention rate is given, where t represents time in frames, and c and k are constants.

[0096] Step 3.2.2, Template fusion:

[0097] After feature extraction, the initial template Z-initial, historical template Zh, and current template Zc are fed into the FCM module for multi-template fusion operation;

[0098] like Figure 3 As shown, in the first frame template initialization stage, the initial template is assigned to the current template Zc, the historical template Zh, and the initial template Z-initial. Weights are then assigned to the three templates through a forgetting curve mechanism. The three templates are then stacked together and fused to obtain the tracking template Z, as shown in the following formula:

[0099] + + )

[0100] In the formula, As the initial template, As a historical template, This is the current template. Indicates the optimal parameters;

[0101] The current template Zc refers to the current reliable tracking result as the template, and the historical template Zh refers to the previous reliable template;

[0102] Step 3.2.3, calculate the tracking results:

[0103] Finally, the response graph is obtained by cross-correlation between the tracking template Z and the search graph X. The specific calculation formula is as follows:

[0104]

[0105] In the formula, D represents a feature embedding function, bII represents the value corresponding to each position, represents a cross-correlation operation.

[0106] Step 4, after obtaining the tracking result response map of step 3, a double-layer template update threshold judgment mechanism is used to judge whether the current tracking result is reliable, if reliable, the template is updated, if not reliable, the template is not updated; if the reliable tracking results reserved in step 3 reach 2, the oldest result is replaced with the latest result;

[0107] As shown in Figure 5 , in step 4, the specific process of the double-layer template update threshold judgment mechanism is as follows:

[0108] First layer: using APCE and maximum response value F-max as the basis for judgment, whether the current tracking result is reliable, whether to update the template; the definition formula of APCE is as follows:

[0109]

[0110] In the formula, , respectively represent the maximum value and the minimum value on the score map, represents the value of the qth row and the pth column of the score map;

[0111] If APCE and F-max are both lower than the pre-set threshold, template update is abandoned, and the next tracking is entered, if both are higher than the threshold, the current tracking result is sent to the FCM module for parameter matching and template fusion, so as to update the template;

[0112] Second layer: if the first layer threshold judgment result is reliable, in the case of template update, the cosine similarity between the current tracking result and the initial template needs to be calculated and compared with the pre-set threshold, to judge whether the tracking result is the tracking target, whether the forgetting curve value needs to be reset, the calculation formula is as follows:

[0113]

[0114] In the formula, the value ranges from [-1, 1], -1 is completely dissimilar, 1 is completely similar, A represents the initial template, represents the tracking result of the ith frame.

[0115] Step 5, repeat steps 3-4 until all video image sequences are tracked, and the tracking task is completed.

[0116] Embodiment 2

[0117] The difference from Embodiment 1 is that the feature extraction network is an Alex-Net deep feature extraction network.

[0118] Embodiment 3

[0119] The difference from Embodiment 2 is that step 5 is specifically: for a tracking task, after the initialization is completed, the overall process is as steps 3-4, and the template updating and tracking calculation are repeatedly performed. In this process, each frame of the video sequence is calculated to obtain the target position of the frame and get a BOX for representation. For the whole video, the motion trajectory of the target is obtained. Until the target positions of all images of the whole video image sequence are obtained, the tracking task is completed. The accuracy and success rate of the method on the test set are shown in the following table: Figure 7 , 8 The effect display diagram in the tracking process is shown in the following figure: Figure 9 .

Claims

1. A twin network object tracking method based on a forgetting curve template update, characterized in that, The implementation is specifically performed according to the following steps: Step 1, constructing an overall network and performing end-to-end training on the overall network structure; Step 2, using the overall network trained in step 1 to perform initialization tracking processing on a video image sequence to be tracked, to obtain an initial target template and initial position information of the target of the current tracking task; Step 3, after obtaining the initial target template in step 2, starting the tracking process, calculating a position of the target in the image for each frame of the video image sequence, and displaying the target at the corresponding position in the image to obtain a tracking result response map; The specific process of step 3 is as follows: Step 3.1, extracting deep features: Taking the target coordinates in the last frame of the image video sequence as the center, a search region is intercepted, and the search region is cropped into a patch and input into a feature extraction network to extract deep features of the search region, to obtain a search map X; Step 3.2, template updating based on a forgetting curve and template fusion: Step 3.2.1, forgetting curve: The initial template Z-initial and the current template Z-c are sent to the FCM module, and the retention value of the forgetting curve is calculated first ， The specific calculation formula is as follows: wherein Retention in percent, t indicates time in frames, c and k are constants; Step 3.2.2, template fusion: After feature extraction, the initial template Z-initial, the historical template Z-h and the current template Z-c are input into the FCM module for multi-template fusion operation; In the first frame template initialization stage, the initial template is assigned to the current template Z-c, the historical template Z-h and the initial template Z-initial, and the three templates are given weights through the forgetting curve mechanism, and then the three templates are stacked together for template fusion to obtain a tracking template Z, and the formula is as follows: + + ) wherein is the initial template, is the historical template, is the current template, denotes the optimal parameters; Step 3.2.3, calculating the tracking result: Finally, the response map is obtained by cross-correlation between the tracking template Z and the search map X, and the specific calculation formula is as follows: wherein, denotes a characteristic embedding function, bII denotes a value corresponding to each position, denotes a cross-correlation operation; Step 4, after obtaining the tracking result response map in step 3, using a double-layer template update threshold judgment mechanism to judge whether the current tracking result is reliable, if yes, updating the template, if not, not updating the template; Step 5, repeating steps 3-4 until all video image sequences are tracked, and the tracking task is completed.

2. The twin network target tracking method based on the forgetting curve template update according to claim 1, characterized in that, In step 1, the overall network structure is divided into four parts: the first part is a twin neural network for deep feature extraction, i.e., a feature extraction network, the second part is a template updating module based on a forgetting curve and template fusion, i.e., an FCM module, the third part is a cross-correlation network for accurately locating the target through a feature map, i.e., a target position prediction module; and the fourth part is a double-layer template update threshold judgment mechanism for judging whether the current result is reliable, i.e., a threshold judgment module. 3.The twin network object tracking method based on the forgetting curve template update according to claim 2, characterized in that, In step 1, the structure of the whole network is trained end-to-end, specifically: using meta-learning one-shot learning, four video frames are randomly selected in each dataset video to form a picture pair, the first three are the target picture, and the last one is the search picture used to train the network, 10 picture pairs are selected from each video, each picture pair contains four video frames, the first video frame is the first frame of the video, the next three pictures are randomly selected in the video, the interval between the second and third pictures is not more than 15 frames, and the interval between the third and fourth pictures is not more than 10 frames, the first three video frames are used as the target picture to synthesize the tracking template, and the last video frame is used as the search picture, during the search picture processing, the three pictures imported into the template update module are the same, all are the last picture of the picture pair, the training is performed for 50 times, the batch_size is set to 8, 32 pairs of pictures are imported at a time, and the loss function adopts the Logistic loss function.

4. The twin network target tracking method based on the forgetting curve template update of claim 3, wherein, To generate training picture pairs, data augmentation needs to be performed on the selected pictures, which is implemented according to the following steps: Step 1.1, first, randomly stretch the selected samples in the training set, the stretching size multiple is set to 0.095~1.005, and the enlarged part needs to be filled using linear interpolation method; then, center crop the training picture pair to obtain a region with a size of 263*263 from the center of the training picture pair, then randomly crop the region with a size of 255*255 from a random position in the training picture pair, and finally, perform cropping conversion, the BOX of the original GOT-10K dataset picture, i.e., the target position frame, is given in the form of (left, top, weight, height), that is, the distance between the target frame and the left and top frames of the picture and the width and height of the target frame, and the coordinate form of the target frame is converted to (y, x, h, w) through the cropping conversion operation, that is, the coordinates of the center point and the height and width of the target frame; Step 1.2, LOSS calculation is performed: The loss function formula used is: where v is the true score of the sample, g {+1, -1} is the label of ground truth, L is the loss of the sample; Step 1.3, after calculating the loss, the gradient descent method is used to optimize the parameters to obtain the optimal parameters After 50 training iterations, the final LOSS of the network is stable below 0.1, and the training process is completed.

5. The twin network target tracking method based on the forgetting curve template update according to claim 1, characterized in that, The specific process of step 2 is: Step 2.1, the position of the target is specified on the first frame of the video image sequence, the target is cut from the image and scaled to obtain a target picture with a size of 127*127*3, and then the target picture is input into the feature extraction network in the whole network, and the output deep feature is the initial template, the size of the initial template is 6*6*256, and the unit is pixel, and the calculation formula of the initial template is as follows: wherein z is the input target picture, and the function represents the feature extraction network, represents the output of the feature extraction network, i.e. the target template; Step 2.2, parameter initialization is performed: In the first frame of the video image sequence, the target position information given by manual calibration is called BOX, which has four pieces of information, namely the horizontal coordinate, vertical coordinate, width, and height of the target, so the first frame does not need to perform tracking operation, only needs to set the initial center coordinates and initial width and height of the corresponding target in the BOX according to the given BOX, that is, the initialization process of the target is completed.

6. The twin network target tracking method based on the forgetting curve template update of claim 1, wherein, The specific process of the double-layer template update threshold judgment mechanism in step 4 is as follows: The first layer: using APCE and the maximum response value F-max as the judgment basis, judging whether the current tracking result is reliable and whether to perform template update; the definition formula of APCE is as follows: wherein , max and min represent the maximum and minimum values on the score map, respectively, represents the value of the qth row and pth column of the score map; If APCE and F-max are both lower than the pre-set threshold, template update is abandoned, and the next tracking is entered; if both are higher than the threshold, the current tracking result is sent to the FCM module for parameter matching and template fusion, so as to perform template update; The second layer: if the first layer threshold judgment result is reliable, in the case of template update, the cosine similarity between the current tracking result and the initial template is calculated and compared with the pre-set threshold, to judge whether the tracking result is the tracking target and whether the forgetting curve value needs to be reset, and the calculation formula is as follows: In the formula, the value ranges from [-1, 1], -1 is completely dissimilar, 1 is completely similar, A represents the initial template, represents the tracking result of the i-th frame.

Images