Unmanned aerial vehicle target tracking method based on twin network and template updating

Abstract

The invention discloses an unmanned aerial vehicle target tracking method based on a twin network and template updating, and the method comprises the steps: carrying out the correlation introduction of a classification branch and a regression branch in a region suggestion network through a template updating mechanism, an improved multi-scale feature fusion module, and a matching network, and dynamically adjusting an auxiliary template of a target, thereby improving the tracking robustness, and improving the tracking precision. The problem that tracking fails due to the fact that an existing target tracking algorithm based on a twin network is prone to failing when a target comes out of a view or is shielded is solved. And semantic information and spatial information of the target are fully acquired, so that the tracking precision in a complex scene is remarkably improved, and the problem that the classification branch and the regression branch of the regional suggestion network are not tightly associated and the tracking accuracy is influenced is solved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a UAV target tracking method based on a twin network and template updating. Background Art

[0002] At present, target tracking algorithms based on Siamese networks have made significant progress in visual target tracking. Such algorithms are trained on large-scale data sets through deep learning technology, so that the model can effectively cope with various deformations and complex scenes of the target.

[0003] Bertinetto et al. first embedded the fully convolutional twin network into the tracking algorithm and proposed the SiamFC algorithm. The target position is determined by calculating the similarity between the template and the search area through the cross-correlation operation. The entire network can be trained offline, and the tracking process does not require fine-tuning. The SiamRPN model proposed by Li et al. uses the Region Proposal Network (RPN) to assist in target positioning. First, the twin network is used to extract features, and the RPN is used to locate the target. The RPN includes two branches, classification and regression. The classification branch distinguishes the target and the background, and the regression branch corrects the bounding box. Compared with the method of using the traditional image pyramid to predict the target scale, this method has significantly faster tracking speed. Later, Zhu et al. analyzed the unreliable response score problem of SiamRPN and found that this was caused by the imbalance of training samples, so they proposed the DaSiamRPN method. This algorithm improves the generalization performance by diversifying the types of positive sample pairs, and uses semantic negative sample pairs to improve the discrimination ability, improve the quality of the response score, and obtain more reliable tracking results.

[0004] However, there are still some challenges in existing technologies, especially in target state estimation and complex scene interference. For example, when the target is out of view, occluded, or interfered by similar targets, the existing twin network tracking algorithm is prone to failure. These problems are particularly prominent in drone aerial videos, because the targets in drone aerial videos often appear out of view and occluded due to factors such as rapid movement, perspective changes, and environmental complexity. Summary of the invention

[0005] The purpose of the present invention is to provide a UAV target tracking method based on twin networks and template updating, so as to solve the problem that the target is easily invalid when it is out of view or blocked, and the target in complex scenes is difficult to track.

[0006] The purpose of the present invention is to achieve a UAV target tracking method based on a twin network and template update by the following technical means, comprising the following steps:

[0007] Get the video of the object to be tracked and divide it into frames. Cut out the image of the object to be tracked from the first frame as the template image, and the complete image of the subsequent frames as the search image.

[0008] The template image and the second frame search image are input into the basic deep residual network respectively to extract their respective block3 and block4 features. Then, the respective block3 and block4 features are fused through the multi-scale fusion module to obtain the multi-scale fusion features of the template image and the multi-scale fusion features of the search image.

[0009] The multi-scale fusion features of the template image and the search image are fused and input into the region proposal network. The regression branch of the region proposal network generates candidate samples F c , the multi-scale fusion features of the template image and the candidate sample F c Input the matching network, output the matching score, and use the matching score to correct the classification branch of the region proposal network. The region proposal network outputs the template prediction result O0;

[0010] The multi-scale fusion features of the search image and the auxiliary template in the template buffer pool are fused and input into the region proposal network, and the region proposal network outputs the auxiliary prediction result O i ; The template buffer pool includes several auxiliary templates, one of which is selected for use, and the template image is initially used as the auxiliary template;

[0011] Auxiliary prediction results i The final tracking result O is obtained by fusing it with the template prediction result O0;

[0012] The final tracking result O is evaluated, and if qualified, it is added to the template buffer pool as an optional auxiliary template;

[0013] Repeat the above steps until the search image tracking and recognition of all frames are completed.

[0014] The basic deep residual network is ResNet-50.

[0015] Specifically, the multi-scale fusion module inputs the feature map of block 3 into four branches respectively, and connects the outputs of the four branches in the channel dimension;

[0016] The feature map output by block4 is refined and adjusted through convolution and deconvolution to achieve feature fusion.

[0017] The spatial feature block3 and the deep semantic feature block4 are fused element by element to obtain multi-scale fusion features;

[0018] After feature fusion, the number of channels is reduced through 1×1 convolution to obtain the final multi-scale fusion features.

[0019] The specific contents of the four branches are: branch one, performing 1×1 convolution, 5×5 convolution and 7×7 convolution on the input in sequence; branch two, performing 1×1 convolution and 3×3 convolution on the input in sequence; branch three, performing 3×3 average pooling and 1×1 convolution on the input in sequence; branch four, performing 1×1 convolution on the input.

[0020] The initial auxiliary template of the template buffer pool is a template image.

[0021] The total number of auxiliary templates in the template buffer pool is fixed. When the number of auxiliary templates reaches an upper limit, new auxiliary templates are added and the earliest auxiliary template is removed.

[0022] Use two 3×3 convolution kernels to replace the 5×5 convolution kernel; convert the 3×3 convolution into a 1×3 feature output after a 3×1 convolution, and then a 1×3 convolution to get a 1×1 feature output.

[0023] The matching network combines the multi-scale fusion features of the template image and the candidate sample F c , spliced ​​in the channel dimension, and then passed through a convolution and three fully connected layers to obtain the matching score between the two. The matching score is element-wise multiplied with the cross-correlation feature map of the classification branch to obtain improved correlation features. The improved correlation features are passed through the convolution layer to obtain the improved classification credibility score. The region proposal network outputs the template prediction result O0 according to the improved classification credibility score.

[0024] The specific method for evaluating the final tracking result O is: N N represents all templates in the template pool APCE The mean is if and only if N APCE >T N When , the current candidate sample feature map is added to the template buffer pool Γ,

[0025] F min is the minimum response value, F w,h is the response value at the coordinate index (w,h), F max is the maximum response value.

[0026] Select the template with the smallest similarity to the template image from the template buffer pool as the auxiliary template.

[0027] The beneficial effects of the present invention are:

[0028] 1. Through the template update mechanism, by introducing and dynamically adjusting the auxiliary template of the target, the robustness of tracking is improved, and the problem that the existing target tracking algorithm based on the twin network is prone to failure when the target is out of view or blocked, resulting in tracking failure is solved.

[0029] 2. By improving the multi-scale feature fusion module, the semantic and spatial information of the target can be fully acquired, significantly improving the tracking accuracy in complex scenes.

[0030] 3. The classification branch and regression branch in the region proposal network are associated through the matching network, which further improves the tracking effect and solves the problem that the classification branch and regression branch of the region proposal network are not closely associated, affecting the tracking accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is the structure diagram of the UAV target tracking method based on twin network and template update;

[0032] Figure 2 This is the Conv block structure diagram;

[0033] Figure 3 This is the Identity block structure diagram;

[0034] Figure 4 Result graph for visualization of deep features in convolutional networks;

[0035] Figure 5 It is the structure diagram of the multi-scale fusion module;

[0036] Figure 6 Schematic diagram of 5×5 convolution kernel decomposition;

[0037] Figure 7 Schematic diagram of 3×3 volume decomposition;

[0038] Figure 8 To match the network module structure diagram;

[0039] Fig. 9 Update the flow chart for the template buffer pool;

[0040] Fig.10 It is a success curve chart;

[0041] Fig.11 and accuracy graphs;

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0043] [Example 1]

[0044] like Figure 1 As shown, a UAV target tracking method based on a twin network and template updating includes the following steps:

[0045] Get the video of the object to be tracked and divide it into frames. Cut out the image of the object to be tracked from the first frame as the template image, and the complete image of the subsequent frames as the search image.

[0046] First, the complete video is divided into frames, and the image of the object to be tracked is cut out from the first frame, and the rest is discarded. The cut out image with the tracked object is used as the template image. The template image is as small as possible while containing the complete tracking image.

[0047] The remaining frames are all search images, and the tracking object needs to be found from the search images.

[0048] Figure 1 The tracking result is annotated based on the search image. Figure 1 In order to highlight the search target, the tracking result image, that is, the local part of the tracked object in the search image, is enlarged to avoid the search image where the target is too small to be seen clearly.

[0049] The template image and the second frame search image are input into the basic deep residual network respectively to extract their respective block3 and block4 features. Then, the respective block3 and block4 features are fused through the multi-scale fusion module to obtain the multi-scale fusion features of the template image and the multi-scale fusion features of the search image.

[0050] The basic deep residual network is ResNet-50.

[0051] The first frame template image and the search image are used to extract features through ResNet-50, and the features extracted to block3 and block4 are fused.

[0052] The multi-scale fusion features of the template image and the multi-scale fusion features of the search image are obtained.

[0053] The algorithm in this chapter uses ResNet-50 as the basic deep residual network. ResNet-50 contains four block modules, each of which consists of 3, 4, 6, and 3 block sub-modules. ResNet-50 has two basic blocks: Conv block and Identity block. Conv blocks cannot be connected in series, but the size of the feature map can be changed. The structure is as follows Figure 2 Identityblocks can be connected in series to increase the depth of the network. Figure 3 .

[0054] During the convolution process, edge details are enhanced. As the number of convolution layers increases, the features transition from shallow edge features to more abstract deep features, containing richer information, and the neural network becomes more sensitive to pixel space.

[0055] The multi-scale feature fusion structure ensures that the deep network does not lose the spatial characteristics of the target position while controlling the number of parameters, improves the network's feature expression for small targets, and thus improves the tracking effect. Figure 4 The figure shows the results of deep feature visualization in convolutional networks.

[0056] In order to combine deep features with shallow features, a multi-scale feature fusion module is designed, such as Figure 5 As shown in the figure, this multi-scale processing method can provide shallow spatial information for small target trackers, so that low-resolution feature maps can also achieve good representation effects.

[0057] like Figure 5 As shown, the multi-scale fusion module is specifically as follows: the feature map of block3 is input into four branches respectively, and the outputs of the four branches are connected in the channel dimension;

[0058] The feature map output by block4 is refined and adjusted through convolution and deconvolution to achieve feature fusion.

[0059] The spatial feature block3 and the deep semantic feature block4 are fused element by element to obtain multi-scale fusion features;

[0060] After feature fusion, the number of channels is reduced through 1×1 convolution to obtain the final multi-scale fusion features.

[0061] The specific contents of the four branches are: branch one, performing 1×1 convolution, 5×5 convolution and 7×7 convolution on the input in sequence; branch two, performing 1×1 convolution and 3×3 convolution on the input in sequence; branch three, performing 3×3 average pooling and 1×1 convolution on the input in sequence; branch four, performing 1×1 convolution on the input.

[0062] Use two 3×3 convolution kernels to replace the 5×5 convolution kernel; convert the 3×3 convolution into a 1×3 feature output after a 3×1 convolution, and then a 1×3 convolution to get a 1×1 feature output.

[0063] In order to ensure that shallow spatial information is not lost, this chapter uses the InceptionV3 method to perform asymmetric decomposition convolution operations on the target area. The essence is to replace the large convolution kernel with multiple layers of small convolution kernels. Figure 6 , decomposing the 5×5 convolution kernel in the branch into two 3×3 convolution kernels, reducing the number of parameters from 25 to 18.

[0064] like Figure 7, the 3×3 convolution in the branch is changed to a 3×1 convolution to obtain a 1×3 feature output, and then a 1×3 convolution to obtain a 1×1 feature output. This is equivalent to directly performing a 3×3 convolution, which can save 33% of parameters.

[0065] The multi-scale fusion features of the template image and the search image are fused and input into the region proposal network. The regression branch of the region proposal network generates candidate samples F c , the multi-scale fusion features of the template image and the candidate sample F c Input the matching network, output the matching score, and use the matching score to correct the classification branch of the region proposal network. The region proposal network outputs the template prediction result O0;

[0066] The RPN module contains a classification branch and a regression branch. Usually, these two branches are optimized independently, which leads to a mismatch between classification and regression. Specifically, the bounding box corresponding to the position with the highest classification confidence is not the most accurate, and even the location is not the target to be tracked. The classification branch loss uses the cross entropy loss. The regression branch loss is defined using the IoU loss, as shown in formula (4.3):

[0067] L reg =1-L IoU (4.3)

[0068] Therefore, a matching network is designed to generate a matching score to correct the difference between the two branches. A major feature of the tracking task is that the tracked target can be an arbitrary object, which greatly increases the difficulty of tracking. Usually the algorithm calculates the difference between the candidate sample and the target through a manual linear comparator (cosine distance, Euclidean distance, Mahalanobis distance). However, when faced with difficult-to-distinguish distractors in complex scenes, traditional manual methods may be ineffective, resulting in reduced tracking accuracy. Based on the above analysis, a matching network module with an adaptive nonlinear comparator is proposed to filter distractors by calculating the similarity relationship between the candidate sample and the target.

[0069] In order to train the matching network to obtain a higher matching degree, a triplet training pair (x, t, b) is designed, where x and t are from the same video sequence and b is a sample from a different sequence. The positive sample pair (x, t) and the negative sample pair (x, b) are obtained. Assume that p y and p n is the matching score of the positive and negative sample pairs, p r and p f They represent sample pairs with matching scores of 1 and 0 respectively, and they are combined in pairs for training, that is, (p r ,p y )、(p r ,p n ) and (pr ,p y / p n ). The matching network uses mean square error (MSE) as the loss function, which is defined as shown in formula (4.4):

[0070]

[0071] Among them, y i,j and represent positive and negative sample pairs respectively.

[0072] The final overall loss function is shown in formula (4.5):

[0073] L=λ1L cls +λ2L reg +λ3L mat (4.5)

[0074] Among them, λ1, λ2 and λ3 represent the weights corresponding to each part of the loss function, and the weight of each part of the loss function is initialized to 1.

[0075] The matching network combines the multi-scale fusion features of the template image and the candidate sample F c , spliced ​​in the channel dimension, and then passed through a convolution and three fully connected layers to obtain the matching score between the two. The matching score is element-wise multiplied with the cross-correlation feature map of the classification branch to obtain improved correlation features. The improved correlation features are passed through the convolution layer to obtain the improved classification credibility score. The region proposal network outputs the template prediction result O0 according to the improved classification credibility score.

[0076] like Figure 8 As shown in the figure, the candidate sample features generated by the RPN region proposal are the regression branch. The goal is to calculate the matching score of the fusion features input by the multi-scale fusion module, and multiply the matching score with the classification branch of the RPN to help the RPN obtain the corrected classification confidence score. In this way, the matching degree of classification and regression information is enhanced.

[0077] The output of the matching network is converted into a matching score of size 25×25×1, and then the matching score is element-wise multiplied with the cross-correlation feature map of the classification branch to filter out ground interference by suppressing false positive locations. Finally, the improved correlation features are passed through the convolutional layer to obtain the improved classification confidence score.

[0078] The region proposal network combines the information of the regression branch and the classification branch according to the corrected classification confidence score and operates together to predict the target location, thereby alleviating the mismatch problem.

[0079] The multi-scale fusion features of the search image and the auxiliary template in the template buffer pool are fused and input into the region proposal network, and the region proposal network outputs the auxiliary prediction result O i ; The template buffer pool includes several auxiliary templates, one of which is selected for use, and the template image is initially used as the auxiliary template;

[0080] The initial auxiliary template of the template buffer pool is a template image.

[0081] The total number of auxiliary templates in the template buffer pool is fixed. When the number of auxiliary templates reaches an upper limit, new auxiliary templates are added and the earliest auxiliary template is removed.

[0082] like Figure 1 As shown in FIG. 1 , the multi-scale fusion features of the search image and the auxiliary template in the template buffer pool are fused and input into the region proposal network, and the region proposal network outputs the auxiliary prediction result O i , no matching network is required.

[0083] The template buffer pool includes multiple auxiliary templates. Initially, there is only one auxiliary template, namely the first frame template image. As new auxiliary templates are added, the first template sample in the template pool is deleted according to the first-in-first-out rule after reaching the upper limit. For example, if the upper limit of the template buffer pool is 5, the first frame template image is the first one, and 4 auxiliary images can be added later. When the sixth auxiliary image is added, the first first frame template image is deleted.

[0084] Auxiliary prediction results i The final tracking result O is obtained by fusing it with the template prediction result O0;

[0085] After the two are fused, the final tracking result O image is obtained, that is, based on the search image, the object to be tracked is identified and framed.

[0086] The final tracking result O is evaluated, and if qualified, it is added to the template buffer pool as an optional auxiliary template;

[0087] The specific method for evaluating the final tracking result O is: N N represents all templates in the template pool APCE The mean is if and only if N APCE >T N When , the current candidate sample feature map is added to the template buffer pool Γ,

[0088] F min is the minimum response value, F w,h is the response value at the coordinate index (w,h), F max is the maximum response value.

[0089] Select the template with the smallest similarity to the template image from the template buffer pool as the auxiliary template.

[0090] like Fig. 9 As shown in the figure, by matching the template and the search area similarity, the twin network tracker achieves good tracking results. However, its template often selects the target features of the first frame of the video sequence, which greatly improves the efficiency of the tracker. However, in the actual tracking process, the scene is often more complex and changeable. For example, there will be interference during the movement of the target, the target disappears and reappears in the tracking field of view, etc. At this time, the actual target template is quite different from the initial frame template T0. If an unchanging template is used, it is easy to cause tracking failure. At this time, it is necessary to dynamically update the template during the tracking process to solve the problem of tracking failure caused by a single template.

[0091] The traditional template update method is to determine whether the matching degree between the target template of the next frame and the best candidate template tracked by the algorithm of the current frame is greater than a set threshold. This method only uses a fixed threshold to determine whether to update the template. If the matching degree of the best candidate sample is greater than the threshold, the sample feature will be used as the template for the subsequent frame. This operation causes the target template of the tracker to deviate after a long period of tracking, that is, the template drift problem occurs. To solve this problem, a template buffer pool Γ is established and continuously updated during the tracking process. According to experience, Γ=5 is set. This operation can not only effectively suppress the accumulation of tracking errors and template drift, but also make the algorithm more robust.

[0092] Evaluate whether the current frame prediction result is reliable and whether to update Γ by average peak correlation energy (APCE) and maximum response value F max This is because using the maximum response value alone to obtain tracking results is one-sided. max The value of must be the peak value of the response, and F max It is not necessarily the maximum response of the matching result. From the above analysis, we can see that the peak response corresponding to the actual position of the target in the current frame may be different from the F max Considering the relationship between the true position and the response map, the design N APCE To determine whether to update Γ, it is used to measure the confidence of the candidate sample response output by the classification branch. The specific definition is shown in formula (4.1):

[0093]

[0094] Among them, F min is the minimum response value, F w,h is the response value at the coordinate index (w,h). N APCEIt can reflect the degree of change in the response graph and the confidence of the candidate sample. When the candidate sample has a greater reliability, N APCE As the value increases, the response feature map output by the classification branch becomes smoother; otherwise, if the candidate sample is unreliable, that is, the object to be tracked is blocked, lost, or the current search area is the background, then N APCE As the value decreases, multiple peaks appear in the response characteristic graph.

[0095] This chapter uses formula (4.1) to update the template buffer pool and uses the variable T N N represents all templates in the template pool APCE The mean is if and only if N APCE >T N When the current candidate sample feature map is added to Γ, the reliability of the template is guaranteed by this operation, thus establishing a valid Γ. In addition, this also enables Γ to characterize the different states of the object to be tracked as much as possible. Due to the limitation of the size of Γ, the algorithm in this chapter updates the elements in Γ in a first-in-first-out manner.

[0096] When the target is out of view or completely blocked for a long time during its motion, selecting a single template in Γ for tracking may also result in tracking failure. To avoid this problem and to compensate for the template degradation problem, the algorithm in this chapter selects a dual template strategy, that is, the first frame target template is not discarded, and the best template in Γ is selected as the auxiliary template for tracking.

[0097] The strategy for selecting the auxiliary template is shown in formula (4.2). By calculating the similarity measure between the first frame template feature and the elements in Γ, the element with the smallest similarity (i.e., the largest difference) is selected as the auxiliary template.

[0098]

[0099] Among them, T i represents the auxiliary template, T0 represents the initial frame template. This chapter calculates the similarity between the two through related operations.

[0100] And when updating the template buffer pool image, only the part identified in the final tracking result O, that is, the part selected by the frame, is added to the template buffer pool, instead of adding irrelevant parts such as the background in the entire final tracking result O image.

[0101] After selecting the best auxiliary template, in order to avoid template contamination, the two template features are respectively used to perform subsequent related operations with the current frame image and the RPN prediction candidate results, and the prediction results of the two templates are fused to obtain the final tracking result. The template update strategy adopted can not only supplement the original template features, but also greatly improve the training efficiency of the tracker.

[0102] The template update strategy is shown in Table 1

[0103] Table 4.1 Template update strategy

[0104]

[0105] Repeat the above steps until the search image tracking and recognition of all frames are completed.

[0106] Repeat the above steps to track and identify all subsequent frames, and select a new image from the template buffer pool as the auxiliary template each time until all images in the entire video are tracked and selected.

[0107] like Fig.10 and 11 As shown in Figure 3, the overall tracking results of the algorithm and six popular tracking methods are the success curve and accuracy curve respectively.

[0108] In the accuracy curve, the precision score of the proposed algorithm is 0.819, which is higher than the accuracy score obtained by SiamCAR (0.809), and 1.8% and 4.6% higher than the SiamRPN++ (0.801) and DaSiamRPN (0.773) methods. In the success curve, the AUC of the proposed method is 0.622, which is 2% and 4% higher than SiamRPN++ (0.602) and DaSiamRPN (0.582), and higher than 0.619 obtained by SiamCAR. The experimental results show that the algorithm in this chapter has good performance. This is mainly because the proposed method predicts the relationship between the features of the classification branch and the regression branch, and adds a template update module, which is more sensitive to the appearance changes of the target. Therefore, the proposed algorithm can effectively deal with target occlusion and drastic appearance changes in complex scenes, and achieve robust tracking.

[0109] The UAVXU dataset contains 20 video sequences. This method is compared with 6 tracking algorithms on the UAVXU dataset. The 6 methods are KCF, SRDCF, SiamRPN, DaSiamRPN, SiamRPN++, and SiamCAR. Among them, KCF and SRDCF are correlation filter tracking methods, and the others are deep learning tracking methods. The accuracy of the tracker when the Euclidean distance threshold is 20 pixels and the success rate of the tracker when the IOU is greater than the threshold of 0.5 are selected as indicators for comparing the performance of the algorithm. The experimental results are shown in Table 4.2.

[0110] Table 4.2 Experimental results of all methods on the UAVXU dataset

[0111]

[0112] As shown in Table 4.2, the method in this chapter achieves good tracking results and ranks first among all tracking methods. Therefore, it is proved that the method proposed in this chapter can effectively cope with the challenges faced by drone aerial photography targets in different scenarios. Due to the use of the template update mechanism, the algorithm in this chapter can effectively deal with the problems caused by the change of target appearance, thereby ensuring the performance of the algorithm.

Claims

1. A UAV target tracking method based on twin network and template updating, characterized in that: The following steps are involved: Get the video of the object to be tracked and divide it into frames. Cut out the image of the object to be tracked from the first frame as the template image, and the complete image of the subsequent frames as the search image. The template image and the second frame search image are input into the basic deep residual network respectively to extract their respective block3 and block4 features. Then, the respective block3 and block4 features are fused through the multi-scale fusion module to obtain the multi-scale fusion features of the template image and the multi-scale fusion features of the search image. The multi-scale fusion features of the template image and the search image are fused and input into the region proposal network. The regression branch of the region proposal network generates candidate samples F c , the multi-scale fusion features of the template image and the candidate sample F c Input the matching network, output the matching score, and use the matching score to correct the classification branch of the region proposal network. The region proposal network outputs the template prediction result O0; The multi-scale fusion features of the search image and the auxiliary template in the template buffer pool are fused and input into the region proposal network, and the region proposal network outputs the auxiliary prediction result O i ; The template buffer pool includes several auxiliary templates, one of which is selected for use, and the template image is initially used as the auxiliary template; Auxiliary prediction results i The final tracking result O is obtained by fusing it with the template prediction result O0; The final tracking result O is evaluated, and if qualified, it is added to the template buffer pool as an optional auxiliary template; Repeat the above steps until the search image tracking and recognition of all frames are completed.

2. According to claim 1, a UAV target tracking method based on twin network and template update is characterized in that: The basic deep residual network is ResNet-50.

3. The method for tracking unmanned aerial vehicle targets based on twin networks and template updating according to claim 1 is characterized in that: Specifically, the multi-scale fusion module inputs the feature map of block 3 into four branches respectively, and connects the outputs of the four branches in the channel dimension; The feature map output by block4 is refined and adjusted through convolution and deconvolution to achieve feature fusion. The spatial feature block3 and the deep semantic feature block4 are fused element by element to obtain multi-scale fusion features; After feature fusion, the number of channels is reduced through 1×1 convolution to obtain the final multi-scale fusion features.

4. The method for tracking unmanned aerial vehicle targets based on twin networks and template updating according to claim 1 is characterized in that: The specific contents of the four branches are: branch one, the input is sequentially subjected to 1×1 convolution, 5×5 convolution and 7×7 convolution; branch two, the input is sequentially subjected to 1×1 convolution and 3×3 convolution; branch three, the input is sequentially subjected to 3×3 average pooling and 1×1 convolution; Branch four performs a 1×1 convolution on the input.

5. The method for tracking unmanned aerial vehicle targets based on twin networks and template updating according to claim 1 is characterized in that: The initial auxiliary template of the template buffer pool is a template image.

6. The method for tracking unmanned aerial vehicle targets based on twin networks and template updating according to claim 1 or 5, characterized in that: The total number of auxiliary templates in the template buffer pool is fixed. When the number of auxiliary templates reaches an upper limit, new auxiliary templates are added and the earliest auxiliary template is removed.

7. The method for tracking unmanned aerial vehicle targets based on twin networks and template updating according to claim 4 is characterized in that: Use two 3×3 convolution kernels to replace the 5×5 convolution kernel; convert the 3×3 convolution into a 1×3 feature output after a 3×1 convolution, and then a 1×3 convolution to get a 1×1 feature output.

8. The method for tracking unmanned aerial vehicle targets based on twin networks and template updating according to claim 1 is characterized in that: The matching network combines the multi-scale fusion features of the template image and the candidate sample F c , spliced ​​in the channel dimension, and then passed through a convolution and three fully connected layers to obtain the matching score between the two. The matching score is element-wise multiplied with the cross-correlation feature map of the classification branch to obtain improved correlation features. The improved correlation features are passed through the convolution layer to obtain the improved classification credibility score. The region proposal network outputs the template prediction result O0 according to the improved classification credibility score.

9. The method for tracking unmanned aerial vehicle targets based on twin networks and template updating according to claim 1, characterized in that: The specific method for evaluating the final tracking result O is: N N represents all templates in the template pool APCE The mean is if and only if N APCE >T N When , the current candidate sample feature map is added to the template buffer pool Γ, F min is the minimum response value, F w,h is the response value at the coordinate index (w,h), F max is the maximum response value.

10. The method for tracking unmanned aerial vehicle targets based on twin networks and template updating according to claim 1, characterized in that: Select the template with the smallest similarity to the template image from the template buffer pool as the auxiliary template.

Images