Adaptive template updating method and system for target tracking
By employing an adaptive template update method, utilizing multi-dimensional confidence evaluation and adversarial cleanup, the template adaptability and purity issues in the Siamese tracker are resolved, achieving stable and rapid recovery in complex scenarios.
Patent Information
- Application Number
- CN202511834293.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-08
- Publication Date
- 2026-03-06
AI Technical Summary
Traditional Siamese trackers suffer from several problems: fixed templates are difficult to adapt to changes in target appearance; simple template update strategies are prone to introducing errors; errors tend to accumulate during long-term tracking; and background noise contaminates the purity of the template.
An adaptive template update method is adopted, which dynamically updates the template through multi-dimensional confidence assessment, adversarial cleanup and memory enhancement mechanisms, removes background noise, maintains template purity and backtracks historical effective templates.
It improves the stability and accuracy of target tracking, especially in complex backgrounds and long-term occlusion scenarios, enabling rapid recovery and high-precision tracking.
Smart Images
Figure CN121617014A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision and pattern recognition technology, and in particular to an adaptive template update method and system for target tracking. Background Technology
[0002] Target tracking is one of the core tasks of computer vision, aiming to locate the position and state (such as scale and pose) of a specific target through continuous video frames. In recent years, trackers based on Siamese networks have become mainstream due to their advantages of end-to-end training and real-time performance. The core idea is to extract target template features in the first frame (initialization frame), and generate a response map in subsequent frames by cross-correlation between the template and the features of the search region. The peak of the response corresponds to the target position.
[0003] However, traditional Siamese trackers and existing template update algorithms have many technical bottlenecks, including the difficulty of adapting fixed templates to changes in target appearance, the ease with which simple template update strategies can introduce errors, the tendency for errors to accumulate during long-term tracking, and the contamination of template purity by background noise. Summary of the Invention
[0004] One of the objectives of this invention is to provide an adaptive template update method and system for target tracking, in order to solve problems such as fixed templates being difficult to adapt to changes in target appearance, simple template update strategies being prone to introducing errors, errors being prone to accumulation during long-term tracking, and background noise contaminating template purity.
[0005] This invention is achieved through the following technical solution: an adaptive template update method for target tracking, comprising:
[0006] Initial template features are extracted from the target region in the first frame of the video, and a template memory pool is initialized.
[0007] For each subsequent frame, a tracking and adaptive update loop is performed until the end of the video, including:
[0008] Extract the search region features of the current frame, perform cross-correlation calculation between the search region features and the current template to generate a cross-correlation response map, and predict the target position based on the cross-correlation response map;
[0009] Based on the cross-correlation response map, the target location, and historical tracking information, the comprehensive confidence level of the current frame is generated through multi-dimensional evaluation.
[0010] Based on the target location, the current target features are extracted, and the current target features are input into a pre-trained generator G for adversarial purification to obtain the purified target features.
[0011] The template memory pool is updated based on the overall confidence level.
[0012] Based on the purified target features and the comprehensive confidence level, historical templates from the template memory pool are fused to obtain a memory reference template, and a new generation template for tracking in the next frame is generated.
[0013] Furthermore, the step of generating the comprehensive confidence level of the current frame based on the cross-correlation response map, the target location, and historical tracking information through multi-dimensional evaluation includes:
[0014] The ratio of the peak value to the mean value of the response graph is calculated to obtain the response characteristic index. The formula is:
[0015] ,
[0016] in, For cross-correlation response diagrams, For the peak value of the response graph, The mean of the response plot;
[0017] Calculate the average intersection-union ratio (IoU) of the predicted target bounding box in the current frame with the target bounding boxes in the previous three frames to obtain the geometric feature index. The formula is:
[0018] ,
[0019] ,
[0020] in, The current frame bounding box. The bounding boxes for the first k frames;
[0021] Calculate the semantic feature index of the feature map of the search region. And determine the semantic feature purity based on semantic feature indicators. The formula is:
[0022] ,
[0023] ,
[0024] in, Let C represent the pixel probability distribution of the i-th channel in the feature map, and C be the number of feature channels. This represents the maximum value of the information entropy of the feature maps of all samples in the training set or benchmark dataset;
[0025] For the response feature index Geometric feature index and semantic feature purity We perform weighted fusion to obtain the overall confidence level. The formula is:
[0026] ,
[0027] in, , , These are weights that can be adaptively learned from training data. and .
[0028] Furthermore, in the weighted fusion, the weights of each indicator are determined by learnable parameters. Obtained by normalization using the softmax function, the formula is:
[0029] , , ,
[0030] Among them, learnable parameters .
[0031] Further, updating the template memory pool based on the comprehensive confidence level includes:
[0032] If the overall confidence level is higher than the preset high confidence level threshold, the purified target features, the overall confidence level, and the time step information of the current frame are stored as a new template in the template memory pool; when the capacity of the template memory pool reaches the upper limit, the template with the longest storage time and the lowest confidence level is removed.
[0033] Furthermore, the fusion of historical templates from the template memory pool to obtain a memory reference template includes:
[0034] Calculate the cosine similarity between the purified target features and historical templates in the template memory pool; based on the cosine similarity and a time decay weight, select the top K historical templates from the template memory pool for weighted fusion to generate a memory reference template, using the following formula:
[0035] ,
[0036] ,
[0037] in, For the purified target characteristics, These are historical templates in the template memory pool. For cosine similarity, For memorization reference template, is the weight of the historical template, and k is the time difference.
[0038] Furthermore, the generation of the next-generation template for tracking in the next frame includes:
[0039] Based on the comprehensive confidence level, the fusion weight of the current purified feature is calculated using a monotonically increasing function. This weight is then used to perform a weighted summation of the purified target feature and the memory reference template to generate the next-generation template. The formula is as follows:
[0040] ,
[0041] ;
[0042] in, It is a monotonically increasing function. For the new generation of templates, For the purified target characteristics, For memorization, a reference template is provided, and s represents the overall confidence level. As a high confidence threshold, This is a low confidence threshold.
[0043] Further, the step of extracting current target features based on the target location and inputting the current target features into a pre-trained generator G for adversarial cleanup processing to obtain cleaned target features includes:
[0044] The current target features are input into a pre-trained generator G, which is a U-Net structure including an encoder, a decoder, and skip connections.
[0045] The encoder consists of multiple convolutional layers, with the number of channels increasing layer by layer, and each convolutional layer is followed by a batch normalization (BatchNorm) and LeakyReLU activation function. The decoder consists of multiple transposed convolutional layers, with the number of channels decreasing layer by layer until it matches the number of input channels, and each convolutional layer is followed by a batch normalization (BatchNorm) and ReLU activation function. The current target feature, after being processed by the encoder and decoder, is used to generate a cleaned feature with the same dimension as the input feature through the Tanh activation function of the generator's output layer.
[0046] Furthermore, the generator G is trained via an adversarial training method that includes a local discriminative mechanism, including:
[0047] Step S1: Select video frames with clear targets and simple backgrounds from the tracking dataset as clean target sub-images, and extract rectangular regions containing contextual information based on the tracking annotation boxes;
[0048] Step S2: Based on the clean target sub-map, add adversarial noise or natural interference noise to generate a noisy target sub-map;
[0049] Step S3: Use a pre-trained Siamese tracker feature extraction network to extract clean features and noisy features from the clean target sub-image and the noisy target sub-image, respectively;
[0050] Step S4: Fix the generator parameters, use clean features as positive samples and clean features output by the generator as negative samples, and train the discriminator to accurately distinguish between real clean features and generated features.
[0051] Step S5: Fix the discriminator parameters, train the generator to minimize the discriminator's accuracy in judging clean features, and at the same time constrain the consistency between clean features and true clean features through L1 reconstruction loss.
[0052] Repeat steps S4-S5 until the model converges to obtain a generator G that can effectively remove background noise from features.
[0053] Another aspect of the present invention provides an adaptive template update system for target tracking, for implementing an adaptive template update method for target tracking as described in any of the above embodiments, the system comprising:
[0054] The Siamese tracker is used to extract features of the target and the search region, including:
[0055] Feature extraction module: used to extract features from the input image;
[0056] Cross-correlation calculation module: used to extract the search region features of the current frame, and perform cross-correlation calculation between the search region features and the current template to generate a cross-correlation response map;
[0057] Tracking head: used to predict the target location based on the cross-correlation response map;
[0058] An adaptive template update network, integrated with the base Siamese tracker, is used to dynamically generate templates and assist in target localization, including:
[0059] Multi-dimensional confidence assessment module: used to generate the comprehensive confidence of the current frame through multi-dimensional assessment based on the cross-correlation response map, the target location and historical tracking information;
[0060] Adversarial template cleanup module: used to extract current target features based on the target location, and perform adversarial cleanup on the current target features to obtain cleaned target features;
[0061] The memory-enhanced dynamic update module is used to update the template memory pool based on the comprehensive confidence level, and to fuse historical templates from the template memory pool based on the purified target features and the comprehensive confidence level to obtain a memory reference template, and generate a new generation template for tracking in the next frame.
[0062] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0063] 1. This invention comprehensively considers matching degree, stability and feature discrimination through confidence evaluation, which can accurately identify low-quality tracking frames caused by occlusion, blur or background interference, avoid template pollution from the source, accurately avoid pollution of low-quality frames, and combined with dynamic update strategy, ensure that the template always maintains a high-precision expression of the target appearance, and shows higher stability in various challenging scenarios.
[0064] 2. This invention uses a memory enhancement mechanism to enable the system to quickly recall and call up high-quality templates from before the occlusion in extreme cases such as the target being occluded for a long time or reappearing after being out of view, thus achieving fast and accurate tracking and recovery and solving the problem of error accumulation.
[0065] 3. The anti-noise purification module of this invention can effectively suppress the mixing of background noise, improve the purity and discriminability of the template, and make tracking more stable in scenarios with complex backgrounds and interference from similar objects.
[0066] 4. The adaptive template update network DCMUNet in this invention can be used as a plug-and-play module and can be seamlessly integrated into various mainstream Siamese trackers such as SiamFC and SiamRPN++ without modifying their backbone network. This facilitates rapid deployment and application in actual projects and has high practical value.
[0067] 5. This invention constructs a fully functional and collaborative target tracking system by deeply integrating the adaptive template update network DCMUNet with the Siamese tracker. In this system, the multi-dimensional confidence evaluation module adaptively fuses multi-source information through learnable weight parameters, thus providing reliable tracking quality judgment; the adversarial template cleanup module achieves end-to-end feature denoising using a pre-trained U-Net generator, effectively removing background noise and improving template discriminability; and the memory-enhanced dynamic update module, based on a cosine similarity attention mechanism and a time decay weighted algorithm, ensures the long-term preservation and intelligent utilization of historical effective information. Attached Figure Description
[0068] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:
[0069] Figure 1 The flowchart illustrates an adaptive template update method for target tracking provided by this invention.
[0070] Figure 2This is a flowchart illustrating the calculation process of the multi-dimensional confidence assessment module in this invention.
[0071] Figure 3 This is a flowchart illustrating the calculation process of the memory-enhanced dynamic update module in this invention.
[0072] Figure 4 This is a flowchart illustrating the reasoning process of the antagonistic purification module in this invention.
[0073] Figure 5 This is a flowchart illustrating the training process of the adversarial purification module in this invention. Detailed Implementation
[0074] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0075] The core task of target tracking is to accurately locate the position and state of a specific target in consecutive video frames. Tracking methods based on Siamese networks have become the mainstream approach due to their end-to-end training mechanism and efficient inference performance. Their basic principle is as follows: In the initial frame, the target's appearance features are extracted as a template; in each subsequent frame, the template features are cross-correlated with candidate search region features to generate a response map; finally, the peak position in the response map corresponds to the predicted position of the target in the current frame.
[0076] However, traditional Siamese trackers have the following key problems in practical applications:
[0077] Regarding the difficulty of adapting to changes in target appearance using fixed templates, early Siamese trackers (such as SiamFC) used fixed initial templates, which could not cope with dynamic changes in the target's appearance (such as changes in lighting, scale scaling, pose deformation, partial occlusion, etc.). For example, when a target moves from sunlight into a shadow area, its color characteristics will change significantly, and the fixed template will cause tracking drift due to feature mismatch.
[0078] Simple template update strategies are prone to introducing errors. Existing template update methods (such as UpdateNet) trigger template updates through a single confidence index (such as the peak value of the response map), which has the following drawbacks: (1) When evaluating confidence, the tracking quality is judged solely by the peak value of the response map, ignoring key information such as bounding box stability and feature discrimination, resulting in feature pollution of the template by low-quality frames (such as blurry or partially occluded frames); (2) The same update weight is applied to all frames that meet the threshold, and it is impossible to dynamically adjust according to the degree of change of the target (such as strong updates for rapid deformation and weak updates for slight changes).
[0079] In terms of the tendency for errors to accumulate during long-term tracking, most existing methods rely solely on updating the template from the previous frame, lacking a long-term memory mechanism. When the target experiences prolonged occlusion (e.g., being completely occluded by an obstacle for more than 50 frames), the template has accumulated a large number of erroneous features, making it impossible to resume effective tracking when the target reappears; and historical effective templates (such as high-confidence templates before occlusion) are not saved, missing the opportunity for backtracking and correction.
[0080] Regarding the purity of the template due to background noise contamination, in complex backgrounds (such as when the target and background colors are similar, or when there is interference from multiple targets), the target features extracted in the current frame are easily mixed with background information. For example, when tracking a white drone, if there is a white billboard in the background, the target features will contain billboard noise, leading to a decrease in template discriminability.
[0081] In view of the shortcomings of the existing technology, the present invention aims to solve the following problems:
[0082] (1) How to evaluate quantitative tracking quality through multiple dimensions and avoid low-quality frames from polluting the template;
[0083] (2) How to design a dynamic update strategy to adaptively adjust the update intensity according to the degree of change in the target;
[0084] (3) How to introduce a long-term memory mechanism to ensure stable tracking even after long-term occlusion;
[0085] (4) How to remove background noise from target features and improve template purity.
[0086] Example 1
[0087] To address the technical bottlenecks of existing technologies, this embodiment proposes an adaptive template update method for target tracking. By employing multi-dimensional confidence assessment, it overcomes the limitations of a single metric, accurately identifying low-quality frames and thus reducing the risk of template contamination. A dynamic template update strategy adaptively adjusts the update intensity based on tracking quality, updating conservatively when the target is stable and proactively when it changes drastically. A long-term memory mechanism is utilized to backtrack historically valid templates through a memory pool, improving tracking robustness in long-term occlusion scenarios. Furthermore, adversarial cleanup removes background noise, enhancing template discriminative power.
[0088] Figure 1 The flowchart of an adaptive template update method for target tracking in this embodiment is shown. As can be seen from the figure, this embodiment includes the following steps:
[0089] Step S100: Extract initial template features based on the target region of the first frame of the video and initialize a template memory pool.
[0090] Extract the initial template features of the target from the first frame of the video. And initialize the template memory pool:
[0091] ,
[0092] in This represents the initial confidence level.
[0093] Step S200: Extract the search region features of the current frame, perform cross-correlation calculation between the search region features and the current template to generate a cross-correlation response map, and predict the target position based on the cross-correlation response map.
[0094] For frame t, the search region feature x is extracted through the Siamese tracker backbone network, and the search region feature x is compared with the current template. Cross-correlation generates a response map, predicting the target location and initial response peak.
[0095] Specifically, for the template feature z, if it is the first frame image, it will be obtained through the feature extraction network of the Siamese tracker based on the template region of the first frame; for subsequent frames, it will be obtained through the memory-enhanced dynamic update module of the adaptive template update network DCMUNet.
[0096] For the search region feature x, it is obtained through the feature extraction network of the Siamese tracker based on the search region of the current frame. The feature extraction network of the Siamese tracker mainly includes common image feature extraction networks such as AlexNet, ResNet, MobileNet, and ShuffleNet.
[0097] The cross-correlation between the search region feature x and the template feature z is calculated to obtain the cross-correlation response map R. This cross-correlation response map R is then input into the tracking head of the Siamese tracker to obtain the output of the Siamese tracker, which mainly includes the target bounding box B. t Confidence level, etc.
[0098] Step S300: Based on the cross-correlation response map, the target location, and historical tracking information, generate the comprehensive confidence level of the current frame through multi-dimensional evaluation.
[0099] The response characteristic index s is calculated using the cross-correlation response graph R. peak The semantic feature index s is calculated by searching the feature map x of the region. entropy Furthermore, the semantic feature purity s is obtained. clean Through target bounding box B t The target bounding boxes B of the first three frames saved in the historical frame output results t-1 B t-2 B t-3 The geometric feature index s was calculated. iou Finally, the response feature index s is fused using weighted methods. peak Semantic feature purity s clean and geometric feature indexes s iou The overall confidence level s is obtained.
[0100] Step S400: Extract the current target features based on the target location, and input the current target features into a pre-trained generator G for adversarial cleanup processing to obtain the cleaned target features.
[0101] Based on the output of the Siamese tracker, through the target bounding box The target region sub-map can be obtained, and then the current target features can be obtained through the Siamese tracker backbone feature extraction network. .
[0102] Next, the current target features Input generator G, remove current target features The background noise in the sample is used to obtain the purified feature G(z). curr This improves the purity of the features.
[0103] Step S500: Update the template memory pool based on the comprehensive confidence level;
[0104] Step S600: Based on the purified target features and the comprehensive confidence level, the historical templates from the template memory pool are fused to obtain a memory reference template, and a new generation template for tracking in the next frame is generated.
[0105] First, calculate the purification characteristic G(z). curr ) and template memory pool Chinese memory template z t-k cosine similarity sim (k>0) k Then select sim k The memory reference template z is obtained by weighted fusion of the top K largest memory templates. mem Next, the weight β(s) is calculated using the comprehensive confidence level s. Finally, the weight β(s) and the memory reference template z are used to calculate the final weight. mem Then you can get the new template z t Use the new template z t Update the template z. Simultaneously, update the template memory pool. When the cleansing feature G(z) is obtained... curr After that, determine whether the overall confidence level s is greater than the preset confidence threshold. If it is greater, then (G(z)) curr The templates ), s, t) are stored as new templates in the template memory pool; when the number of templates in the template memory pool exceeds the maximum capacity N, the oldest low-confidence template is removed; if the overall confidence s is less than the preset confidence threshold, the template memory pool is not updated.
[0106] Example 2
[0107] This embodiment provides an adaptive template update system for target tracking, including an adaptive template update network DCMUNet and a Siamese tracker. The Siamese tracker can be any of the mainstream Siamese trackers such as SiamFC, SiamRPN++, SiamBAN, and SiamCAR. A Siamese tracker generally consists of a feature extraction network, a cross-correlation calculation module, and a tracking head. A detailed description follows.
[0108] The Siamese tracker is used to extract features of the target and the search region, including:
[0109] Feature extraction module: Used to extract features (features with target discriminative properties) from the input image;
[0110] Cross-correlation calculation module: used to extract the search region features of the current frame, perform cross-correlation calculation between the search region features and the current template to generate a cross-correlation response map; by performing similarity matching between template features and search region features, a cross-correlation response map is generated, and the peak position on the response map indicates the most likely location of the target;
[0111] Tracking head: used to predict the target position based on the cross-correlation response map; and to regress the target's precise bounding box and its initial confidence level based on the cross-correlation response map.
[0112] The Adaptive Template Update Network (DCMUNet), integrated with the underlying Siamese tracker, is used to dynamically generate templates and assist in target localization, including:
[0113] Multi-dimensional confidence assessment module: used to generate the comprehensive confidence of the current frame through multi-dimensional assessment based on the cross-correlation response map, the target location and historical tracking information;
[0114] Adversarial template cleanup module: used to extract current target features based on the target location, and perform adversarial cleanup on the current target features to obtain cleaned target features;
[0115] The memory-enhanced dynamic update module is used to update the template memory pool based on the comprehensive confidence score, and to fuse historical templates from the template memory pool based on the purified target features and the comprehensive confidence score to obtain a memory reference template, and generate a new generation template for tracking in the next frame.
[0116] The following is a detailed description of each module in the Adaptive Template Update Network (DCMUNet).
[0117] Multi-dimensional confidence assessment module, such as Figure 2 As shown. This module takes the output of the Siamese tracker for the current frame and the output of previous frames as input, and outputs a comprehensive confidence score 's'. This module constructs a comprehensive confidence score by fusing response features, geometric features, and semantic features to accurately evaluate the reliability of the tracking results for the current frame. Specifically,
[0118] Calculate response characteristic indices separately Geometric feature index Semantic feature purity , and weighted and fused into a comprehensive confidence level s. Response feature index The ratio of the peak value to the mean value of the cross-correlation response plot reflects the degree of matching between the target and the template. The formula is:
[0119] ,
[0120] in, This is the cross-correlation response map between the template feature map z and the search region feature map x. For the peak value of the response graph, The mean of the response plot. Response characteristic indicators. The larger the value, the higher the match rate.
[0121] Geometric feature index The average Interchange of Union (IOU) between the predicted bounding box of the current frame and the bounding boxes of the previous three frames reflects the stability of the tracking. The formula is as follows:
[0122] ,
[0123] ,
[0124] in, The current frame bounding box. For the bounding boxes of the first k frames, geometric feature indices The larger the value, the more stable the bounding box.
[0125] Semantic feature index Let x be the information entropy of the feature map of the search region, reflecting the discriminative power of the features. The lower the entropy value, the less uncertainty there is, and the more significant the difference between the target and the background. The formula is:
[0126] ,
[0127] in, Let C be the pixel probability distribution of the i-th channel in the feature map (obtained through normalization), and C be the number of feature channels. To ensure a positive correlation between semantic feature metrics and reliability, semantic feature purity is defined. :
[0128] ,
[0129] in, This represents the maximum value of the information entropy of the feature maps of all samples in the training or benchmark dataset. It is a priori determined constant used to unify the entropy scale across different scenarios and samples. When semantic feature purity... At that time, semantic feature indicators This indicates that the target and background distribution in the feature map is extremely unbalanced (highly discriminative), the features are pure, and the features have high reliability. When the semantic feature purity... At that time, semantic feature indicators This indicates that the target and background are evenly distributed in the feature map (there is no significant difference between the target and the background), the features are chaotic, and the feature reliability is low.
[0130] By weighting and combining the above indicators, the overall confidence level can be obtained. :
[0131] ,
[0132] in, , , These are weights that can be adaptively learned from training data. and Specifically, three unconstrained learnable parameters are introduced. They are mapped to weights that satisfy the constraints by using softmax normalization:
[0133] .
[0134] The reasoning process of the adversarial purification module is as follows: Figure 4 As shown. By using the trained generator G to remove noise from the current target feature zcurr, the cleaned feature G(z) can be obtained. curr The adversarial template purification module of this invention improves template purity by generating an adversarial network (GAN) to remove background noise from target features.
[0135] Specifically, the generator G of the adversarial template cleansing module adopts a U-Net structure (both input and output channels are C, consistent with the feature dimension), preserving details through skip connections. Specifically, its encoder consists of 3 convolutional layers with input channels of C, 2C, and 4C, and output channels of 2C, 4C, and 8C, respectively. Each layer is followed by BatchNorm and LeakyReLU. The encoder also consists of 3 transposed convolutional layers with input channels of 8C, 4C, and 2C, and output channels of 4C, 2C, and C, respectively. Each layer is followed by BatchNorm and ReLU. The output layer outputs cleaned features through the Tanh activation function. ,in, It has the same dimension as the input features.
[0136] The discriminator D in the adversarial template cleanup module adopts a PatchGAN structure, outputting a probability map of features belonging to the "clean target". Specifically, it contains four convolutional layers with input channels of C, 2C, 4C, and 8C, and output channels of 2C, 4C, 8C, and 8C, respectively. Each layer is followed by BatchNorm and LeakyReLU. The output layer mainly consists of a 1×1 convolution and a sigmoid activation function. The 1×1 convolution has 8C input channels and 1 output channel. The output of the output layer is a 3×3 probability map, where each pixel value... This represents the probability that the corresponding local region in the input feature map belongs to the "pure target feature". Specifically, the 3×3 probability map determines whether each of the nine non-overlapping local regions in the input feature map of the adversarial template cleanup module is a pure target feature. Compared with global judgment (such as outputting a single probability value), this local discrimination mechanism can capture noisy regions (such as background information mixed in by edges) in the feature more precisely, forcing the generator G to clean up local noise in a targeted manner.
[0137] The memory-enhanced dynamic update module is used to manage historical templates and dynamically generate new generation templates based on comprehensive confidence. This module builds a template memory pool and designs a dynamic update mechanism to realize the memory and utilization of historically valid templates.
[0138] Specifically, when constructing the template memory pool, the memory pool... Store the most recent N frames that satisfy the overall confidence level Templates with a high confidence threshold, such as 0.7. Template features This represents the overall confidence level for the corresponding frame. For time steps, This is the confidence threshold.
[0139] The weight of historical templates decreases as the time step increases to avoid excessive influence of distant templates on current updates. ,
[0140] in, The weights of the historical templates, where k is the time difference ( ), The attenuation coefficient (e.g.) (Control the decay rate).
[0141] When the target's appearance changes drastically (e.g., rapid rotation), an attention-based filtering mechanism selects the template most similar to the current features from the memory pool to assist in updating. First, the current cleansing features are calculated. With memory template cosine similarity Then, the top K templates with the highest cosine similarity (e.g., K=2) are selected and weighted and fused into a memory reference template. . Specifically,
[0142] ,
[0143] ;
[0144] New template Based on current purification characteristics With memory reference template Fusion generation, weight The overall confidence level s is dynamically adjusted and can be expressed by the following formula:
[0145] ,
[0146] in, It is a monotonically increasing function:
[0147] ,
[0148] when When the confidence level is low, it should not be used. Update template; when When, it indicates a medium confidence level, using Weak update template; when When, it indicates a high level of confidence, using Forced template update. Among them, For a low confidence threshold, for example ; For a high confidence threshold, such as .
[0149] This invention employs a joint training strategy, pre-training and end-to-end jointly training the Siamese tracker feature extraction network, adversarial sanitization module, and multi-dimensional confidence evaluation module, respectively. Specifically, the Siamese tracker feature extraction network (such as AlexNet, ResNet, MobileNet, ShuffleNet, etc.) can be pre-trained using a general image dataset (such as ImageNet), or directly using pre-trained weights provided by PyTorch, enabling it to extract target-discriminating features from image sub-images. In this embodiment, the latter is chosen.
[0150] The training process for the adversarial purification module is as follows: Figure 5 As shown, the pre-training of the adversarial cleanup module involves two steps. First, a pre-trained Siamese tracker feature extraction network is used to extract clean features z from both the "clean target sub-image" and the "noisy target sub-image". clean and noisy features z curr , forming training pairs. Then, using z curr The generator G is trained to generate clean features, and the discriminator D is trained to distinguish z. clean and G(z) curr By alternating between training D with G fixed first, and then training G with D fixed, until both converge, G learns to generate a generator that can "deceive" D and approximate z. clean Its purification characteristics.
[0151] Clean target sub-images are selected from tracking datasets (such as GOT10k, LaSOT, and self-made datasets) and are free from interference. Specifically, they are rectangular sub-images containing contextual information obtained from frames in the original video that are "clear in target, simple in background, and free from blur, occlusion, or adversarial attacks," based on tracking bounding boxes. Noisy target sub-images are rectangular sub-images obtained by adding "adversarial noise" or "simulated interference noise" to the clean target sub-images. Adversarial noise includes generating minute noise (such as pixel-level perturbations, invisible to the human eye but causing feature deviation) on the clean target sub-image using adversarial attack algorithms such as FGSM and PGD. Natural interference noise includes blur, occlusion, and lighting changes that simulate real-world tracking scenarios.
[0152] During discriminator training, labels are assigned to each local patch of the input feature map. Specifically, if the input is a clean target feature (positive sample), all locations in the 3×3 probability map are labeled 1; if the input is a noisy feature (negative sample), the labels are dynamically assigned based on the noise region: patches covered by noise are labeled 0, and patches in clean regions are labeled 1. This label design enables the discriminator not only to distinguish between "pure noise" and "pure" features, but also to identify noisy regions in "partially contaminated" features, further improving the generator's purification accuracy.
[0153] Generator G and discriminator D are trained through minimax games:
[0154] ,
[0155] in, These are pure target features (extracted from manually labeled target regions during training). For the current features containing noise (from training with) Extraction after introducing noise interference into the same target area). The loss of the discriminator D in judging true pure features is... P represents the loss of the discriminator D in judging the pseudo-pure features generated by the generator G. data The true distribution, i.e., the pure target feature z clean The probability distribution of P; noise The noise distribution, i.e., the noise-containing feature z curr The probability distribution; It represents the mathematical expectation.
[0156] To enhance the fidelity of the generator G to the target features, in Adding an L1 regularization term yields the final loss function L. G :
[0157] ,
[0158] in, For weights, such as .
[0159] The multi-dimensional confidence evaluation module is pre-trained using labels such as "occluded / clear / blurred" from the tracking dataset to train a lightweight CNN evaluation model, enabling it to output reliable confidence scores based on response feature metrics, semantic feature metrics, geometric feature metrics, etc.
[0160] End-to-end joint training is used to collaboratively optimize the Siamese tracker and the adaptive template update network DCMUNet. All modules are integrated into a complete network and jointly trained on the tracked datasets (such as GOT10k, LaSOT, and custom datasets).
[0161] Real-world examples:
[0162] A local drone dataset was built to address the core challenges of drone tracking in real-world scenarios. Data collection scenarios covered urban building clusters, suburban open areas, mountainous regions, and complex low-altitude scenes (such as high-voltage lines and billboards), encompassing daytime, dusk, and nighttime lighting conditions, and including sunny, cloudy, and light fog weather. Drone types included consumer drones, medium-sized industrial drones, and micro drones. Drone movement modes included uniform straight-line flight, rapid directional changes, hovering and acceleration switching, and dive / stretch. Interference factors included occlusion and background obfuscation. Occlusion primarily included tree occlusion, building occlusion, occlusion by birds and other interfering objects, and partial occlusion, with the longest continuous occlusion reaching 30 frames. Background obfuscation mainly included objects of similar color (such as a white billboard and a white drone) and dynamic backgrounds (such as fluttering leaves).
[0163] The dataset contains 1200 video sequences, each ranging from 200 to 1000 frames in length, with a frame rate of 25 FPS. All sequences were manually annotated by professional annotators using bounding boxes, with an annotation accuracy error of ≤2 pixels.
[0164] This invention uses the SiamCAR network as a Siamese tracker as an example, and verifies the beneficial effects of the invention using metrics such as Success Rate (SR), Center Location Error (CLE), and Occlusion Recovery Rate (ORR). SR is defined as the proportion of frames with an Intersection over Union (IoU) ≥ 0.5 between the tracked results and the manually labeled ground truth; a higher value indicates higher overall tracker stability. CLE is defined as the Euclidean distance (in pixels) between the center coordinates of the tracked bounding box and the center coordinates of the ground truth; a lower value indicates higher localization accuracy. ORR is defined as the proportion of sequences with an IoU ≥ 0.5 within 10 frames after the target recovers visibility from complete occlusion (IoU = 0 for ≥ 5 frames); a higher value indicates higher robustness of the tracker after occlusion.
[0165] The experimental results are shown in Table 1. SiamCAR represents the selected Siamese tracker, SiamCAR-UpdateNet is the target tracking method using UpdateNet as the template update network, and SiamCAR-DCMUNet is the target tracking method based on the adaptive template update network DCMUNet designed in this invention. The results effectively demonstrate the effectiveness and practicality of this invention.
[0166] Table 1 Results of different tracking methods
[0167] method SR CLE (pixels) ORR SiamCAR 63.5% 6.3 48.1% SiamCAR-UpdateNet 68.6% 5.1 56.3% SiamCAR-DCMUNet 75.2% 2.6 68.7%
[0168] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An adaptive template update method for target tracking, characterized in that, The method comprises the following steps: extracting initial template features based on a target region of a first frame of a video, and initializing a template memory pool; performing a tracking and adaptive updating cycle for each subsequent frame until the end of the video, comprising: extracting search region features of the current frame, performing cross-correlation calculation on the search region features and the current template to generate a cross-correlation response map, and predicting a target position based on the cross-correlation response map; based on the cross-correlation response map, the target position and historical tracking information, generating a comprehensive confidence of the current frame through multi-dimensional evaluation; extracting current target features based on the target position, and inputting the current target features into a pre-trained generator G for adversarial purification processing to obtain purified target features; updating the template memory pool based on the comprehensive confidence; based on the purified target features and the comprehensive confidence, fusing historical templates from the template memory pool to obtain a memory reference template, and generating a new generation template for the next frame tracking.
2. The adaptive template update method for object tracking according to claim 1, wherein, The method of generating a comprehensive confidence of the current frame based on the cross-correlation response map, the target position and historical tracking information, comprises: calculating the ratio of the peak value to the average value of the response map to obtain a response feature indicator; calculating the average intersection over union of the predicted target bounding box of the current frame and the target bounding boxes of the previous three frames to obtain a geometric feature indicator; calculating the semantic feature indicator of the search region feature map, and determining the semantic feature purity based on the semantic feature indicator; and performing weighted fusion on the response feature indicator, the geometric feature indicator and the semantic feature purity to obtain the comprehensive confidence.
3. The method for adaptive template update for object tracking according to claim 2, wherein, In the weighted fusion, the weight of each indicator is obtained by normalizing the learnable parameter through the softmax function.
4. The method for adaptive template update for object tracking of claim 1, wherein, The method of updating the template memory pool based on the comprehensive confidence comprises: if the comprehensive confidence is higher than a preset high confidence threshold, the purified target features of the current frame, the comprehensive confidence and time step information are stored in the template memory pool as a new template; when the capacity of the template memory pool reaches the upper limit, the template with the longest storage time and the lowest confidence is removed.
5. The adaptive template update method for object tracking of claim 1, wherein, The method of fusing historical templates from the template memory pool to obtain a memory reference template comprises: calculating the cosine similarity between the purified target features and the historical templates in the template memory pool; and according to the cosine similarity and a time decay weight, selecting the top K historical templates from the template memory pool for weighted fusion to generate a memory reference template.
6. The method for adaptive template update for object tracking of claim 1, wherein, The method of generating a new generation template for the next frame tracking comprises: calculating the fusion weight of the current purified features through a monotonically increasing function according to the comprehensive confidence, and performing weighted summation on the purified target features and the memory reference template using the weight to generate the new generation template.
7. The adaptive template update method for object tracking of claim 1, wherein: The method of extracting current target features based on the target position, and inputting the current target features into a pre-trained generator G for adversarial purification processing to obtain purified target features comprises: inputting the current target features into a pre-trained generator G, wherein the generator G is a U-Net structure comprising an encoder, a decoder and a skip connection. The encoder is composed of multiple convolution layers, the number of channels increases layer by layer, and each convolution layer is followed by a batch normalization BatchNorm and a LeakyReLU activation function; the decoder is composed of multiple transposed convolution layers, the number of channels decreases layer by layer to be consistent with the input channel number, and each convolution layer is followed by a batch normalization BatchNorm and a ReLU activation function; after the current target feature is processed by the encoder and the decoder, the purified feature with the same dimension as the input feature is generated through the Tanh activation function of the output layer of the generator.
8. The method for adaptive template update for object tracking of claim 1, wherein, The generator G is trained through an adversarial training method including a local discrimination mechanism, including: Step S1, selecting a video frame with a clear target and a simple background from a tracking data set as a clean target subgraph, and extracting a rectangular region containing context information based on a tracking bounding box; Step S2, adding adversarial noise or natural interference noise to the clean target subgraph to generate a noisy target subgraph; Step S3, using a pre-trained Siamese tracker feature extraction network to extract pure features and noisy features from the clean target subgraph and the noisy target subgraph respectively; Step S4, fixing the generator parameters, using the pure features as positive samples and the purified features output by the generator as negative samples to train the discriminator to accurately distinguish between real pure features and generated features; Step S5, fixing the discriminator parameters, training the generator to minimize the discrimination accuracy of the discriminator on the purified features, and simultaneously constraining the consistency between the purified features and the real pure features through L1 reconstruction loss; Repeat steps S4-S5 until the model converges to obtain a generator G that can effectively remove background noise from the features.
9. An adaptive template update system for target tracking for implementing the method of any one of claims 1 to 8, characterized in that, The system comprises: A Siamese tracker for extracting target and search region features, including: A feature extraction module for extracting features in an input image; A cross-correlation calculation module for extracting search region features of a current frame, performing cross-correlation calculation on the search region features and a current template to generate a cross-correlation response map; A tracking head for predicting a target position based on the cross-correlation response map; An adaptive template update network integrated with the basic Siamese tracker for dynamically generating a template and assisting target positioning, including: A multi-dimensional confidence evaluation module for generating a comprehensive confidence of a current frame based on the cross-correlation response map, the target position and historical tracking information through multi-dimensional evaluation; An adversarial template purification module for extracting a current target feature based on the target position and performing adversarial purification processing on the current target feature to obtain a purified target feature; A memory-enhanced dynamic update module for updating a template memory pool based on the comprehensive confidence, and fusing historical templates from the template memory pool based on the purified target feature and the comprehensive confidence to obtain a memory reference template and generate a new generation template for the next frame tracking.