Template-based hierarchical memory and three-stage update based target tracking method and system

Abstract

The present application relates to the technical field of computer vision and image processing, and proposes a target tracking method and system based on template hierarchical memory and three-stage updating, which comprises: using a global template to represent an initialization as a three-level hierarchical memory library; extracting search area features, and sequentially splicing and time-series joint interaction modeling the search area features and the hierarchical memory sequence provided by the three-level hierarchical memory library through a time-series joint interaction modeling module; inputting the enhanced search features into a prediction head for prediction to obtain the current frame target bounding box coordinates and confidence score map, and obtaining the current frame target tracking result based on the current frame target bounding box coordinates and confidence score map. Through the template feature compression and context aggregation mechanism, the first frame target appearance information is converted into a compact and discriminative global representation, which can reduce the template redundancy information while improving the template expression quality, and improve the stability and efficiency of subsequent matching.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a target tracking method and system based on template hierarchical memory and three-stage update. Background Technology

[0002] Visual object tracking is one of the fundamental research tasks in the field of computer vision. Its main goal is to continuously locate and estimate the position and state of the target in subsequent video frames, given the target's position annotation in the first frame of a video sequence. With the continuous development of deep learning technology, object tracking methods based on convolutional neural networks and Transformer architectures have gradually become a research hotspot. Among them, Transformer-based methods utilize self-attention mechanisms to model global contextual relationships, possessing strong feature representation and target association capabilities, and thus have become an important technical approach in the current field of object tracking.

[0003] In existing Transformer-based tracking frameworks, a static template is typically constructed using the target region features in the first frame. In subsequent frames, the template features are matched with the search region features to achieve target localization. However, in actual tracking, the target appearance is constantly changing due to factors such as deformation, occlusion, lighting variations, motion blur, and pose changes. This causes the difference between the initial static template and the target appearance in the current frame to gradually increase, leading to a decrease in feature matching accuracy and potentially causing tracking drift or even tracking failure.

[0004] To address these issues, some existing technologies introduce dynamic template update mechanisms. By incorporating current frame features into the template representation, these mechanisms enhance the template's adaptability to temporal changes in the target. However, existing dynamic update strategies typically lack effective evaluation and screening of template quality. In situations where the target is occluded, background interference is strong, or tracking results are unstable, low-quality features are easily introduced into the template library, leading to noise accumulation and template contamination. This, in turn, reduces the stability and robustness of the tracking system.

[0005] Current technologies still have shortcomings in historical information modeling, lacking a unified representation and hierarchical management mechanism for the changing patterns of target appearance across different time scales, and making it difficult to balance the long-term stability and short-term dynamic adaptability of target appearance. Therefore, how to achieve hierarchical modeling, adaptive updating, and effective management of long-term and short-term changes in target appearance while compressing historical information remains a key technical problem to be solved in the current target tracking field. Summary of the Invention

[0006] In view of the above, the main objective of this invention is to propose a target tracking method and system based on template hierarchical memory and three-stage update to solve the aforementioned technical problems.

[0007] This invention proposes a target tracking method based on template hierarchical memory and three-stage update, the method comprising the following steps: Step 1: The tracking model is constructed based on the image segmentation and embedding module, the feature extraction backbone network, the temporal joint interaction modeling module, the three-level hierarchical memory, the attention aggregation and compression module, and the prediction head; Step 2: Obtain the template image of the initial frame of the video sequence. Using the image segmentation and embedding module and the feature extraction backbone network, perform image segmentation and deep feature extraction on the template image in sequence. Then, aggregate it into a global template representation through the attention aggregation and compression module. Use the global template representation to initialize a three-level hierarchical memory bank. Each level in the three-level hierarchical memory bank stores hierarchical memory sequences of the appearance characteristics of the target at different time scales. Step 3: Obtain the current frame search image, extract search region features using the image segmentation and embedding module and the feature extraction backbone network, and then use the temporal joint interaction modeling module to sequentially splice the search region features with the hierarchical memory sequence provided by the three-level hierarchical memory bank and perform temporal joint interaction modeling to generate enhanced search features. Input the enhanced search features into the prediction head for prediction to obtain the current frame tracking result. Step 4: Based on the enhanced search features and according to the saturation state of the three-level hierarchical memory, a three-stage adaptive update strategy is adopted to adaptively perform memory updates and obtain the updated three-level hierarchical memory. Step 5: Based on the updated three-level hierarchical memory, repeat steps 3 and 4 iteratively for subsequent frames to obtain the target bounding box coordinates and confidence score for each frame. Based on the target bounding box coordinates and confidence score for each frame, obtain the target tracking result. Step 6: Construct a loss function based on the target tracking results to optimize the tracking model and obtain an optimized tracking model; use the optimized tracking model to track the target and obtain the final target tracking result.

[0008] This invention also proposes a target tracking system based on template hierarchical memory and three-stage update, the system comprising: Build modules are used for: The tracking model is composed of an image segmentation and embedding module, a feature extraction backbone network, a temporal joint interaction modeling module, a three-level hierarchical memory, an attention aggregation and compression module, and a prediction head. Hierarchical memory module, used for: A template image of the initial frame of the video sequence is obtained. The template image is then segmented and deep feature extraction is performed sequentially using an image segmentation and embedding module and a feature extraction backbone network. The template image is then aggregated into a global template representation using an attention aggregation and compression module. The global template representation is then initialized into a three-level hierarchical memory. Each level of the three-level hierarchical memory stores hierarchical memory sequences of the appearance characteristics of the target at different time scales. The timing union module is used for: The current frame search image is acquired, and the search region features are extracted using the image segmentation and embedding module and the feature extraction backbone network. The search region features are then sequentially spliced ​​with the hierarchical memory sequence provided by the three-level hierarchical memory bank and subjected to temporal joint interaction modeling through the temporal joint interaction modeling module to generate enhanced search features. The enhanced search features are then input into the prediction head for prediction to obtain the current frame tracking result. The memory update module is used for: Based on enhanced search features and according to the saturation state of the three-level hierarchical memory, a three-stage adaptive update strategy is adopted to adaptively perform memory updates and obtain the updated three-level hierarchical memory. The iterative tracing module is used for: Based on the updated three-level hierarchical memory, the steps of the temporal joint module and the memory update module are repeated iteratively for subsequent frames to obtain the target bounding box coordinates and confidence scores for each frame. Based on the target bounding box coordinates and confidence scores for each frame, the target tracking results are obtained. The joint training module is used for: A loss function is constructed based on the target tracking results to optimize the tracking model, resulting in an optimized tracking model. The optimized tracking model is then used to track the target, yielding the final target tracking result.

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention transforms the target appearance information in the first frame into a compact and discriminative global representation through template feature compression and context aggregation mechanisms. This can improve the template expression quality while reducing template redundancy information, thereby improving the stability and efficiency of subsequent matching.

[0010] 2. This invention introduces a hierarchical memory and dynamic template joint modeling mechanism to achieve continuous accumulation and effective utilization of target historical appearance information. It can simultaneously take into account the long-term consistency of target appearance and short-term adaptability to changes, and alleviate the mismatch problem of traditional static templates in long-term tracking.

[0011] 3. This invention enhances the model's ability to distinguish between targets and interference objects in complex backgrounds by using temporal joint interactive modeling of template information, historical context information and current search features, thereby improving target positioning accuracy and reducing tracking drift risk.

[0012] 4. This invention achieves dynamic maintenance of templates and historical memory through an adaptive update strategy based on confidence and positioning quality. It can suppress the introduction of low-quality features and reduce performance degradation caused by memory pollution in scenarios such as occlusion, deformation, lighting changes and background interference.

[0013] 5. This invention enables the effective organization and iterative updating of historical information under the constraint of limited storage length, and improves the robustness of long-term time-series tracking while ensuring computational controllability. It is suitable for engineering application scenarios with high requirements for real-time performance and stability.

[0014] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0015] Figure 1 The flowchart shows the target tracking method based on template hierarchical memory and three-stage update proposed in this invention. Figure 2 This is a diagram illustrating the overall framework of the target tracking method based on template hierarchical memory and three-stage update proposed in this invention.

[0016] Figure 3 This is a structural diagram of the temporal joint modeling module of the target tracking method based on template hierarchical memory and three-stage update proposed in this invention.

[0017] Figure 4 This is a state diagram of the three-stage update of the three-level hierarchical memory bank for the target tracking method based on template hierarchical memory and three-stage update proposed in this invention.

[0018] Figure 5 This is the overall architecture diagram of the target tracking system based on template hierarchical memory and three-stage update proposed in this invention. Detailed Implementation

[0019] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0020] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0021] Please see Figure 1 This embodiment provides a target tracking method based on template hierarchical memory and three-stage update, the method including the following steps: Step 1: The tracking model is constructed based on the image segmentation and embedding module, the feature extraction backbone network, the temporal joint interaction modeling module, the three-level hierarchical memory, the attention aggregation and compression module, and the prediction head.

[0022] Furthermore, the image segmentation and embedding module is used to segment and linearly embed the target region image of the first frame and the candidate region image of the current frame to be tracked; the feature extraction backbone network module is used to extract deep features from the embedded image patch sequence, generating template patch feature sequences and search region patch feature sequences respectively; at the same time, an attention aggregation compression module is integrated into the network path for template feature processing, and a three-level hierarchical memory is constructed to store and manage the target's historical appearance information at multiple time scales; the temporal joint interaction modeling module is used to establish a global contextual association between the template hierarchical memory features and the current search features through a self-attention mechanism; the prediction head is used to output the classification confidence score map and bounding box regression results of the target in the current frame.

[0023] Step 2: Obtain the template image of the initial frame of the video sequence. Using the image segmentation and embedding module and the feature extraction backbone network, perform image segmentation and deep feature extraction on the template image in sequence. Then, aggregate it into a global template representation through the attention aggregation and compression module. Use the global template representation to initialize a three-level hierarchical memory bank. Each level in the three-level hierarchical memory bank has a hierarchical memory sequence that stores the appearance characteristics of the target at different time scales.

[0024] Please see Figure 2 In step 2, a template image of the initial frame of the video sequence is obtained. Using an image segmentation and embedding module and a feature extraction backbone network, the template image is sequentially segmented and deep feature extracted. Then, it is aggregated into a global template representation using an attention aggregation and compression module. This global template representation is then used to initialize a three-level hierarchical memory. Each level of the three-level hierarchical memory stores hierarchical memory sequences of the target's appearance characteristics at different time scales, specifically including the following steps: The template image in the first frame of the video sequence is obtained, and the template image is sequentially divided into image blocks and linearly embedded using the image segmentation and embedding module. Then, deep features are extracted through the feature extraction backbone network module to obtain the template patch feature sequence. The template patch feature sequence is input into the attention aggregation and compression module. Spatial location prior information is introduced by superimposing two-dimensional sine and cosine position codes. Then, the normalized attention weight of each patch is calculated by the attention prediction network composed of alternating stacks of linear transformation and activation functions. Based on the normalized attention weight, the template patch features in the template patch feature sequence are weighted and summed to aggregate the template patch feature sequence into a global template representation. The three-level hierarchical memory is initialized based on the global template representation to obtain the initialized three-level hierarchical memory; wherein, the first level of the initialized three-level hierarchical memory is composed of the global template representation repeated a preset number of times; the second level long-term aggregate memory and the third level short-term dynamic features of the initialized three-level hierarchical memory are initialized to empty.

[0025] Step 3: Obtain the current frame search image, extract search region features using the image segmentation and embedding module and the feature extraction backbone network, and then use the temporal joint interaction modeling module to sequentially concatenate the search region features with the hierarchical memory sequence provided by the three-level hierarchical memory bank and perform temporal joint interaction modeling to generate enhanced search features. Input the enhanced search features into the prediction head for prediction to obtain the current frame tracking result.

[0026] Please see Figure 3 In step 3, the current frame search image is acquired, and the search region features are extracted using the image segmentation and embedding module and the feature extraction backbone network. The search region features are then sequentially concatenated with the hierarchical memory sequence provided by the three-level hierarchical memory bank through the temporal joint interaction modeling module, generating enhanced search features. These enhanced search features are then input into the prediction head for prediction to obtain the current frame tracking result. Specifically, the steps include the following: The current frame search image is obtained, and the search image is sequentially segmented and linearly embedded using the image segmentation and embedding module. Then, deep features are extracted through the feature extraction backbone network module to obtain the search patch feature sequence. The search patch feature sequence is concatenated with the features of each level in the initialized three-level hierarchical memory bank to obtain a three-level hierarchical memory sequence. This three-level hierarchical memory sequence is then input into the temporal joint interaction modeling module. A self-attention mechanism is used to jointly model the three-level hierarchical memory sequence and the search patch feature sequence to obtain enhanced search features. The three-level hierarchical memory bank consists of three layers: the first layer is the initial template layer; the second layer is the long-term memory layer; and the third layer is the short-term memory layer. The first-level initial template representation in the three-level hierarchical memory sequence provides stable appearance anchors; the second-level long-term aggregate memory provides historical trends in the target's appearance; and the third-level short-term dynamic features provide recent fine-grained appearance states of the target. These three layers work together to guide the search feature expression in the current frame. The search features are input into the prediction head for prediction, and the target bounding box coordinates and the confidence score map of the current frame are obtained. Based on the target bounding box coordinates and the confidence score map of the current frame, the target tracking result of the current frame is obtained.

[0027] Specifically, the three-level hierarchical memory sequence and the search patch feature sequence are jointly modeled using a self-attention mechanism, and the corresponding relationship in the process is as follows: ; in, This represents the result of self-attention calculation. , , These represent the query matrix, key matrix, and value matrix, respectively. This represents the similarity calculation result between the query matrix and the key matrix. Represents the transpose of the key matrix , This represents the dimension of the key vector. This indicates scaling the similarity score. This represents the normalization function.

[0028] It should be noted that, Used to prevent values ​​from being too large; Used to convert similarity into a weighted distribution.

[0029] Step 4: Based on the enhanced search features and according to the saturation state of the three-level hierarchical memory, a three-stage adaptive update strategy is adopted to adaptively perform memory updates and obtain the updated three-level hierarchical memory.

[0030] Please see Figure 4 In step 4, based on the enhanced search features and according to the saturation state of the three-level hierarchical memory, a three-stage adaptive update strategy is adopted to adaptively perform memory updates, resulting in the updated three-level hierarchical memory. Specifically, this includes the following steps: Based on enhanced search features, the upper limit of the short-term memory layer capacity is set to... The maximum capacity of the long-term memory layer is The initial template layer retains a lower limit of 1. In the At each frame time, the current dynamic feature block is generated. And based on the current saturation state of the three-level hierarchical memory, it adaptively selects to enter the append stage, the decay and compression stage, or the sliding aggregation and update stage; If the short-term memory layer has not yet reached its capacity limit. When selecting to enter the append phase, the current dynamic feature block is used. Write directly to the short-term memory layer to obtain the updated three-level hierarchical memory bank; If the short-term memory layer reaches its capacity limit Furthermore, the initial template layer still has compressible space, so the decay compression stage is selected. The feature block written first in the short-term memory layer is chosen, and the attention aggregation compression module compresses the feature block written first in the short-term memory layer to generate a long-term memory representation. The long-term memory representation is written to the long-term memory layer, while the oldest template token in the initial template layer is removed, and the current dynamic feature block is... Write to the short-term memory layer to obtain the updated three-level hierarchical memory bank; If both the long-term memory layer and the initial template layer reach their respective capacity constraints, the sliding aggregation update phase is initiated. A sliding update strategy is then executed simultaneously on both the short-term memory layer and the long-term memory layer. The newly generated long-term memory token replaces the oldest element in the long-term memory layer, and the current dynamic feature block is used. Replace the oldest element in the shortest layer to obtain the updated three-level hierarchical memory.

[0031] If the short-term memory layer has not yet reached its capacity limit. When selecting to enter the append phase, the current dynamic feature block is used. The relationship between directly writing to the short-term memory layer and the corresponding process is as follows: ; in, This indicates the number of feature tokens contained in the corresponding memory layer. Indicates the first Short-term memory layer at frame time, Indicates the first Short-term memory layer at frame time, This represents the set expansion operation. Represents a set.

[0032] If the short-term memory layer reaches its capacity limit Furthermore, the initial template layer still has compressible space. Therefore, the decay compression stage is selected, and the feature block written first in the short-term memory layer is chosen. This feature block is then compressed using the attention aggregation compression module to generate a long-term memory representation. The corresponding relationship in this process is as follows: ; in, Indicates the first The initial template layer at frame time, Representing long-term memory, Represents the feature aggregation function, This represents the earliest written feature block in the short-term memory layer. express A dimensional real vector space.

[0033] If both the long-term memory layer and the initial template layer reach their respective capacity constraints, the sliding aggregation update phase is initiated. A sliding update strategy is then executed simultaneously on both the short-term memory layer and the long-term memory layer. The newly generated long-term memory token replaces the oldest element in the long-term memory layer, and the current dynamic feature block is used. The process of replacing the oldest element in the shortest layer corresponds to the following relationship: ; in, Indicates the first The long-term memory layer at each frame time, Indicates the first The long-term memory layer at each frame time, Indicates the first The initial template layer at frame time, This indicates the preset total memory capacity.

[0034] It should be noted that, through the above mechanism, the present invention can achieve effective modeling and continuous updating of long-term target appearance information under fixed resource constraints.

[0035] It should be noted that, Figure 4 In this context, C represents the splicing operation.

[0036] Step 5: Based on the updated three-level hierarchical memory, repeat steps 3 and 4 iteratively for subsequent frames to obtain the target bounding box coordinates and confidence score for each frame. Based on the target bounding box coordinates and confidence score for each frame, obtain the target tracking result.

[0037] Furthermore, based on the updated three-level hierarchical memory in step 4, steps 3 to 4 are repeated to perform frame-by-frame iterative processing on the subsequent search region images, continuously outputting the target bounding box coordinates and confidence scores, thereby achieving long-term stable tracking of the target in the video sequence.

[0038] Step 6: Construct a loss function based on the target tracking results to optimize the tracking model and obtain an optimized tracking model; use the optimized tracking model to track the target and obtain the final target tracking result.

[0039] In step 6, a loss function is constructed based on the target bounding box coordinates of each frame and the confidence score map of each frame in the target tracking results to optimize the tracking model.

[0040] It should be noted that the classification loss is constructed based on the confidence score map of each frame, and the bounding box regression loss, IoU loss and coordinate regression loss are constructed based on the target bounding box coordinates of each frame.

[0041] Furthermore, the target tracking model is trained end-to-end using a joint loss function. The joint loss function includes at least classification loss and bounding box regression loss. The classification loss is used to constrain the ability to distinguish between the target foreground and background, and the bounding box regression loss is used to improve the target localization accuracy. In some embodiments, the joint loss function also includes IoU loss and coordinate regression loss to further improve the accuracy and robustness of bounding box prediction.

[0042] A preferred embodiment of the joint loss function can be expressed as: ; in, Denotes the joint loss function. , , , These represent the weight coefficients corresponding to classification loss, bounding box regression loss, IoU loss, and coordinate regression loss, respectively. Represents classification loss, This represents the bounding box regression loss. Indicates IoU loss. This represents the coordinate regression loss.

[0043] In summary, this invention constructs a target tracking method based on template hierarchical memory and three-stage update. Through the synergistic effect of various components, it achieves effective preservation and continuous updating of target appearance information with a limited sequence length, taking into account real-time performance, stability and long-term tracking performance.

[0044] Please see Figure 5 The present invention also proposes a target tracking system based on template hierarchical memory and three-stage update, the system comprising: Build modules are used for: The tracking model is composed of an image segmentation and embedding module, a feature extraction backbone network, a temporal joint interaction modeling module, a three-level hierarchical memory, an attention aggregation and compression module, and a prediction head. Hierarchical memory module, used for: A template image of the initial frame of the video sequence is obtained. The template image is then segmented and deep feature extraction is performed sequentially using an image segmentation and embedding module and a feature extraction backbone network. The template image is then aggregated into a global template representation using an attention aggregation and compression module. The global template representation is then initialized into a three-level hierarchical memory. Each level of the three-level hierarchical memory stores hierarchical memory sequences of the appearance characteristics of the target at different time scales. The timing union module is used for: The current frame search image is acquired, and the search region features are extracted using the image segmentation and embedding module and the feature extraction backbone network. The search region features are then sequentially spliced ​​with the hierarchical memory sequence provided by the three-level hierarchical memory bank and subjected to temporal joint interaction modeling through the temporal joint interaction modeling module to generate enhanced search features. The enhanced search features are then input into the prediction head for prediction to obtain the current frame tracking result. The memory update module is used for: Based on enhanced search features and according to the saturation state of the three-level hierarchical memory, a three-stage adaptive update strategy is adopted to adaptively perform memory updates and obtain the updated three-level hierarchical memory. The iterative tracing module is used for: Based on the updated three-level hierarchical memory, the steps of the temporal joint module and the memory update module are repeated iteratively for subsequent frames to obtain the target bounding box coordinates and confidence scores for each frame. Based on the target bounding box coordinates and confidence scores for each frame, the target tracking results are obtained. The joint training module is used for: A loss function is constructed based on the target tracking results to optimize the tracking model, resulting in an optimized tracking model. The optimized tracking model is then used to track the target, yielding the final target tracking result.

[0045] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0046] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0047] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A target tracking method based on template hierarchical memory and three-stage update, characterized in that, The method includes the following steps: Step 1: The tracking model is constructed based on the image segmentation and embedding module, the feature extraction backbone network, the temporal joint interaction modeling module, the three-level hierarchical memory, the attention aggregation and compression module, and the prediction head; Step 2: Obtain the template image of the initial frame of the video sequence. Using the image segmentation and embedding module and the feature extraction backbone network, perform image segmentation and deep feature extraction on the template image in sequence. Then, aggregate it into a global template representation through the attention aggregation and compression module. A three-level hierarchical memory is initialized using a global template representation; wherein each level of the three-level hierarchical memory has a hierarchical memory sequence storing the appearance characteristics of the target at different time scales; Step 3: Obtain the current frame search image, extract search region features using the image segmentation and embedding module and the feature extraction backbone network, and then use the temporal joint interaction modeling module to sequentially splice the search region features with the hierarchical memory sequence provided by the three-level hierarchical memory bank and perform temporal joint interaction modeling to generate enhanced search features. Input the enhanced search features into the prediction head for prediction to obtain the current frame tracking result. Step 4: Based on the enhanced search features and according to the saturation state of the three-level hierarchical memory, a three-stage adaptive update strategy is adopted to adaptively perform memory updates, resulting in the updated three-level hierarchical memory. This includes the following steps: Based on enhanced search features, the upper limit of the short-term memory layer capacity is set to... The maximum capacity of the long-term memory layer is The initial template layer retains a lower limit of 1. In the At each frame time, the current dynamic feature block is generated. And based on the current saturation state of the three-level hierarchical memory, it adaptively selects to enter the append stage, the decay and compression stage, or the sliding aggregation and update stage; If the short-term memory layer has not yet reached its capacity limit. When selecting to enter the append phase, the current dynamic feature block is used. Write directly to the short-term memory layer to obtain the updated three-level hierarchical memory bank; If the short-term memory layer reaches its capacity limit Furthermore, the initial template layer still has compressible space, so the decay compression stage is selected. The feature block written first in the short-term memory layer is chosen, and the attention aggregation compression module compresses the feature block written first in the short-term memory layer to generate a long-term memory representation. The long-term memory representation is written to the long-term memory layer, while the oldest template token in the initial template layer is removed, and the current dynamic feature block is... Write to the short-term memory layer to obtain the updated three-level hierarchical memory bank; If both the long-term memory layer and the initial template layer reach their respective capacity constraints, the sliding aggregation update phase is initiated. A sliding update strategy is then executed simultaneously on both the short-term memory layer and the long-term memory layer. The newly generated long-term memory token replaces the oldest element in the long-term memory layer, and the current dynamic feature block is used. Replace the oldest element in the shortest layer to obtain the updated three-level hierarchical memory; Step 5: Based on the updated three-level hierarchical memory, repeat steps 3 and 4 iteratively for subsequent frames to obtain the target bounding box coordinates and confidence score for each frame. Based on the target bounding box coordinates and confidence score for each frame, obtain the target tracking result. Step 6: Construct a loss function based on the target tracking results to optimize the tracking model and obtain an optimized tracking model; use the optimized tracking model to track the target and obtain the final target tracking result.

2. The target tracking method based on template hierarchical memory and three-stage update according to claim 1, characterized in that, In step 2, a template image of the initial frame of the video sequence is obtained. Using an image segmentation and embedding module and a feature extraction backbone network, the template image is sequentially segmented and deep feature extracted. Then, it is aggregated into a global template representation using an attention aggregation and compression module. This global template representation is then used to initialize a three-level hierarchical memory, specifically including the following steps: The template image in the first frame of the video sequence is obtained, and the template image is sequentially divided into image blocks and linearly embedded using the image segmentation and embedding module. Then, deep features are extracted through the feature extraction backbone network module to obtain the template patch feature sequence. The template patch feature sequence is input into the attention aggregation and compression module. Spatial location prior information is introduced by superimposing two-dimensional sine and cosine position codes. Then, the normalized attention weight of each patch is calculated by the attention prediction network composed of alternating stacks of linear transformation and activation functions. Based on the normalized attention weight, the template patch features in the template patch feature sequence are weighted and summed to aggregate the template patch feature sequence into a global template representation. The three-level hierarchical memory is initialized based on the global template representation to obtain the initialized three-level hierarchical memory; wherein, the first level of the initialized three-level hierarchical memory is composed of the global template representation repeated a preset number of times; the second level long-term aggregate memory and the third level short-term dynamic features of the initialized three-level hierarchical memory are initialized to empty.

3. The target tracking method based on template hierarchical memory and three-stage update according to claim 2, characterized in that, In step 3, the current frame search image is acquired, and the search region features are extracted using the image segmentation and embedding module and the feature extraction backbone network. The search region features are then sequentially concatenated with the hierarchical memory sequence provided by the three-level hierarchical memory bank and subjected to temporal joint interactive modeling through the temporal joint interactive modeling module to generate enhanced search features. These enhanced search features are then input into the prediction head for prediction to obtain the current frame tracking result. Specifically, the steps include the following: The current frame search image is obtained, and the search image is sequentially segmented and linearly embedded using the image segmentation and embedding module. Then, deep features are extracted through the feature extraction backbone network module to obtain the search patch feature sequence. The search patch feature sequence is concatenated with the features of each level in the initialized three-level hierarchical memory bank to obtain a three-level hierarchical memory sequence. This three-level hierarchical memory sequence is then input into the temporal joint interaction modeling module. A self-attention mechanism is used to jointly model the three-level hierarchical memory sequence and the search patch feature sequence to obtain enhanced search features. The three-level hierarchical memory bank consists of three layers: the first layer is the initial template layer; the second layer is the long-term memory layer; and the third layer is the short-term memory layer. The first-level initial template representation in the three-level hierarchical memory sequence provides stable appearance anchors; the second-level long-term aggregate memory provides historical trends in the target's appearance; and the third-level short-term dynamic features provide recent fine-grained appearance states of the target. These three layers work together to guide the search feature expression in the current frame. The enhanced search features are input into the prediction head for prediction, and the target bounding box coordinates and the confidence score map of the current frame are obtained. Based on the target bounding box coordinates and the confidence score map of the current frame, the target tracking result of the current frame is obtained.

4. The target tracking method based on template hierarchical memory and three-stage update according to claim 3, characterized in that, The three-level hierarchical memory sequence and the search patch feature sequence are jointly modeled using a self-attention mechanism. The corresponding relationship in the process is as follows: ； in, This represents the result of self-attention calculation. , , These represent the query matrix, key matrix, and value matrix, respectively. This represents the similarity calculation result between the query matrix and the key matrix. Represents the transpose of the key matrix , This represents the dimension of the key vector. This indicates scaling the similarity score. This represents the normalization function.

5. The target tracking method based on template hierarchical memory and three-stage update according to claim 1, characterized in that, If the short-term memory layer has not yet reached its capacity limit. When selecting to enter the append phase, the current dynamic feature block is used. The relationship between directly writing to the short-term memory layer and the corresponding process is as follows: ； in, This indicates the number of feature tokens contained in the corresponding memory layer. Indicates the first Short-term memory layer at frame time, Indicates the first Short-term memory layer at frame time, This represents the set expansion operation. Represents a set.

6. The target tracking method based on template hierarchical memory and three-stage update according to claim 5, characterized in that, If the short-term memory layer reaches its capacity limit Furthermore, the initial template layer still has compressible space. Therefore, the decay compression stage is selected, and the feature block written first in the short-term memory layer is chosen. This feature block is then compressed using the attention aggregation compression module to generate a long-term memory representation. The corresponding relationship in this process is as follows: ； in, Indicates the first The initial template layer at frame time, Representing long-term memory, Represents the feature aggregation function, This represents the earliest written feature block in the short-term memory layer. express A dimensional real vector space.

7. The target tracking method based on template hierarchical memory and three-stage update according to claim 6, characterized in that, If both the long-term memory layer and the initial template layer reach their respective capacity constraints, the sliding aggregation update phase is initiated. A sliding update strategy is then executed simultaneously on both the short-term memory layer and the long-term memory layer. The newly generated long-term memory token replaces the oldest element in the long-term memory layer, and the current dynamic feature block is used. The process of replacing the oldest element in the shortest layer corresponds to the following relationship: ； in, Indicates the first The long-term memory layer at each frame time, Indicates the first The long-term memory layer at each frame time, Indicates the first The initial template layer at frame time, This indicates the preset total memory capacity.

8. A target tracking system based on template hierarchical memory and three-stage update, characterized in that, The system applies the target tracking method based on template hierarchical memory and three-stage update as described in any one of claims 1-7, and the system includes: Build modules are used for: The tracking model is composed of an image segmentation and embedding module, a feature extraction backbone network, a temporal joint interaction modeling module, a three-level hierarchical memory, an attention aggregation and compression module, and a prediction head. Hierarchical memory module, used for: A template image of the initial frame of the video sequence is obtained. The template image is then segmented and deep feature extraction is performed sequentially using an image segmentation and embedding module and a feature extraction backbone network. The template image is then aggregated into a global template representation using an attention aggregation and compression module. The global template representation is then initialized into a three-level hierarchical memory. Each level of the three-level hierarchical memory stores hierarchical memory sequences of the appearance characteristics of the target at different time scales. The timing union module is used for: The current frame search image is acquired, and the search region features are extracted using the image segmentation and embedding module and the feature extraction backbone network. The search region features are then sequentially spliced ​​with the hierarchical memory sequence provided by the three-level hierarchical memory bank and subjected to temporal joint interaction modeling through the temporal joint interaction modeling module to generate enhanced search features. The enhanced search features are then input into the prediction head for prediction to obtain the current frame tracking result. The memory update module is used for: Based on enhanced search features and according to the saturation state of the three-level hierarchical memory, a three-stage adaptive update strategy is adopted to adaptively perform memory updates and obtain the updated three-level hierarchical memory. The iterative tracing module is used for: Based on the updated three-level hierarchical memory, the steps of the temporal joint module and the memory update module are repeated iteratively for subsequent frames to obtain the target bounding box coordinates and confidence scores for each frame. Based on the target bounding box coordinates and confidence scores for each frame, the target tracking results are obtained. The joint training module is used for: A loss function is constructed based on the target tracking results to optimize the tracking model, resulting in an optimized tracking model. The optimized tracking model is then used to track the target, yielding the final target tracking result.

Images