Target tracking method and system based on multi-stage feature fusion and residual enhancement

Abstract

The application provides a target tracking method and system based on multi-stage feature fusion and residual enhancement, which comprises the following steps: extracting the features of a template and a search image and adding position coding to obtain template image features and search image features; enhancing the search features through a two-stage enhancement module and generating a confidence score; then, using a learnable sparse modulation and a dynamic gating bottleneck module to further filter and differentially enhance the features; then, training the model using a large-scale dataset, optimizing the parameters through joint classification and IoU loss; finally, splicing the enhanced search features and the template features, interacting through an encoder, and outputting the tracking results by a prediction head. Through multi-stage enhancement, residual fusion and dynamic adjustment mechanism, the application can effectively deal with difficult scenes such as target occlusion, complex background, scale transformation and rapid motion, and improve the robustness and accuracy of target tracking.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a target tracking method and system based on multi-stage feature fusion and residual enhancement. Background Technology

[0002] In recent years, visual tracking has been a crucial task in the field of computer vision, typically referring to the continuous localization of target objects within video sequences. It is currently widely applied in areas such as security and intelligent surveillance, autonomous driving and intelligent transportation, drone and robot vision, medical diagnosis and assisted treatment, and intelligent manufacturing. Despite continuous improvements in related algorithms, real-world scenarios still present numerous challenges due to various uncertainties and interferences, such as target occlusion, lighting interference, shape changes, and interference from similar objects. These factors mean that there is still significant room for improvement in accurate tracking and robustness. With the continuous development of deep learning, visual tracking has seen substantial advancements.

[0003] The introduction of the Transformer architecture further propelled the development of tracking technology. Its core self-attention mechanism effectively captures dependencies between image patches, overcoming the limitations of the local receptive field in traditional convolutional neural networks. Furthermore, through end-to-end training, the model autonomously focuses on key regions, thereby improving tracking performance. Many advanced object tracking models are now based on the Transformer and have achieved excellent results. However, the Transformer's self-attention mechanism tends to focus on global contextual information while neglecting much local information. This loss of local information can potentially have irreversible effects on tracking performance.

[0004] On the other hand, traditional trackers based on Siamese networks typically use a correlation module to match templates with search regions to obtain response maps. Although the similarity performance of small-scale local feature information is efficiently calculated, large-scale global feature information is ignored and the lack of utilization of global semantic information leads to information loss. When facing long-term tracking and occlusion problems, it is very likely to cause tracking errors.

[0005] Therefore, one of the key breakthroughs in current tracking algorithms lies in how to effectively fuse global contextual information and local detail information and enhance them, thereby improving the model's adaptability and robustness in some complex environments, and applying more efficient feature interaction and enhancement mechanisms to improve tracking performance. Summary of the Invention

[0006] In view of the above situation, the main objective of this invention is to propose a target tracking method and system based on multi-stage feature fusion and residual enhancement to solve the above-mentioned technical problems.

[0007] This invention proposes a target tracking method based on multi-stage feature fusion and residual enhancement, the method comprising the following steps:

[0008] Step 1: Construct a two-stage enhancement module based on the Transformer architecture; construct a confidence scoring module and a learnable sparse modulation module based on the multilayer perceptron and the Sigmoid function, respectively; construct a dynamic gating bottleneck enhancement module based on the gating mechanism; and use the two-stage enhancement module, the confidence scoring module, the learnable sparse modulation module, and the dynamic gating bottleneck enhancement module to form a target tracking model.

[0009] Step 2: Input the template image and the search region image into the convolutional network for feature extraction and flattening, and add positional encoding to obtain the features of the template image and the search image respectively.

[0010] Step 3: Use the two-stage enhancement module to enhance the features of the search image, and use the confidence scoring module to generate confidence, so as to obtain the enhanced features and gated confidence after weighting.

[0011] Step 4: Use the learnable sparse modulation module to perform sparse modulation processing on the weighted enhanced features and gated confidence, and use the dynamic gated bottleneck enhancement module to perform enhancement processing to obtain the final enhanced search image features.

[0012] Step 5: Train the target tracking model using a large-scale tracking training dataset, and introduce a classification loss function and an IoU loss function based on template image features and the final enhanced search image features to obtain the trained target tracking model.

[0013] Step 6: Use the trained target tracking model to determine the final template features and the final search features; concatenate the final template features and the final search features and input them into the encoder to obtain the encoder features; input the encoder features into the classification and regression prediction head for prediction to obtain the target tracking results.

[0014] This invention also proposes a target tracking system based on multi-stage feature fusion and residual enhancement, wherein the system employs the target tracking method based on multi-stage feature fusion and residual enhancement as described above, and the system includes:

[0015] Build modules are used for:

[0016] A two-stage enhancement module is constructed based on the Transformer architecture; a confidence scoring module and a learnable sparse modulation module are constructed based on the multilayer perceptron and the Sigmoid function, respectively; a dynamic gating bottleneck enhancement module is constructed based on the gating mechanism; and a target tracking model is constructed using the two-stage enhancement module, the confidence scoring module, the learnable sparse modulation module, and the dynamic gating bottleneck enhancement module.

[0017] The feature extraction module is used for:

[0018] The template image and the search region image are input into a convolutional network for feature extraction and flattening, and positional encoding is added to obtain the features of the template image and the search image, respectively.

[0019] Enhancement modules are used for:

[0020] A two-stage enhancement module is used to enhance the features of the search image, and a confidence scoring module is used to generate confidence, so as to obtain the enhanced features and gated confidence after weighting.

[0021] Sparse modulation module, used for:

[0022] The learnable sparse modulation module is used to perform sparse modulation processing on the weighted enhanced features and gated confidence, and the dynamic gated bottleneck enhancement module is used for enhancement processing to obtain the final enhanced search image features.

[0023] The training module is used for:

[0024] The target tracking model is trained using a large-scale tracking training dataset, and a classification loss function and an IoU loss function are introduced based on template image features and the final enhanced search image features to obtain the trained target tracking model.

[0025] The tracking module is used for:

[0026] The final template features and final search features are determined using the trained target tracking model. The final template features and final search features are concatenated and input into the encoder to obtain the encoder features. The encoder features are then input into the classification and regression prediction head for prediction to obtain the target tracking results.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] 1. This invention designs a two-stage enhancement module that refines features without changing the number and arrangement of tokens. It employs a lightweight, adaptive global gating, local context enhancement, token-level adaptive fusion, and self-attention token confidence estimation, all linked together to highlight key region features. Compared to existing enhancement methods that rely solely on single attention or local convolutions, this invention's two-stage enhancement module simultaneously affects global semantics, local enhancement, and token-level global dependencies. This improves feature discriminability while avoiding redundant computation, resulting in stronger expressive power and stability.

[0029] 2. This invention designs a learnable sparse modulation module, mainly used for adaptive sparse modulation of enhanced features to reduce the interference of background noise on the target. The learnable sparse modulation module jointly utilizes the importance weights of tokens and CE masks to adaptively predict the retention strength of each token, thereby achieving sparse modulation. Compared with traditional hard Top-K screening or hard pruning methods, the learnable sparse modulation module proposed in this invention avoids the problem of training instability, and effectively suppresses redundant background information while maintaining the integrity of foreground information, thus improving the robustness of the overall features.

[0030] 3. This invention designs a dynamic gating bottleneck enhancement module, mainly used for differentiated processing of token features. Based on the importance weights of the tokens, this module divides features into tokens with high confidence and tokens with low confidence, and processes them separately through enhancement and compression branches. Tokens with high confidence are further enhanced through the enhancement branch, while tokens with low confidence have redundant information reduced through the compression branch. The outputs of the two branches are dynamically fused according to the token weights and restored to the original feature space through linear projection and residual connections. Compared with existing Transformer or attention modules that uniformly process all tokens, the dynamic gating bottleneck enhancement module of this invention more explicitly distinguishes between foreground and background, effectively enhancing key features without changing the number of tokens, while suppressing background redundancy.

[0031] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the steps of a target tracking method based on multi-stage feature fusion and residual enhancement proposed in this invention.

[0033] Figure 2This is a diagram illustrating the overall framework of a target tracking method based on multi-stage feature fusion and residual enhancement proposed in this invention.

[0034] Figure 3 This is a diagram showing the overall structure of the target tracking model proposed in this invention;

[0035] Figure 4 for Figure 3 The structure diagram of channel-level adaptive enhancement (AG) in the middle;

[0036] Figure 5 This is a system architecture diagram of a target tracking system based on multi-stage feature fusion and residual enhancement proposed in this invention. Detailed Implementation

[0037] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0038] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0039] Please see Figure 1 This embodiment provides a target tracking method based on multi-stage feature fusion and residual enhancement, the method including the following steps:

[0040] Step 1: Construct a two-stage enhancement module based on the Transformer architecture; construct a confidence scoring module and a learnable sparse modulation module based on the multilayer perceptron and the Sigmoid function, respectively; construct a dynamic gating bottleneck enhancement module based on the gating mechanism; and use the two-stage enhancement module, the confidence scoring module, the learnable sparse modulation module, and the dynamic gating bottleneck enhancement module to form a target tracking model.

[0041] Step 2: Obtain the template image and the search region image. Input the template image and the search region image into the convolutional network for feature extraction and flattening in sequence, and add position encoding to obtain the template image features and the search image features respectively.

[0042] Please see Figure 2In step 2, the template image and the search region image are input into the convolutional network for feature extraction and flattening, and positional encoding is added to obtain the template image features and the search image features, specifically including the following sub-steps:

[0043] The template image and the search region image are input into a convolutional network for feature extraction in sequence to obtain the feature map of the template image and the feature map of the search image.

[0044] The feature maps of the template image and the search image are flattened in the spatial dimension to obtain the template feature sequence and the search feature sequence.

[0045] A linear mapping layer is used to project the template feature sequence and the search feature sequence to the same feature dimension, and learnable positional encodings are added to each to obtain template image features and search image features.

[0046] Step 3: Use the two-stage enhancement module to enhance the features of the search image, and use the confidence scoring module to generate confidence, so as to obtain the enhanced features and gated confidence after weighting.

[0047] Please see Figure 3 and Figure 4 In step 3, the search image features are enhanced using a two-stage enhancement module, and confidence is generated using a confidence scoring module to obtain the enhanced features and gated confidence after weighting. This includes the following sub-steps:

[0048] The search image features are processed by global average pooling to obtain global average pooling features. The following relationship exists in the corresponding process:

[0049] ;

[0050] in, This represents the global average pooling feature. This indicates that global average pooling has been applied. Indicates the search image features;

[0051] The global average pooling features are processed sequentially through a multilayer perceptron and a sigmoid function to obtain channel-level importance weights. The following relationship exists in the corresponding process:

[0052] ;

[0053] in, Indicates channel-level importance weights. This represents the Sigmoid function. This indicates that the data has been processed by a multilayer perceptron.

[0054] The channel-level importance weights are multiplied element-wise with the search image features, and a residual connection is performed to obtain the enhanced search image features. The following relationship exists in the corresponding process:

[0055] ;

[0056] in, This represents the enhanced search image features. This indicates element-wise multiplication;

[0057] The enhanced search image features are subjected to layer normalization to obtain normalized search image features. The following relationship exists in the corresponding process:

[0058] ;

[0059] in, This represents the normalized search image features. This indicates that the process has undergone layer normalization.

[0060] The normalized search image features are sequentially processed through one-dimensional depthwise convolution, GELU activation function, and one-dimensional pointwise convolution to obtain local context-enhanced features. The following relationship exists in the corresponding process:

[0061] ;

[0062] in, This represents local context-enhanced features. This indicates that a one-dimensional point convolution operation has been performed. This represents the GELU activation function. This indicates that a one-dimensional depthwise convolution operation has been performed.

[0063] The normalized search image features are processed using a multilayer perceptron to obtain a high-frequency compensation term. The following relationship exists in the corresponding process:

[0064] ;

[0065] in, Indicates high-frequency compensation terms;

[0066] The enhanced search image features, local context enhancement features, and high-frequency compensation term are connected in a triple residual connection to obtain triple residual connection features. The following relationship exists in the corresponding process:

[0067] ;

[0068] in, This indicates a triple residual connectivity feature;

[0069] The triple residual connection features are sequentially processed through a linear layer and a sigmoid function to obtain fusion coefficients that operate independently on each token. Based on these fusion coefficients, the enhanced search image features are adaptively weighted and fused with the triple residual connection features to obtain the fused feature representation. The following relationship exists in this process:

[0070] ;

[0071] in, This represents the fusion coefficient for independent operation on a token-by-token basis, used for channel-level adaptive fusion of each token; This represents the feature representation after fusion;

[0072] The fused feature representation is processed based on a multi-head self-attention mechanism to obtain the features after multi-head self-attention interaction. The following relationship exists in the corresponding process:

[0073] ;

[0074] in, This represents the features after multi-head self-attention interaction. This indicates that the process has undergone multi-head self-attention mechanism processing;

[0075] The features after multi-head self-attention interaction are input into a multilayer perceptron for processing to obtain confidence scores. The confidence scores are then processed by scaling and a sigmoid function to obtain gated confidence scores. The following relationship exists in the corresponding process:

[0076] ;

[0077] in, This represents the confidence score. Indicates the gate confidence level. This represents the scaling factor, used to adjust the smoothness of the confidence weights generated by the Sigmoid function;

[0078] The fused feature representation is multiplied element-wise with the gated confidence to obtain the weighted enhanced features. The following relationship exists in the corresponding process:

[0079] ;

[0080] in, This represents the feature after weighting;

[0081] Top-K candidate masks and weighted enhancement operations are performed based on gated confidence to obtain the CE mask. The following relationship exists in the corresponding process:

[0082] ;

[0083] in, This indicates the number of tokens that are dynamically retained. This indicates taking the minimum value. This indicates taking the maximum value. This represents the minimum number of tokens that can be retained. This indicates the maximum number of tokens that can be retained. This represents the average confidence score. This represents the total number of tokens. Indicates the CE mask. This represents the Top-K function.

[0084] Step 4: Use the learnable sparse modulation module to perform sparse modulation processing on the weighted enhanced features and gated confidence, and use the dynamic gated bottleneck enhancement module to perform enhancement processing to obtain the final enhanced search image features.

[0085] In step 4, a learnable sparse modulation module is used to perform sparse modulation processing on the weighted enhanced features and gated confidence, and a dynamic gated bottleneck enhancement module is used for enhancement processing to obtain the final enhanced search image features. This includes the following sub-steps:

[0086] The gated confidence score is concatenated with the weighted enhanced features to obtain the concatenated features. The following relationship exists in the process:

[0087] ;

[0088] in, Indicates the features after splicing. This indicates that a feature splicing operation has been performed;

[0089] The concatenated features are processed sequentially through a multilayer perceptron and a sigmoid function to obtain the background preservation confidence score. The following relationship exists in the corresponding process:

[0090] ;

[0091] in, Indicates the confidence level of the background;

[0092] The background-preserving confidence score and the CE mask are weighted and fused to obtain a sparse modulation factor. The sparse modulation factor is then multiplied element-wise with the weighted enhanced features to obtain the sparsely modulated features. The following relationship exists in the corresponding process:

[0093] ;

[0094] in, This represents a sparse adjustment factor, which achieves the effects of forced foreground preservation and dynamic background suppression; This indicates the characteristics after sparse modulation.

[0095] An enhancement branch is constructed based on layer normalization, multilayer perceptron, and GELU activation function; a compression branch is constructed based on layer normalization, linear transformation, and GELU activation function.

[0096] The sparsely modulated features are input into the enhancement branch and the compression branch for processing, respectively, to obtain the output of the token with high confidence after the enhancement branch and the output of the token with low confidence after the compression branch. The following relationship exists in the corresponding process:

[0097] ;

[0098] in, This represents the output of the token with high confidence after the enhanced branch. This represents the output of the compressed branch after the tokens with low confidence are processed. Represents a linear layer of increased dimensionality. Represents a linear layer for dimensionality reduction;

[0099] It should be noted that the enhancement branch strengthens the tokens with high confidence in the sparsely modulated features, while the compression branch compresses the tokens with low confidence in the sparsely modulated features. The compression branch reduces the computation by first compressing the dimension of the input features and then restoring the original dimension, which reduces the redundancy of the background tokens while retaining the basic information.

[0100] Based on the gated confidence level, the output of the token with high confidence after the enhancement branch and the output of the token with low confidence after the compression branch are weighted and fused to obtain the mixed features after dynamic weighted summation. The following relationship exists in the corresponding process:

[0101] ;

[0102] in, This represents the mixed characteristics after dynamic weighted summation;

[0103] A linear layer is used to process the dynamically weighted and summed mixed features, and a residual link is performed with the sparsely modulated features to obtain the final enhanced search image features. The following relationship exists in the corresponding process:

[0104] ;

[0105] in, This represents the final enhanced search image features. This indicates that the data has been processed through a linear layer.

[0106] Step 5: Train the target tracking model using a large-scale tracking training dataset, and introduce a classification loss function and an IoU loss function based on template image features and the final enhanced search image features to obtain the trained target tracking model.

[0107] In step 5, the target tracking model is trained using a large-scale tracking training dataset. A classification loss function and an IoU loss function, constructed based on template image features and the final enhanced search image features, are introduced to obtain the trained model. The expression for the loss function during training is as follows:

[0108] ;

[0109] in, Indicates total training loss. Represents classification loss. Indicates IoU loss. Indicates L1 loss, and All of these represent hyperparameters used to balance the weights of various losses.

[0110] It should be noted that this embodiment was trained in a PyTorch environment with Torch version 2.1.1 and an RTX 5060 Ti graphics card. The network was trained using training sets from the GOT-10K, LaSOT, uav123, and TrackingNet datasets. 60,000 images were randomly sampled from the video frames in the training set for 300 training rounds. The batch size was set to 16, and the learning rate was set to 4×10⁻⁶. -5 To prevent overfitting, the learning rate is reduced after 240 training epochs.

[0111] Step 6: Use the trained target tracking model to determine the final template features and the final search features; concatenate the final template features and the final search features and input them into the encoder (which consists of 12 stacked Transformer encoders) to obtain the encoder features; input the encoder features into the classification and regression prediction head for prediction to obtain the target tracking results.

[0112] In this embodiment, the present invention segments, flattens, and linearly projects the template image and search image, then separates them into template features and search features after interaction, and fuses and enhances the search features. First, channel attention guidance is used to calibrate and adaptively weight the input features along the channel dimension to enhance the responsiveness of key channels. The weighted features are then locally enhanced through normalization, local correlation modeling, and multi-level residual fusion, achieving refined interaction and enhancement of features in the spatial dimension. Global feature modeling is used to model the context of the enhanced features, improving their global expressiveness and overcoming the limitation of local enhancement in covering global information. Second, sparse feature filtering is used to filter feature responses, highlighting foreground-related features and suppressing background noise, making the feature distribution more sparse and focused. Finally, a dynamic gating bottleneck enhancement module dynamically adjusts the enhancement based on feature importance, achieving differentiated enhancement and compression while preserving the basic information of the original features. The template features and the final search features are fused and fed into the encoder, whose output enters the classification and regression prediction head to obtain the target's state information and achieve accurate estimation of the target's position. This invention, through multi-stage enhancement, residual fusion, and dynamic adjustment mechanisms, can effectively cope with difficult scenarios such as target occlusion, complex backgrounds, scale changes, and rapid motion, thereby improving the robustness and accuracy of target tracking.

[0113] Please see Figure 5 This embodiment also provides a target tracking system based on multi-stage feature fusion and residual enhancement, wherein the system applies the target tracking method based on multi-stage feature fusion and residual enhancement as described above, and the system includes:

[0114] Build modules are used for:

[0115] A two-stage enhancement module is constructed based on the Transformer architecture; a confidence scoring module and a learnable sparse modulation module are constructed based on the multilayer perceptron and the Sigmoid function, respectively; a dynamic gating bottleneck enhancement module is constructed based on the gating mechanism; and a target tracking model is constructed using the two-stage enhancement module, the confidence scoring module, the learnable sparse modulation module, and the dynamic gating bottleneck enhancement module.

[0116] The feature extraction module is used for:

[0117] The template image and the search region image are input into a convolutional network for feature extraction and flattening, and positional encoding is added to obtain the features of the template image and the search image, respectively.

[0118] Enhancement modules are used for:

[0119] A two-stage enhancement module is used to enhance the features of the search image, and a confidence scoring module is used to generate confidence, so as to obtain the enhanced features and gated confidence after weighting.

[0120] Sparse modulation module, used for:

[0121] The learnable sparse modulation module is used to perform sparse modulation processing on the weighted enhanced features and gated confidence, and the dynamic gated bottleneck enhancement module is used for enhancement processing to obtain the final enhanced search image features.

[0122] The training module is used for:

[0123] The target tracking model is trained using a large-scale tracking training dataset, and a classification loss function and an IoU loss function are introduced based on template image features and the final enhanced search image features to obtain the trained target tracking model.

[0124] The tracking module is used for:

[0125] The final template features and final search features are determined using the trained target tracking model. The final template features and final search features are concatenated and input into the encoder to obtain the encoder features. The encoder features are then input into the classification and regression prediction head for prediction to obtain the target tracking results.

[0126] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0127] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0128] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0129] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A target tracking method based on multi-stage feature fusion and residual enhancement, characterized in that, The method includes the following steps: Step 1: Construct a two-stage enhancement module based on the Transformer architecture; construct a confidence scoring module and a learnable sparse modulation module based on the multilayer perceptron and the Sigmoid function, respectively. A dynamic gate control bottleneck enhancement module is constructed based on the gate control mechanism; A target tracking model is constructed using a two-stage enhancement module, a confidence scoring module, a learnable sparse modulation module, and a dynamic gating bottleneck enhancement module. Step 2: Input the template image and the search region image into the convolutional network for feature extraction and flattening, and add positional encoding to obtain the features of the template image and the search image respectively. Step 3: Enhance the search image features using a two-stage enhancement module and generate confidence scores using a confidence scoring module to obtain the enhanced features and gated confidence scores after weighting. This includes the following sub-steps: The search image features are processed by global average pooling to obtain global average pooling features. The global average pooling features are processed sequentially through a multilayer perceptron and a sigmoid function to obtain channel-level importance weights; The channel-level importance weights are multiplied element-wise with the search image features and then residual connections are performed to obtain the enhanced search image features. The enhanced search image features are subjected to layer normalization to obtain the normalized search image features; The normalized search image features are sequentially processed by one-dimensional depthwise convolution, GELU activation function, and one-dimensional point convolution to obtain local context-enhanced features. The normalized search image features are processed using a multilayer perceptron to obtain a high-frequency compensation term; The enhanced search image features, local context enhancement features, and high-frequency compensation terms are connected in a triple residual connection to obtain triple residual connection features. The triple residual connection features are sequentially processed through a linear layer and a sigmoid function to obtain fusion coefficients that operate independently on each token. Based on the fusion coefficients that operate independently on each token, the enhanced search image features and the triple residual connection features are adaptively weighted and fused to obtain the fused feature representation. The fused feature representation is processed based on a multi-head self-attention mechanism to obtain the features after multi-head self-attention interaction. The features after multi-head self-attention interaction are input into a multilayer perceptron for processing to obtain a confidence score; the confidence score is then processed by scaling and a sigmoid function to obtain a gated confidence score. The fused feature representation is multiplied element-wise with the gated confidence to obtain the weighted enhanced features. Top-K candidate masks and weighted enhancement operations are performed based on gate confidence to obtain the dynamically retained number of tokens and CE mask; Step 4: Use the learnable sparse modulation module to perform sparse modulation processing on the weighted enhanced features and gated confidence, and use the dynamic gated bottleneck enhancement module to perform enhancement processing to obtain the final enhanced search image features. Step 5: Train the target tracking model using a large-scale tracking training dataset, and introduce a classification loss function and an IoU loss function based on template image features and the final enhanced search image features to obtain the trained target tracking model. Step 6: Use the trained target tracking model to determine the final template features and the final search features; The final template features and the final search features are concatenated and then input into the encoder to obtain the encoder features. The encoder features are then input into the classification and regression prediction head for prediction to obtain the target tracking results.

2. The target tracking method based on multi-stage feature fusion and residual enhancement according to claim 1, characterized in that, In step 2, the template image and the search region image are input into a convolutional network for feature extraction and flattening, and positional encoding is added to obtain the template image features and the search image features, specifically including the following sub-steps: The template image and the search region image are input into a convolutional network for feature extraction in sequence to obtain the feature map of the template image and the feature map of the search image. The feature maps of the template image and the search image are flattened in the spatial dimension to obtain the template feature sequence and the search feature sequence. A linear mapping layer is used to project the template feature sequence and the search feature sequence to the same feature dimension, and learnable positional encodings are added to each to obtain template image features and search image features.

3. The target tracking method based on multi-stage feature fusion and residual enhancement according to claim 1, characterized in that, In the step of performing global average pooling on the search image features to obtain global average pooled features, the following relationship exists: ； in, This represents the global average pooling feature. This indicates that global average pooling has been applied. Indicates the search image features; In the step of processing the global average pooled features sequentially through a multilayer perceptron and a sigmoid function to obtain channel-level importance weights, the following relationship exists: ； in, Indicates channel-level importance weights. This represents the Sigmoid function. This indicates that the data has been processed by a multilayer perceptron. In the step of element-wise multiplying the channel-level importance weights with the search image features and performing residual connections to obtain the enhanced search image features, the following relationship exists: ； in, This represents the enhanced search image features. This indicates element-wise multiplication; In the step of performing layer normalization on the enhanced search image features to obtain the normalized search image features, the following relationship exists: ； in, This represents the normalized search image features. This indicates that the process has undergone layer normalization. In the process of sequentially passing the normalized search image features through one-dimensional depthwise convolution, GELU activation function processing, and one-dimensional pointwise convolution to obtain local context-enhanced features, the following relationship exists: ； in, This represents local context-enhanced features. This indicates that a one-dimensional point convolution operation has been performed. This represents the GELU activation function. This indicates that a one-dimensional depthwise convolution operation has been performed. In the step of processing the normalized search image features using a multilayer perceptron to obtain the high-frequency compensation term, the following relationship exists: ； in, Indicates high-frequency compensation terms; In the step of performing a triple residual connection on the enhanced search image features, local context enhancement features, and high-frequency compensation term to obtain triple residual connection features, the following relationship exists: ； in, This represents the triple residual connection feature.

4. The target tracking method based on multi-stage feature fusion and residual enhancement according to claim 3, characterized in that, In the steps of processing the triple residual connection features sequentially through a linear layer and a sigmoid function to obtain fusion coefficients for token-by-token independent operation, and then adaptively weighting and fusing the enhanced search image features with the triple residual connection features based on the token-by-token independent operation fusion coefficients to obtain the fused feature representation, the following relationship exists: ； in, This represents the fusion coefficient for independent operation on a token-by-token basis, used for channel-level adaptive fusion of each token; This represents the feature representation after fusion; In the step of processing the fused feature representation based on the multi-head self-attention mechanism to obtain the features after multi-head self-attention interaction, the following relationship exists: ； in, This represents the features after multi-head self-attention interaction. This indicates that the process has undergone multi-head self-attention mechanism processing; In the process of inputting the features after multi-head self-attention interaction into a multilayer perceptron for processing to obtain a confidence score, and then processing the confidence score sequentially through scaling and a sigmoid function to obtain the gated confidence score, the following relationship exists: ； in, This represents the confidence score. Indicates the gate confidence level. This represents the scaling factor, used to adjust the smoothness of the confidence weights generated by the Sigmoid function; In the step of element-wise multiplying the fused feature representation with the gated confidence to obtain the weighted enhanced features, the following relationship exists: ； in, This represents the feature enhanced by weights; In the steps of performing Top-K candidate masks and weighted enhancement operations based on gated confidence to obtain the CE mask, the following relationship exists: ； in, This indicates the number of tokens that are dynamically retained. This indicates taking the minimum value. This indicates taking the maximum value. This represents the minimum number of tokens that can be retained. This indicates the maximum number of tokens that can be retained. This represents the average confidence score. This represents the total number of tokens. Indicates the CE mask. This represents the Top-K function.

5. The target tracking method based on multi-stage feature fusion and residual enhancement according to claim 4, characterized in that, In step 4, a learnable sparse modulation module is used to perform sparse modulation processing on the weighted enhanced features and gated confidence, and a dynamic gated bottleneck enhancement module is used for enhancement processing to obtain the final enhanced search image features. Specifically, this includes the following sub-steps: The gated confidence score is concatenated with the weighted enhanced features to obtain the concatenated features. The concatenated features are processed sequentially through a multilayer perceptron and a sigmoid function to obtain the background preservation confidence score. The background retention confidence and the CE mask are weighted and fused to obtain the sparse adjustment factor; The sparse adjustment factor is multiplied element-wise with the weighted enhanced features to obtain the sparsely modulated features. An enhanced branch is constructed based on layer normalization, multilayer perceptron, and GELU activation function; A compressed branch is constructed based on layer normalization, linear transformation, and the GELU activation function; The sparsely modulated features are input into the enhancement branch and the compression branch respectively for processing, to obtain the output of the token with high confidence after the enhancement branch and the output of the token with low confidence after the compression branch respectively. Based on the gated confidence level, the output of the token with high confidence level after the enhancement branch and the output of the token with low confidence level after the compression branch are weighted and fused to obtain the mixed features after dynamic weighted summation. A linear layer is used to process the dynamically weighted summed hybrid features, and residual linking is performed with the sparsely modulated features to obtain the final enhanced search image features.

6. The target tracking method based on multi-stage feature fusion and residual enhancement according to claim 5, characterized in that, In the step of concatenating the gated confidence scores with the weighted enhanced features to obtain the concatenated features, the following relationship exists: ； in, Indicates the features after splicing. This indicates that a feature splicing operation has been performed; In the step of processing the concatenated features sequentially through a multilayer perceptron and a sigmoid function to obtain the background preservation confidence, the following relationship exists: ； in, Indicates the background confidence level; In the steps of weighted fusion of background-preserving confidence and CE mask to obtain sparse modulation factor, and element-wise multiplication of sparse modulation factor with the weighted enhanced features to obtain sparsely modulated features, the following relationship exists: ； in, This represents a sparse adjustment factor, which achieves the effects of forced foreground preservation and dynamic background suppression; This represents the characteristics after sparse modulation.

7. The target tracking method based on multi-stage feature fusion and residual enhancement according to claim 6, characterized in that, In the process of inputting the sparsely modulated features into the enhancement branch and the compression branch respectively to obtain the output of the token with high confidence after the enhancement branch and the output of the token with low confidence after the compression branch, the following relationship exists: ； in, This represents the output of the token with high confidence after the enhanced branch. This represents the output of the compressed branch after the tokens with low confidence are processed. Represents a linear layer of increased dimensionality. Represents a linear layer for dimensionality reduction; In the step of weightedly fusing the output of tokens with high confidence after the enhancement branch and the output of tokens with low confidence after the compression branch, to obtain the dynamically weighted summed hybrid features, the following relationship exists: ； in, This represents the mixed characteristics after dynamic weighted summation; In the steps of processing the dynamically weighted and summed hybrid features using a linear layer and then linking them with the sparsely modulated features using residuals to obtain the final enhanced search image features, the following relationship exists: ； in, This represents the final enhanced search image features. This indicates that the data has been processed through a linear layer.

8. The target tracking method based on multi-stage feature fusion and residual enhancement according to claim 2, characterized in that, In step 6, the target tracking model is trained using a large-scale tracking training dataset. A classification loss function and an IoU loss function, constructed based on template image features and the final enhanced search image features, are introduced to obtain the trained model. The expression for the loss function during training is as follows: ； in, Indicates total training loss. Represents classification loss. Indicates IoU loss. Indicates L1 loss, and Both represent hyperparameters.

9. A target tracking system based on multi-stage feature fusion and residual enhancement, characterized in that, The system employs the target tracking method based on multi-stage feature fusion and residual enhancement as described in any one of claims 1 to 8, and the system comprises: Build modules are used for: A two-stage enhancement module is constructed based on the Transformer architecture; a confidence scoring module and a learnable sparse modulation module are constructed based on the multilayer perceptron and the Sigmoid function, respectively; a dynamic gating bottleneck enhancement module is constructed based on the gating mechanism; and a target tracking model is constructed using the two-stage enhancement module, the confidence scoring module, the learnable sparse modulation module, and the dynamic gating bottleneck enhancement module. The feature extraction module is used for: The template image and the search region image are input into a convolutional network for feature extraction and flattening, and positional encoding is added to obtain the features of the template image and the search image, respectively. Enhancement modules are used for: A two-stage enhancement module is used to enhance the features of the search image, and a confidence scoring module is used to generate confidence, so as to obtain the enhanced features and gated confidence after weighting. Sparse modulation module, used for: The learnable sparse modulation module is used to perform sparse modulation processing on the weighted enhanced features and gated confidence, and the dynamic gated bottleneck enhancement module is used for enhancement processing to obtain the final enhanced search image features. The training module is used for: The target tracking model is trained using a large-scale tracking training dataset, and a classification loss function and an IoU loss function are introduced based on template image features and the final enhanced search image features to obtain the trained target tracking model. The tracking module is used for: The final template features and final search features are determined using the trained target tracking model. The final template features and final search features are concatenated and input into the encoder to obtain the encoder features. The encoder features are then input into the classification and regression prediction head for prediction to obtain the target tracking results.

Images