A single-target tracking method based on multi-scale attention and adaptive fusion

Abstract

This invention provides a single-target tracking method based on multi-scale attention and adaptive fusion, belonging to the field of target tracking technology. First, an anchor-free tracking framework is constructed based on Siamese networks. Target position is output through shared parameter feature extraction, combined with classification, regression, and centrality branches. Second, a multi-scale attention module enhances features in the channel and spatial dimensions. Through a spatial attention structure with multiple branches and different receptive fields, it adaptively highlights salient target regions and suppresses background redundancy, improving discrimination capabilities under occlusion, scale changes, and similar interference. Finally, an adaptive gating fusion module is proposed to dynamically weight and fuse multi-level features, reducing information redundancy and conflicts, and improving localization stability and robustness.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and in particular to a single-target tracking method based on multi-scale attention and adaptive fusion. Background Technology

[0002] In recent years, with the development of science and technology and the continuous breakthroughs in artificial intelligence, computer vision has been increasingly applied in various intelligent systems. More and more visual algorithms are being used to replace manual labor in completing simple, repetitive, time-sensitive, or high-risk tasks. In applications such as smart security, intelligent transportation, drone inspection, and industrial testing, video target tracking, as a crucial link in the visual perception chain, directly affects the effectiveness of subsequent target recognition, behavior analysis, and decision control. As application areas continue to expand, the requirements for the real-time performance, accuracy, and stability of target tracking algorithms are constantly increasing. In particular, trackers need to be able to lock onto targets stably and for extended periods in complex environments. Therefore, achieving high-precision and robust target tracking in complex scenarios has significant research value and application prospects.

[0003] Single Object Tracking (SOT) refers to continuously predicting the target's position and scale changes in subsequent frames after the initial position of the target is given in the first frame of a video sequence, providing a continuous and reliable target state estimate for intelligent systems. Target tracking technology has a long history, evolving from early methods relying on handcrafted features and correlation filtering to end-to-end tracking frameworks based on deep learning in recent years, with continuous advancements in related research. Traditional tracking methods have advantages in computational efficiency and can meet real-time requirements to a certain extent. However, in practical applications, factors such as occlusion, drastic scale changes, cluttered backgrounds, interference from similar targets, lighting variations, and motion blur often exist. These factors significantly affect feature representation and matching stability, ultimately leading to tracker drift, inaccurate positioning, or even target loss, thus reducing the overall accuracy and robustness of the system. Therefore, how to suppress interference, highlight the target region, and improve tracking stability under complex backgrounds and multiple challenging factors has become a key research direction in target tracking.

[0004] Currently, deep learning has significantly improved the accuracy of target tracking. Transformer-type methods, with their global modeling capabilities, possess strong feature representation and relationship modeling abilities in complex scenes; however, their model parameter count and computational complexity are typically high, limiting their engineering deployment and real-time applications. In contrast, Siamese Network tracking methods are simple in structure and computationally efficient, becoming one of the mainstream approaches in target tracking. Methods represented by anchor-free Siamese tracking frameworks such as SiamCAR achieve efficient matching through shared parameter template branches and search branches, and combine classification, regression, and centrality mechanisms to complete target localization, balancing speed and accuracy. However, in complex scenes, Siamese Networks still suffer from problems such as the weakening of key features, difficulty in effectively suppressing background redundancy, and a lack of adaptability in multi-level feature fusion methods. Especially under conditions of occlusion, rapid scale changes, and strong background interference, they are prone to tracking drift and short-term failures. Furthermore, some methods are sensitive to the distribution of training data, have limited cross-scene generalization capabilities, and struggle to stably adapt to more challenging application scenarios such as drone aerial photography and low-resolution applications. Based on the above problems, there is an urgent need for a target tracking scheme that can adaptively highlight key target features and dynamically fuse multi-level information while ensuring high efficiency, so as to improve tracking accuracy and robustness in complex scenarios. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a single-target tracking method based on multi-scale attention and adaptive fusion; based on a Siamese network anchorless tracking framework, it reduces drift, mistracking, and loss phenomena during the tracking process in complex scenarios, thereby improving the accuracy and robustness of the target tracking algorithm in complex scenarios.

[0006] On the one hand, this invention provides a single-target tracking method based on multi-scale attention and adaptive fusion, comprising the following steps:

[0007] Step 1: Construct an anchorless single-target tracking framework based on Siamese networks;

[0008] The target location prediction is performed using the anchorless twin tracking baseline SiamCAR, and improvements are made on the baseline.

[0009] Specifically, the process involves: obtaining the initial bounding box of the target to be tracked in the initial frame of the video sequence; expanding the initial bounding box outwards to obtain a template cropping region, thus obtaining a template image; in subsequent frames of the video sequence, determining the search center position of the current frame based on the tracking result output from the previous frame, and generating a search cropping region centered on the search center position to obtain a search image. The template image and the search image are then input into a ResNet-50 feature extraction network with shared parameters to extract multi-level template features and multi-level search features, respectively; the template features and search features are matched through a cross-correlation operation to generate a feature map for the prediction head input, denoted as... Where C, H, and W represent the number of channels, height, and width of the feature map, respectively. It serves as the common input feature map for the classification branch, regression branch, and centrality branch.

[0010] The system includes a classification branch, a regression branch, and a centrality branch. The classification branch is used for foreground and background discrimination, the regression branch is used for bounding box regression, and the centrality branch is used to suppress anomalous responses that deviate from the target center, thereby improving localization stability. Specifically, the input feature map... The output feature map is obtained after classification branching. The two-dimensional vector at each position (i, j) represents the foreground probability and background probability, respectively; the regression branch outputs a feature map. Each position (i, j) corresponds to a four-dimensional vector (l, t, r, b), representing the distance from that point to the left, top, right, and bottom sides of the predicted bounding box, respectively. Here, W is the feature map width, H is the feature map height, i∈{1,…,H}, and j∈{1,…,W} represent the spatial location index.

[0011] Step 2: Design a multi-scale attention module (MSAM) for feature enhancement. Use ResNet-50 as the backbone network and divide it into multiple convolutional blocks conv1, conv2, conv3, conv4, and conv5. After embedding the MSAM into the output of the last convolutional block conv5 of the backbone network, perform attention enhancement on the deep features of the template branch and the search branch respectively.

[0012] Step 2.1: Construct the channel attention submodule;

[0013] By performing global pooling aggregation on template features and search features in the spatial dimension, a channel-level global description vector is obtained; channel weights are generated based on the channel-level global description vector, and feature channels are adaptively recalibrated.

[0014] Specifically, let the template feature map output by the feature extraction network in the last layer be... The search feature map is Where C represents the number of channels, and H and W represent the height and width of the feature map, respectively. For the template feature map... and search feature map Independent computation of channel attention weights: For each feature map, two distinct channel description vectors are obtained by compression along the spatial dimension using global average pooling and global max pooling. These vectors are then fed into a shared multilayer perceptron for feature transformation. The shared multilayer perceptron consists of two fully connected layers connected in series, with a non-linear activation function between them. The two channel features output from the shared multilayer perceptron are summed element-wise and normalized using a sigmoid activation function to obtain the channel attention weight vectors for the corresponding feature maps. Finally, these weight vectors are multiplied element-wise by each channel of the corresponding feature map to obtain the recalibrated template feature maps. Or search feature maps This allows for weighted adjustments to the characteristics of different channels;

[0015] Step 2.2: Construct a multi-scale spatial attention submodule, generate a spatial attention map using convolutional branches with different receptive fields, and perform adaptive weighted fusion;

[0016] Step 2.2.1: Spatial feature encoding;

[0017] For the recalibrated template feature map Or search feature maps First, average pooling and max pooling are performed along the channel dimension to obtain two spatial feature maps. These two maps are then concatenated along the channel dimension to form a dual-channel fused feature. ;

[0018] Step 2.2.2: Multi-branch spatial attention generation; design multiple parallel branches, each branch using a convolutional kernel of a different size;

[0019] Specifically, the fused feature map obtained by stitching together the channel dimensions... As input, the data are fed into different branches for processing. A 3×3 convolution kernel is used to focus on fine-grained local structures; a 7×7 convolution kernel is used to capture broader contextual information through dense sampling modeling; in addition, a 3×3 dilated convolution is introduced with a dilation rate of 2 to expand the receptive field to 5×5, enhancing the ability to perceive medium-scale spatial patterns; the output of each branch is normalized by the Sigmoid function to generate the corresponding spatial attention map.

[0020] Step 2.2.3: Adaptive branch weight fusion;

[0021] Introducing learnable weight parameters K represents the number of branches, and the weights are normalized using Softmax to ensure that the sum of the weights is 1. The spatial attention maps of each branch are weighted and summed to obtain the final fused spatial attention map. ;

[0022] Step 2.2.4: Spatial attention recalibration;

[0023] The fused spatial attention map M s Acting on feature map or feature map Recalibration in spatial dimensions is achieved through element-wise multiplication;

[0024] Step 2.3: Attention-enhanced feature output;

[0025] The channel attention module and the multi-scale spatial attention module are concatenated and applied to the template feature map respectively. Search Feature Map Output the enhanced template features and search features;

[0026] Step 3: Design the Adaptive Gated Fusion Module (AGFM) for multi-level fusion;

[0027] Multi-level fusion of features from three layers in the backbone network is performed, corresponding to the outputs of ResNet-50's conv3, conv4, and conv5 after passing through a multi-scale attention module. The fusion weights of the response features at each layer are dynamically learned to achieve information complementarity and redundancy suppression.

[0028] Step 3.1: Feature aggregation;

[0029] Let the response characteristics of layer l be... The previous layer's fused output features are Initially, the features from the first layer are used as the starting fusion result, and the features from the last layer are the features enhanced by the multi-scale attention module. The two sets of features are concatenated along the channel dimension: one set is the accumulated features obtained from the previous layer fusion, and the other set is the newly generated features from the current layer. The two sets are concatenated along the channel dimension to obtain the joint feature map. Where C represents the number of channels, and H and W represent the height and width of the feature map.

[0030] Step 3.2: Dynamic weight generation;

[0031] Joint feature map The data is fed into a gating network to generate two weight maps, which respectively represent the contribution of the fused features from the previous layer and the features of the current layer in the current scene, thus achieving dynamic allocation; the gating network consists of two 1×1 convolutional layers and a non-linear activation function.

[0032] Step 3.3: Perform adaptive weighted fusion;

[0033] The two features are summed element-wise according to their corresponding weights to obtain the fused features. By performing gated fusion iteratively layer by layer, more reliable feature levels can be adaptively selected under conditions of occlusion, scale variation, and background interference.

[0034] Step 4: Target localization and tracking output based on fused features;

[0035] Through multi-level fusion features A highly discriminative response feature map is generated, where L is the deepest level. This response feature map serves as the shared input for the classification branch, regression branch, and centrality branch, outputting foreground and background confidence maps, bounding box offset maps, and centrality response maps, respectively. The foreground and background confidence maps predict the probability that each location belongs to the foreground or background. The bounding box offset map indicates the suppression of low-quality predictions far from the target center. The centrality response map represents the distance directly regressed to the target bounding box. Finally, the classification score is multiplied by the centrality to obtain the final response map. The location with the highest score on the response map is selected as the target center, and the final tracking box is decoded by combining the corresponding offset, thus achieving single-target tracking.

[0036] On the other hand, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the single-target tracking method based on multi-scale attention and adaptive fusion.

[0037] Thirdly, this application proposes a computer program product, including a computer program or instructions that, when executed by a processor, implement the single-target tracking method based on multi-scale attention and adaptive fusion.

[0038] The beneficial effects of adopting the above technical solution are as follows:

[0039] This invention provides a single-target tracking method based on multi-scale attention and adaptive fusion. Compared with existing anchorless single-target tracking methods based on Siamese networks, this invention introduces a multi-scale attention module (MSAM) in the feature extraction stage to jointly model channel and spatial saliency. This adaptively highlights key target regions and suppresses redundant background information, thereby reducing mismatches and drift in scenarios with occlusion, cluttered backgrounds, and interference from similar targets. By introducing an adaptive gated fusion module (AGFM) in the multi-level feature fusion stage, fusion weights can be dynamically allocated based on the complementarity and redundancy of features at different levels, avoiding information conflicts caused by direct splicing in existing technologies and improving localization stability and robustness. Simultaneously, by combining anchorless classification branches, regression branches, and centrality branches to suppress abnormal responses, this invention reduces mistracking and loss caused by pseudo-peaks, improving tracking accuracy and stability in complex scenarios while maintaining the computational efficiency of Siamese networks. Attached Figure Description

[0040] Figure 1 An overall structural diagram provided for a specific embodiment of the present invention;

[0041] Figure 2 A multi-scale attention structure diagram provided for specific embodiments of the present invention;

[0042] Figure 3 A multi-scale spatial attention structure diagram provided for a specific embodiment of the present invention;

[0043] Figure 4 An adaptive gating fusion structure diagram provided for a specific embodiment of the present invention;

[0044] Figure 5 Comparison chart of ablation experiments using the OTB100 dataset provided for specific embodiments of the present invention;

[0045] (a) - Success rate curve, (b) - Accuracy rate curve;

[0046] Figure 6 Comparison chart of ablation experiments using the UAV123 dataset provided for specific embodiments of the present invention;

[0047] (a) - Success rate curve, (b) - Accuracy rate curve;

[0048] Figure 7 A comparison chart of different sequence tracking results in the OTB100 dataset provided for specific embodiments of the present invention;

[0049] Among them, (a)-Coupon sequence, (b)-Liquor sequence, and (c)-Vase sequence;

[0050] Figure 8A comparison chart of tracking performance of different sequences in the UAV123 dataset provided for specific embodiments of the present invention;

[0051] Among them, (a) is the car5 sequence, (b) is the person18 sequence, and (c) is the truck1 sequence. Detailed Implementation

[0052] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0053] Example 1:

[0054] On the one hand, this invention provides a single-target tracking method based on multi-scale attention and adaptive fusion, such as... Figure 1 As shown, it includes the following steps:

[0055] Step 1: Construct an anchorless single-target tracking framework based on Siamese networks;

[0056] In the target tracking system, a Siamese network framework is used to achieve efficient matching between template images and search images, and target localization is achieved through classification and regression. Specifically, the anchor-free Siamese tracking baseline SiamCAR is used for target localization prediction, and improvements are made on the baseline. Since the bounding box prediction accuracy is low in areas far from the target center, it affects the overall tracking performance. To address this, SiamCAR introduces a center-ness branch on top of the classification branch to suppress anomalous responses in areas far from the center, thereby improving localization accuracy and tracking robustness. The improved model structure is as follows: Figure 1 As shown;

[0057] In the initial frame of the video sequence, the initial bounding box of the target to be tracked is obtained. Based on the initial bounding box, a template cropping region is obtained by expanding outwards, thus obtaining a template image. In subsequent frames of the video sequence, the search center position of the current frame is determined based on the tracking result output from the previous frame, and a search cropping region is generated centered on the search center position, resulting in a search image. The template image and the search image are respectively input into a feature extraction network with shared parameters. Specifically, the SiamCAR framework, a single-target tracking framework without anchor boxes based on Siam networks, is used to extract multi-level template features and multi-level search features. The template features and search features are matched through a cross-correlation operation to generate a feature map for the prediction head input, denoted as . Where C, H, and W represent the number of channels, height, and width of the feature map, respectively. It serves as the common input feature map for the classification branch, regression branch, and centrality branch.

[0058] The system includes a classification branch, a regression branch, and a centrality branch. The classification branch is used for foreground and background discrimination, the regression branch is used for bounding box regression, and the centrality branch is used to suppress anomalous responses that deviate from the target center, thereby improving localization stability. Specifically, the input feature map... The output feature map is obtained after classification branching. The two-dimensional vector at each position (i, j) represents the foreground probability and background probability, respectively; the regression branch outputs a feature map. Each position (i, j) corresponds to a four-dimensional vector (l, t, r, b), representing the distance from that point to the left, top, right, and bottom sides of the predicted bounding box, respectively. Here, W is the feature map width, H is the feature map height, i∈{1,…,H}, and j∈{1,…,W} represent the spatial location index.

[0059] Step 2: Design a multi-scale attention module (MSAM) for feature enhancement;

[0060] In complex scenes, redundant background information and similar interference can easily lead to insufficient feature representation. To highlight key target regions and suppress irrelevant background, this invention employs a multi-scale attention module to enhance features during the feature extraction stage. For example... Figure 2 As shown, this invention uses ResNet-50 as the backbone network and divides it into multiple convolutional blocks conv1, conv2, conv3, conv4, and conv5. To maximize feature enhancement while maintaining computational efficiency, the MSAM module is embedded after the output of the last convolutional block conv5 of the backbone network, and attention enhancement is applied to the deep features of the template branch and the search branch respectively. The features at this position have strong semantic expressive power, and after MSAM enhancement, they can effectively highlight the discriminative region of the target, while avoiding introducing too much computational overhead on shallow features.

[0061] Step 2.1: Construct the channel attention submodule;

[0062] By performing global statistics on template features and search features, channel weights are generated and feature channels are adaptively recalibrated, thereby strengthening channels that contribute more to target discrimination and suppressing redundant channel information, thus improving feature discrimination ability.

[0063] Specifically, let the template feature map output by the feature extraction network in the last layer be... The search feature map is Where C represents the number of channels, and H and W represent the height and width of the feature map, respectively. For the template feature map... and search feature map Independent computation of channel attention weights: For each feature map, two distinct channel description vectors are obtained by compressing along the spatial dimension using global average pooling and global max pooling. These vectors are then fed into a shared multilayer perceptron (SLP) for feature transformation. The SLP consists of two fully connected layers cascaded together, with a ReLU activation function between them to enhance feature representation capabilities. The two channel features output from the SLP are element-wise summed and normalized using a sigmoid activation function to obtain the corresponding channel attention weight vector for each feature map. Finally, this weight vector is multiplied element-wise by each channel of the corresponding feature map to obtain the recalibrated template feature maps. Or search feature maps This allows for weighted adjustments of features across different channels. This operation effectively highlights key channels and suppresses redundant channels, thereby improving the model's expressive power and discriminative performance.

[0064] Step 2.2: Construct a multi-scale spatial attention submodule, using convolutional branches with different receptive fields to generate a spatial attention map, and perform adaptive weighted fusion to enable the system to capture salient target regions at different scales and in different local ranges, improving response stability under scale variations and local occlusion conditions. For example... Figure 3 As shown, the specific process is as follows:

[0065] Step 2.2.1: Spatial feature encoding;

[0066] For the recalibrated template feature map Or search feature maps First, average pooling and max pooling are performed along the channel dimension to obtain two spatial feature maps. This operation can capture the overall distribution and local extremum information respectively, which helps in the subsequent generation of spatial attention. Then, the two maps are concatenated along the channel dimension to form a dual-channel fused feature. .

[0067] Step 2.2.2: Multi-branch spatial attention generation;

[0068] The core idea of ​​spatial attention is to compress the input feature map along the channel dimension to obtain saliency information of spatial distribution. To capture spatial relationships at different scales, multiple parallel branches are designed, each using a convolutional kernel of a different size (such as 3×3 and 7×7) to adapt to the modeling needs of local and global spatial information.

[0069] Specifically, the fused feature map obtained by stitching together the channel dimensions As input, data is fed into different branches for processing. A 3×3 convolution kernel is used to focus on fine-grained local structures; a 7×7 convolution kernel is used to capture broader contextual information through dense sampling modeling, effectively mitigating attention shifts caused by target deformation or cluttered backgrounds. Furthermore, a 3×3 dilated convolution is introduced with a dilation rate of 2 to expand the receptive field to 5×5 while maintaining computational efficiency, enhancing the ability to perceive medium-scale spatial patterns. The output of each branch is normalized by a sigmoid function to generate the corresponding spatial attention map.

[0070] Step 2.2.3: Adaptive branch weight fusion;

[0071] To achieve adaptive fusion between branches, learnable weight parameters are introduced. K represents the number of branches, and the weights are normalized using Softmax to ensure that the sum of the weights is 1. The spatial attention maps of each branch are weighted and summed to obtain the final fused spatial attention map. This mechanism allows the model to dynamically adjust the contribution of spatial features at different scales according to task requirements, improving the flexibility and expressive power of feature fusion.

[0072] Step 2.2.4: Spatial attention recalibration;

[0073] The fused spatial attention map M s Acting on feature map or feature map This method achieves spatial recalibration through element-wise multiplication. This operation effectively highlights key regions, suppresses redundant background, and thus optimizes feature distribution, providing more discriminative feature representations for subsequent tasks.

[0074] Step 2.3: Attention-enhanced feature output;

[0075] The channel attention module and the multi-scale spatial attention module are concatenated and applied to the template feature map respectively. Search Feature Map The enhanced template features and search features are output, providing more reliable input features for subsequent cross-correlation matching and reducing false matches caused by background interference.

[0076] Step 3: Design the Adaptive Gated Fusion Module (AGFM) for multi-level fusion;

[0077] Existing technologies for multi-layer feature fusion typically employ direct addition or concatenation, which struggles to adaptively handle redundancy and conflicts between different layers. To address this, this invention proposes an adaptive gating fusion module that performs multi-level fusion on features from three layers (L=3) in the backbone network, corresponding to the outputs of ResNet-50's conv3, conv4, and conv5 after passing through a multi-scale attention module. The module dynamically learns the fusion weights for each layer's response features, achieving information complementarity and redundancy suppression. Figure 4 As shown, the specific process is as follows:

[0078] Step 3.1: Feature aggregation;

[0079] Let the response characteristics of layer l be... The previous layer's fused output features are Initially, the features from the first layer are used as the starting fusion result, while the features from the last layer are enhanced by a multi-scale attention module. To construct a comprehensive fusion context, this embodiment concatenates the features from both layers along the channel dimension: one set consists of the accumulated features obtained from the previous layer's fusion, and the other set consists of the newly generated features from the current layer. This concatenation along the channel dimension yields a joint feature map. , where H and W represent the height and width of the feature map.

[0080] Step 3.2: Dynamic weight generation;

[0081] In this embodiment, the joint feature map The input is fed into a gating network, generating two weight maps to represent the contributions of the fused features from the previous layer and the features from the current layer in the current scene, respectively, thus achieving dynamic allocation. This network consists of two 1×1 convolutional layers and a non-linear activation function.

[0082] Step 3.3: Adaptive weighted fusion;

[0083] The two features are summed element-wise according to their corresponding weights to obtain the fused features. By performing gated fusion iteratively layer by layer, the system can adaptively select more reliable feature layers under conditions of occlusion, scale changes, and background interference, thereby improving positioning accuracy and robustness. This design allows the network to dynamically adjust the weights of historical and new information at each spatial location, achieving more flexible and accurate feature fusion.

[0084] Step 4: Target localization and tracking output based on fused features;

[0085] The multi-level fusion features obtained through the adaptive gating fusion module AGFM The deepest level, L, is input to the cross-correlation matching module to generate a highly discriminative response feature map. This feature map serves as the shared input for the classification, regression, and centrality branches, outputting foreground and background confidence maps, bounding box offset maps, and centrality response maps, respectively. The foreground / background confidence maps primarily predict the probability that each location belongs to the foreground or background. The bounding box offset map indicates the suppression of low-quality predictions far from the target center. The centrality response map represents the distance directly regressed to the target bounding box. Finally, the classification score is multiplied by the centrality to obtain the final response map. The location with the highest score on the response map is selected as the target center, and the final tracking box is decoded using the corresponding offset.

[0086] Because this invention effectively suppresses background interference and redundant channels through a multi-scale attention module, and dynamically integrates complementary information from details to semantics using an adaptive gating fusion module, it significantly improves feature discrimination and localization stability. Therefore, in complex scenes (such as target deformation, partial occlusion, interference from similar objects, or rapid movement), it can effectively reduce tracking drift, mistracking, and target loss, achieving high-precision and highly robust single-target tracking.

[0087] Example 2:

[0088] By writing a program in Python under the Linux system, a single target tracking method based on multi-scale attention and adaptive fusion was implemented according to the above process. The OTB100 and UAV123 datasets were used as examples, and the comparison with the original model is shown in Table 1.

[0089] Table 1. Comparison of ablation experimental performance of various model variants:

[0090] Table 1 shows that the gradual introduction of multi-scale attention modules and adaptive gating fusion modules significantly improves the model's tracking performance. After introducing the adaptive gating fusion module, SR and PR increase to 0.640 and 0.866 on OTB100, and to 0.582 and 0.797 on UAV123, demonstrating that the fusion and selection of multi-level features have a positive effect on improving localization accuracy and robustness. After introducing the multi-scale attention module into the backbone network, SR and PR increase to 0.649 and 0.865 on OTB100, and to 0.586 and 0.790 on UAV123, indicating that enhanced feature representation improves attention to target regions in complex scenes and effectively suppresses background interference. After integrating the two modules, the SR and PR were further improved by 0.019 and 0.033 on the OTB100, and further improved to 0.029 and 0.035 on the UAV123, achieving optimal performance.

[0091] like Figure 5 and Figure 6 As shown, (a) is the success rate curve and (b) is the accuracy curve. For the OTB100 and UAV123 datasets, all models exhibit a consistent trend in success rate and accuracy as the evaluation threshold changes. Specifically, as the overlap threshold gradually increases, the success rate of all models decreases, but the complete model D consistently maintains the highest success rate at all thresholds. This indicates that the model is more stable in target location prediction, especially exhibiting stronger robustness in complex scenarios such as occlusion and scale changes. Meanwhile, in the accuracy curve, the accuracy of all models increases with the increase of the positioning error threshold. However, the complete model D consistently outperforms the other models, achieving more accurate target boundary positioning in both traditional scenarios and UAV platforms. The above results fully demonstrate that the proposed multi-scale attention module and adaptive gating fusion module work synergistically to effectively improve the model's discrimination ability and positioning accuracy, giving it better generalization ability and robustness in complex environments.

[0092] To more intuitively demonstrate the performance differences of various tracking algorithms under different challenge attributes, this paper further employs visualization to analyze the experimental results. Three video sequences—Coupon, Liquor, and Vase—were selected from the OTB100 dataset, and three video sequences—car5, person18, and truck1—were selected from the UAV123 dataset. The benchmark algorithm and the algorithm proposed in this paper were used to track the videos, and the tracking bounding boxes of each tracker are displayed on the video frames, allowing for a more intuitive analysis of the tracking performance of each tracker. In the diagram, the purple-red boxes represent the benchmark algorithm, and the green boxes represent the algorithm proposed in this paper.

[0093] like Figure 7 and Figure 8 As shown, Figure 7 (a) is the Coupon sequence, (b) is the Liquor sequence, and (c) is the Vase sequence. Figure 8(a) is the car5 sequence, (b) is the person18 sequence, and (c) is the truck1 sequence. For different video sequences, the improved model demonstrates more accurate and stable tracking capabilities. In the Coupon sequence, despite complex environments such as occlusion and rapid target movement, the proposed algorithm can still stably track the target, exhibiting strong robustness. In the Liquor sequence, due to the fast target movement, the baseline model experienced tracking failures, while the improved model accurately captured and continuously tracked the target, significantly improving the tracking success rate. For the Vase sequence, due to the cluttered background, the baseline model showed insufficient accuracy in target localization, while the improved model effectively distinguished the target from the background, achieving more accurate tracking. Figure 8 As shown, significant viewpoint changes exist in sequences such as car5, person18, and truck1. In these scenarios, the baseline model often exhibits low tracking accuracy or even target loss, while the improved model better adapts to viewpoint changes, accurately and stably tracking the target. Therefore, regardless of complex scenarios such as rapid target movement, occlusion, cluttered backgrounds, or viewpoint changes, the improved model demonstrates higher tracking accuracy and robustness, fully validating the effectiveness and broad adaptability of the proposed method in improving target tracking performance.

[0094] Further analysis revealed that the multi-scale attention module can capture salient regions of the target at different spatial scales, enhancing the model's adaptability to scale variations and complex backgrounds. The adaptive gating fusion module, by dynamically adjusting feature fusion weights, effectively filters useful information, suppresses redundancy and conflicts, and strengthens feature complementarity. The synergistic effect of these two modules significantly enhances the model's feature representation capabilities and fusion flexibility, providing a solid technical foundation for subsequent target tracking tasks in complex scenarios.

[0095] Example 3:

[0096] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0097] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of a single target tracking method based on multi-scale attention and adaptive fusion as described in various embodiments of this application.

[0098] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, and when executed by a processor, they can implement the various steps of the aforementioned single-target tracking method based on multi-scale attention and adaptive fusion.

[0099] Example 4:

[0100] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the single-target tracking method based on multi-scale attention and adaptive fusion.

[0101] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0102] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0103] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of the methods disclosed herein and their equivalents, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A single-target tracking method based on multi-scale attention and adaptive fusion, characterized in that, Includes the following steps: Step 1: Construct an anchorless single-target tracking framework based on Siamese networks; The target location prediction is performed using the anchorless twin tracking baseline SiamCAR, and improvements are made on the baseline. Step 2: Design a multi-scale attention module (MSAM) for feature enhancement. Use ResNet-50 as the backbone network and divide it into multiple convolutional blocks conv1, conv2, conv3, conv4, and conv5. After embedding the MSAM into the output of the last convolutional block conv5 of the backbone network, perform attention enhancement on the deep features of the template branch and the search branch respectively. Step 3: Design the Adaptive Gated Fusion Module (AGFM) for multi-level fusion; Multi-level fusion of features from three layers in the backbone network is performed, corresponding to the outputs of ResNet-50's conv3, conv4, and conv5 after passing through a multi-scale attention module. The fusion weights of the response features at each layer are dynamically learned to achieve information complementarity and redundancy suppression. Step 4: Target localization and tracking output based on fused features; Through multi-level fusion features A highly discriminative response feature map is generated, where L is the deepest level. This response feature map serves as the shared input for the classification branch, regression branch, and centrality branch, outputting foreground and background confidence maps, bounding box offset maps, and centrality response maps, respectively. The foreground and background confidence maps predict the probability that each location belongs to the foreground or background. The bounding box offset map indicates the suppression of low-quality predictions far from the target center. The centrality response map represents the distance directly regressed to the target bounding box. Finally, the classification score is multiplied by the centrality to obtain the final response map. The location with the highest score on the response map is selected as the target center, and the final tracking box is decoded by combining the corresponding offset, thus achieving single-target tracking.

2. The single-target tracking method based on multi-scale attention and adaptive fusion according to claim 1, characterized in that, Step 1 specifically involves: Specifically, the process involves: obtaining the initial bounding box of the target to be tracked in the initial frame of the video sequence; expanding the initial bounding box outwards to obtain a template cropping region, thereby obtaining a template image; in subsequent frames of the video sequence, determining the search center position of the current frame based on the tracking result output from the previous frame, and generating a search cropping region centered on the search center position to obtain a search image; and inputting the template image and the search image into a feature extraction network ResNet-50 with shared parameters to extract multi-level template features and multi-level search features, respectively. Template features and search features are matched through a cross-correlation operation to generate a feature map for predicting the head input, denoted as . Where C, H, and W represent the number of channels, height, and width of the feature map, respectively. The feature map serves as the common input for the classification branch, regression branch, and centrality branch; The system includes a classification branch, a regression branch, and a centrality branch. The classification branch is used for foreground and background discrimination, the regression branch is used for bounding box regression, and the centrality branch is used to suppress anomalous responses that deviate from the target center, thereby improving localization stability. Specifically, the input feature map... The output feature map is obtained after classification branching. The two-dimensional vector at each position (i, j) represents the foreground probability and background probability, respectively; the regression branch outputs a feature map. Each position (i, j) corresponds to a four-dimensional vector (l, t, r, b), which represents the distance from the point to the left, top, right, and bottom sides of the predicted bounding box, respectively; where W is the feature map width, H is the feature map height, i∈{1, ..., H}, and j∈{1, ..., W} represent the spatial location index.

3. The single-target tracking method based on multi-scale attention and adaptive fusion according to claim 2, characterized in that, Step 2 specifically includes the following steps: Step 2.1: Construct the channel attention submodule; By performing global pooling aggregation on template features and search features in the spatial dimension, a channel-level global description vector is obtained; channel weights are generated based on the channel-level global description vector, and feature channels are adaptively recalibrated. Specifically, let the template feature map output by the feature extraction network in the last layer be... The search feature map is Where C represents the number of channels, and H and W represent the height and width of the feature map, respectively; for the template feature map and search feature map Independent computation of channel attention weights: For each feature map, two distinct channel description vectors are obtained by compression along the spatial dimension using global average pooling and global max pooling. These vectors are then fed into a shared multilayer perceptron for feature transformation. The shared multilayer perceptron consists of two fully connected layers connected in series, with a non-linear activation function between them. The two channel features output from the shared multilayer perceptron are summed element-wise and normalized using a sigmoid activation function to obtain the channel attention weight vectors for the corresponding feature maps. Finally, these weight vectors are multiplied element-wise by each channel of the corresponding feature map to obtain the recalibrated template feature maps. Or search feature maps This allows for weighted adjustments to the characteristics of different channels; Step 2.2: Construct a multi-scale spatial attention submodule, generate a spatial attention map using convolutional branches with different receptive fields, and perform adaptive weighted fusion; Step 2.3: Attention-enhanced feature output; The channel attention module and the multi-scale spatial attention module are concatenated and applied to the template feature map respectively. Search Feature Map The output includes enhanced template features and search features.

4. The single-target tracking method based on multi-scale attention and adaptive fusion according to claim 3, characterized in that, Step 2.2 specifically includes the following steps: Step 2.2.1: Spatial feature encoding; For the recalibrated template feature map Or search feature maps First, average pooling and max pooling are performed along the channel dimension to obtain two spatial feature maps. These two maps are then concatenated along the channel dimension to form a dual-channel fused feature. ; Step 2.2.2: Multi-branch spatial attention generation; design multiple parallel branches, each branch using a convolutional kernel of a different size; Specifically, the fused feature map obtained by stitching together the channel dimensions... As input, the data are fed into different branches for processing. A 3×3 convolution kernel is used to focus on fine-grained local structures; a 7×7 convolution kernel is used to capture broader contextual information through dense sampling modeling; in addition, a 3×3 dilated convolution is introduced with a dilation rate of 2 to expand the receptive field to 5×5, enhancing the ability to perceive medium-scale spatial patterns; the output of each branch is normalized by the Sigmoid function to generate the corresponding spatial attention map. Step 2.2.3: Adaptive branch weight fusion; Introducing learnable weight parameters K is the number of branches, and the weights are normalized using Softmax to ensure that the sum of the weights is 1. The spatial attention maps of each branch are weighted and summed to obtain the final fused spatial attention map. ; Step 2.2.4: Spatial attention recalibration; The fused spatial attention map M s Acting on feature map or feature map Recalibration in spatial dimensions is achieved through element-wise multiplication.

5. The single-target tracking method based on multi-scale attention and adaptive fusion according to claim 1, characterized in that, Step 3 includes the following steps: Step 3.1: Feature aggregation; Let the response characteristics of layer l be... The previous layer's fused output features are Initially, the features from the first layer are used as the starting fusion result, and the features from the last layer are the features enhanced by the multi-scale attention module. The two sets of features are concatenated along the channel dimension: one set is the accumulated features obtained from the previous layer fusion, and the other set is the newly generated features from the current layer. The two sets are concatenated along the channel dimension to obtain the joint feature map. Where C represents the number of channels, and H and W represent the height and width of the feature map; Step 3.2: Dynamic weight generation; Joint feature map The data is fed into a gating network to generate two weight maps, which respectively represent the contribution of the fused features from the previous layer and the features of the current layer in the current scene, thus achieving dynamic allocation; the gating network consists of two 1×1 convolutional layers and a non-linear activation function. Step 3.3: Perform adaptive weighted fusion; The two features are summed element-wise according to their corresponding weights to obtain the fused features. By performing gated fusion iteratively layer by layer, it is possible to adaptively select more reliable feature layers under conditions of occlusion, scale changes and background interference.

6. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed, cause the processor to perform a single-target tracking method based on multi-scale attention and adaptive fusion as described in any one of claims 1-5.

7. A computer program product, characterized in that, Includes a computer program or instructions that, when executed by a processor, implement the single-target tracking method based on multi-scale attention and adaptive fusion as described in any one of claims 1-5.

Images