Target attitude tracking method based on enhanced Mama attention network

Abstract

The invention discloses a target attitude tracking method based on an enhanced Mama attention network, belongs to the technical field of computer vision, and aims to solve the problems that precision and efficiency are difficult to balance, dynamic scene adaptability is poor and semantic and spatial synchronization cannot be considered in an existing target attitude tracking method. Through deep collaboration of feature extraction and preliminary aggregation, time sequence dependence modeling based on Mama, feature optimization of attention enhancement, four-branch multi-scale adaptive attention mask generation, dynamic feature fusion and enhancement, and attitude output and assessment multi-link technical innovation, a multi-link attitude estimation method is provided. The three core problems that precision and efficiency are difficult to balance, dynamic complex scene adaptability is poor and semantic space cannot be synchronized in an existing target attitude tracking method are solved in a targeted mode, and a better solution is provided for attitude tracking requirements of industrial and other actual scenes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to target pose tracking technology, and more particularly to a target pose tracking method based on an enhanced Mamba attention network, which can be applied to scenarios requiring high-precision, real-time pose estimation, such as industrial automation (e.g., precise grasping by robotic arms), augmented reality (AR), and autonomous driving. Background Technology

[0002] Target pose tracking, as a core technology in computer vision, has irreplaceable application value in key scenarios such as industrial robotic arm assembly, AR / VR fusion, and autonomous driving environmental perception. In industrial production, robotic arms rely on real-time pose tracking to achieve millimeter-level precision in grasping and assembly operations, ensuring the accuracy of the production process. AR systems need to maintain spatial consistency between virtual objects and the real environment through stable estimation of target poses in dynamic scenes, improving the user interaction experience. In the field of autonomous driving, sub-pixel-level real-time tracking of pose changes of surrounding pedestrians, vehicles, and other objects is required to provide reliable environmental information support for decision-making systems.

[0003] Currently, deep learning-based pose tracking methods have become mainstream research, with the paper "Alphapose: Whole-body regional multi-person pose estimation and tracking in real-time," published in *IEEE Transactions on Pattern Analysis and Machine Intelligence* in 2023, being a representative example. This method introduces a temporal feature Transformer architecture, fusing features from historical frames and the current frame with static weights, which improves tracking robustness in dynamic scenes to some extent. Its PCK metric reached 89.2% on the public dataset Human3.6M, and it became an important reference scheme in the field of real-time pose tracking.

[0004] However, the Alphapose method still has key technical shortcomings in practical applications, making it difficult to meet the stringent requirements of complex scenarios. First, this method relies on the traditional Alphapose self-attention mechanism for temporal feature modeling, which offers high accuracy but slow computation speed; conversely, fast computation speed but low accuracy. The secondary computational complexity of self-attention leads to an exponential increase in computational power consumption with input resolution. For example, when processing 1080p video streams commonly used in industrial scenarios, its single-frame inference latency reaches as high as 118ms, far exceeding the 30ms real-time threshold required for scenarios such as industrial robotic arm assembly and autonomous driving. Furthermore, its large memory footprint limits its deployment on edge computing devices. Second, its static weight fusion strategy lacks scene adaptability. When faced with non-rigid deformation of the target or brief occlusion, the fixed fusion ratio cannot dynamically adjust the feature weights. In scenarios where the target briefly disappears and then reappears, it is difficult to efficiently activate prior information about historical poses, easily leading to pose drift and tracking interruption. Its PCK index drops sharply to 62.5% in occluded scenarios. Finally, this method fails to address the inherent semantic space paradox of deep neural networks: while shallow features retain rich spatial details, they lack semantic abstraction capabilities and struggle to distinguish between targets and background interference; deep features possess strong semantic discrimination capabilities, but suffer spatial accuracy loss due to multiple downsampling, and the unidirectional information flow fusion method it employs cannot establish a bidirectional complementary mechanism between high and low layer features. When the target scale is small, the localization error of key structural points can reach 8-12 pixels, far exceeding the accuracy standard of less than 2 pixels required in industrial scenarios.

[0005] In summary, existing pose tracking methods, represented by Alphapose, suffer from difficulties in balancing accuracy and efficiency, poor adaptability to dynamic scenes, and an inability to simultaneously address semantic and spatial synchronization. Summary of the Invention

[0006] This invention addresses the problems of existing target pose tracking methods, such as difficulty in balancing accuracy and efficiency, poor adaptability to dynamic scenes, and inability to simultaneously consider semantic and spatial synchronization. It provides a target pose tracking method based on an enhanced Mamba attention network, aiming to achieve synergistic optimization of accuracy, efficiency, and robustness by introducing the Mamba State Space Model (SSM), multi-scale attention masking, and dynamic feature fusion mechanism.

[0007] The solution to the technical problem of this invention is:

[0008] A target pose tracking method based on an enhanced Mamba attention network includes the following steps:

[0009] Step 1: Feature extraction and preliminary aggregation. A deep layer aggregation (DLA) backbone network is used to extract multi-scale features. Progressive downsampling is completed through a 6-level resolution module. A tree-structured recursive aggregation framework is used to preserve shallow edge texture and deep semantic information. At the same time, the DLAUp module is used to realize semantic guidance from deep to shallow layers. The multi-scale feature weight allocation is optimized through iterative aggregation using the IDAUp module.

[0010] Step 2: Based on Mamba, temporal dependency modeling is performed by introducing Mamba SSM to construct a bidirectional adaptive Mamba architecture. Hierarchical adaptive Mamba dynamically adjusts the state dimension and convolutional kernel parameters according to the feature hierarchy, allocating higher dimensions to shallow features to preserve details and expanding the convolutional kernels for deeper features to capture global dependencies. Bidirectional Mamba (BiMamba) constructs a bidirectional mapping of spatiotemporal features through parallel forward and backward processing paths, fusing spatial neighborhood dependencies with temporal sequence associations.

[0011] Step 3: Feature optimization with enhanced attention. Based on DLA, a spatial-channel dual attention collaborative mechanism is introduced. The spatial attention module expands the receptive field through dilated convolution, calculates the importance weights of each position in the feature map, accurately locates key regions such as joints, and enhances feature reasoning ability in occluded scenes; the channel attention module adopts joint pooling (average pooling + max pooling) and a two-layer fully connected network to enhance the activation intensity of action semantic related channels.

[0012] Step 4: Four-branch multi-scale adaptive attention mask generation. A four-branch multi-scale attention mask generator is constructed, combining channel attention and spatially deformable attention mechanisms. Four-branch feature extraction captures multi-scale information from local details to global semantics through 1×1 convolution, 3×3 standard convolution, dilated convolution I, and dilated convolution II, respectively. Dynamic mask modulation dynamically adjusts the weights of each branch based on a target scale-aware gating mechanism (large targets enhance global branches, small targets enhance local branches), and optimizes mask strength by combining spatial consistency and confidence (such as joint heatmaps), guiding the model to focus on unoccluded key structures in occluded scenarios.

[0013] Step 5: Dynamic Feature Fusion and Enhancement. A dynamic fusion framework incorporating temporal memory buffers and bidirectional information flow is constructed to improve the stability of long-term tracking. Quality-aware dynamic fusion models the current frame features, previous frame features, and mask values ​​using a weight prediction network, outputting adaptive fusion weights—increasing the weight of previous frame features to maintain continuity during occlusion and increasing the weight of the current frame to capture instantaneous dynamics during rapid movement. Cross-layer feature enhancement maps high-level semantic features to low-level feature dimensions using a channel adapter and combines cross-attention to calculate cross-layer semantic associations, achieving collaborative optimization of "semantic guidance-spatial localization." Bidirectional information flow establishes bidirectional information exchange channels between adjacent layers; high-level features provide semantic guidance to low-level features, while low-level features supplement spatial details for high-level features, forming a closed-loop information flow. The memory enhancement mechanism selectively integrates historical features through a gating network, constructing an implicit temporal window.

[0014] Step 6: Attitude Output and Evaluation. Input the optimized features from the above steps into the attitude prediction head and output the target's 2D / 3D attitude parameters; use PCK, ADD, median@pix, and median@mm metrics to evaluate tracking accuracy.

[0015] The beneficial effects of this invention are as follows:

[0016] This invention proposes a target pose tracking method based on an enhanced Mamba attention network. Through deep collaboration of multiple technological innovations, including feature extraction, temporal modeling, feature fusion, and adaptation to complex scenes, it specifically addresses three core pain points commonly found in existing target pose tracking methods: difficulty in balancing accuracy and efficiency, poor adaptability to dynamic and complex scenes, and low synchronization of semantic space. This provides a better solution for pose tracking needs in industrial and other practical scenarios.

[0017] In the feature extraction stage, this invention innovatively employs a technical solution combining a Deep Layer Aggregation (DLA) network with an attention mechanism. This solution, on the one hand, can accurately focus on key target region information through the attention mechanism, effectively weakening the influence of irrelevant background interference factors, thus improving the purity and quality of feature data from the source and providing high-quality input for subsequent pose modeling. On the other hand, leveraging the inter-layer aggregation capability of the DLA network, feature information from different levels can be efficiently integrated, reducing redundancy caused by information dispersion in traditional feature extraction processes, significantly reducing the computational pressure of subsequent processing stages, and laying the foundation for overall efficiency improvement.

[0018] In the temporal modeling stage, to overcome the limitations of traditional Transformer architecture in terms of high computational cost and low efficiency when capturing long temporal dependencies, this invention adopts a bidirectional adaptive Mamba architecture to replace the traditional approach. This architecture, relying on an efficient state-space modeling mechanism, can not only accurately capture the long- and short-term temporal dependencies of the target pose in the time dimension and fully reconstruct the dynamic process of pose changes, but also significantly reduce the high computational cost caused by the self-attention mechanism in traditional Transformers by adaptively adjusting model parameters. While ensuring the accuracy of simultaneous processing of spatial and semantic information, it significantly improves the efficiency of temporal modeling, achieving a synergistic optimization of accuracy and efficiency.

[0019] In the feature fusion stage, this invention abandons the traditional extensive strategy of "full fusion" and instead designs a targeted dynamic fusion mechanism. This mechanism can focus on fusing only key features that affect attitude tracking accuracy by evaluating the importance of features in real time, effectively avoiding ineffective fusion operations on redundant features and concentrating limited computing resources on critical steps. This precise fusion method further reduces the computation time of the feature fusion stage without sacrificing attitude tracking accuracy, making the method more adaptable to resource-constrained scenarios such as edge devices and expanding the application scope of the technology.

[0020] To address the insufficient robustness of traditional methods in complex dynamic scenarios such as target occlusion and scale changes, this invention proposes a multi-technology collaborative solution. On one hand, multi-scale attention masking technology dynamically adjusts the attention range based on real-time changes in the target scale, ensuring accurate focus on the target region even when the target scale is scaled. On the other hand, a memory enhancement mechanism is introduced to temporarily store key pose information before target occlusion, allowing for rapid retrieval of historical information after occlusion is resolved, aiding pose recovery and preventing tracking interruptions or pose drift caused by occlusion, thus significantly improving tracking stability in complex scenarios.

[0021] This method not only meets the high-precision and high-efficiency requirements of attitude tracking in complex real-world scenarios such as industrial production and intelligent monitoring, but also has stronger environmental adaptability and practical value, providing strong support for the industrial application of target attitude tracking technology. Attached Figure Description

[0022] Figure 1 This is an overall flowchart of a target pose tracking method based on an enhanced Mamba attention network according to the present invention.

[0023] Figure 2 This is a schematic diagram of the hierarchical adaptive Mamba and bidirectional information flow module structure of the present invention;

[0024] Figure 3This is a schematic diagram of the multi-scale attention mask generation and dynamic modulation of the present invention. Detailed Implementation

[0025] The present invention will now be described in further detail with reference to the accompanying drawings.

[0026] like Figure 1 As shown, a target pose tracking method based on an enhanced Mamba attention network includes the following steps:

[0027] Step 1: Feature Extraction and Preliminary Aggregation; Multi-scale features are extracted using a Deep Layer Aggregation (DLA) backbone network. Progressive downsampling is achieved through a 6-level resolution module. A tree-structured recursive aggregation framework is used to preserve shallow edge textures and deep semantic information. At the same time, the DLAUp module is used to achieve semantic guidance from deep to shallow layers. The IDAUp module is used to iteratively aggregate and optimize the multi-scale feature weight allocation, thereby solving the information loss problem of traditional feature pyramids.

[0028] A DLA backbone network (such as DLA-34) pre-trained on the ImageNet dataset was selected as the basic feature extraction framework, with the input image resolution fixed at 480×480 pixels. This network contains six resolution modules, each achieving a 2x downsampling through three convolutional operations. The output feature map resolutions are 1 / 4, 1 / 8, 1 / 16, 1 / 32, 1 / 64, and 1 / 128 of the input image, corresponding to 16, 32, 64, 128, 256, and 512 channels, respectively.

[0029] The tree-structured recursive aggregation framework is implemented through residual connections: for the output of three convolutions of each resolution module, the number of channels is first unified through a 1×1 convolution, and then recursively aggregated through addition operations, preserving shallow details and semantic information such as edges and textures. The DLAUp module uses bilinear interpolation to upsample deep features to the corresponding shallow feature size, and then concatenates them with shallow features through a concatenation operation (e.g., upsampling 1 / 64 resolution features to 1 / 32 and then concatenating them with native 1 / 32 features); the IDAUp module calculates cross-scale feature weights through a 3-layer convolutional network, with the initial weight set to 0.5, and then dynamically adjusted according to the feature entropy (the weight of features with higher entropy values ​​increases to 0.7, and the weight of features with lower entropy values ​​decreases to 0.3), and finally completes multi-scale feature aggregation through weighted summation.

[0030] Step 2: Mamba-based time-series dependency model; such as Figure 2As shown, there are hierarchical adaptive Mamba and bidirectional Mamba (BiMamba): Hierarchical adaptive Mamba dynamically adjusts the state dimension and convolution kernel parameters according to the feature level, allocating higher dimensions to shallow features to preserve details and expanding the convolution kernel to capture global dependencies for deep features; Bidirectional Mamba (BiMamba) constructs a bidirectional mapping of spatiotemporal features through forward and backward parallel paths, integrating spatial neighborhood dependencies and temporal correlations, breaking through the quadratic complexity bottleneck of Transformer while maintaining linear complexity (O(L)).

[0031] A bidirectional adaptive architecture is constructed using the Mamba state-space model, with pre-trained Mamba-2.8B parameters loaded during model initialization. The feature hierarchy is divided into 0-5 levels (0 for shallow layers, 5 for deep layers), and the parameters are dynamically adjusted according to the following rules:

[0032] State dimension: set according to the formula d_state(i)=32×2^min(i,2), with 32, 64 and 128 for levels 0-2 respectively, and 128 for levels 3-5 (shallow layers need higher dimensions to preserve edge details, while deep semantic feature dimensions do not need to be increased further);

[0033] Kernel size: Set according to d_conv(i)=min(4,2+i), 2×2 for levels 0-1, 3×3 for levels 2-3, and 4×4 for levels 4-5 (deep features require larger kernels to capture global dependencies);

[0034] Nonlinear transformation intensity: expand(i) = 2 + i, which is achieved by adjusting the channel expansion factor before the activation function (the higher the level, the stronger the feature transformation capability).

[0035] The bidirectional processing path employs parallel computation logic: the forward path receives a chronologically ordered sequence of frames (x1 to x_t), and the backward path receives a reverse-order sequence of frames (x_t to x1). The output features of the two paths are concatenated along the channel dimension (doubling the feature dimension), then compressed back to the original dimension via a 1×1 convolution and activated by a sigmoid function (weights ranging from 0 to 1), achieving bidirectional fusion of spatiotemporal features. This architecture reduces the temporal modeling complexity from O(L) of the Transformer. 2 The inference speed is reduced to O(L), resulting in a more than 3x speedup when processing 1080p video at 30 frames per second.

[0036] Step 3: Feature optimization for attention enhancement; such as... Figure 3As shown, a spatial-channel dual attention module is added on top of DLA; the spatial attention module expands the receptive field through dilated convolution, calculates the importance weight of each position in the feature map, accurately locates key regions such as joints, and enhances the feature reasoning ability in occluded scenes; the channel attention module adopts joint pooling (average pooling + max pooling) and two fully connected networks to enhance the activation intensity of action semantic related channels and suppress feature interference from irrelevant channels.

[0037] After DLA feature aggregation, a spatial-channel dual attention module is added, as implemented below:

[0038] The spatial attention module includes a dilated convolution unit, a global average pooling unit, and a sigmoid activation unit. The dilated convolution unit is configured with a 3×3 dilated convolution with a dilation rate of 2 (hereinafter referred to as "dilated convolution one"). Through discontinuous sampling, it expands the receptive field from 3×3 to 7×7 and convolves the pre-processed feature map to capture key target structures (such as robotic arm joints and workpiece edges) and neighborhood spatial correlation information. The global average pooling unit compresses the multi-channel feature map output by the dilated convolution into a single-channel feature map, realizing spatial information aggregation and dimensional simplification, laying the foundation for weight generation. The sigmoid activation unit receives the single-channel feature map and generates a spatial weight map with the same size as the input feature map through the sigmoid function. The weight of the target key region in this weight map is 2-3 times that of the background region. In the case of partial occlusion (such as the robotic arm being occluded by the workpiece), the weight of the unoccluded region is automatically increased to strengthen key features, suppress noise, guide the model to focus on the effective region, and ensure continuous and accurate posture tracking.

[0039] The access channel attention module includes a global average pooling unit, a global max pooling unit, two fully connected network layers, and a sigmoid activation component. The global average pooling unit extracts overall information from the feature map, while the global max pooling unit highlights local salient features. The two fully connected network layers respectively achieve 1 / 4 channel compression and restoration of the original channel count. The sigmoid activation component generates channel weights. This module first performs two types of pooling in parallel on the feature map. After the concatenated result is filtered and optimized by the fully connected network, channel weights are generated using the sigmoid function. The weights of action-related channels such as "robotic arm rotation" and "end-effector grasping" are increased by more than 40%, while the weights of irrelevant channels such as background textures are reduced to below 0.3, thereby strengthening key semantic channels and suppressing interference.

[0040] The output features of the dual attention module are fused through element-wise multiplication, which strengthens key spatial locations and highlights important semantic channels.

[0041] Step 4: Four-branch multi-scale adaptive attention mask generation; includes a four-branch feature extraction structure and a dynamic mask modulation module. The four-branch feature extraction structure captures multi-scale information from local details to global semantics through four different convolution methods: 1×1 convolution, 3×3 standard convolution, 3×3 dilated convolution I (dilation=2, receptive field 7×7), and dilated convolution II (dilation=4, receptive field 17×17). The dynamic mask modulation dynamically adjusts the weights of each branch based on a target scale-aware gating mechanism (large targets enhance global branches, small targets enhance local branches), and optimizes the mask strength by combining spatial consistency and confidence (such as joint heatmaps). In occluded scenarios, it guides the model to focus on unoccluded key structures (such as the edge of a robotic arm's end effector), ensuring that the mask selectively targets effective features.

[0042] The four-branch, multi-scale adaptive attention mask generation uses a parallel convolutional structure, with each branch configured as follows:

[0043] Branch 1 (pixel-level details): 1×1 convolution (without padding), outputs a feature map with stride=1, capturing local information such as object edges and textures;

[0044] Branch 2 (Mesoscale Structure): 3×3 standard convolution (padding=1), receptive field 3×3, extracting component structure (such as the joint connection of the robotic arm);

[0045] Branch 3 (Mid-to-long-range association): 3×3 convolution 1 (dilation=2, padding=2), receptive field 7×7, capturing spatial relationships between parts (such as the positional association of "arm-end gripper");

[0046] Branch 4 (Global Semantics): 3×3 convolution II (dilation=4, padding=4), receptive field 17×17, to obtain scene context (such as the relative position of "robotic arm-workbench").

[0047] The outputs of each branch are concatenated to a 64-channel output via 1×1 convolution and then fused into a single-channel mask (value range 0-1) via 3×3 convolution. The dynamic modulation logic is implemented using the bounding box area S_obj obtained from the object detection module: when S_obj is greater than the training set mean μ (e.g., 3000 pixels²), the weight of branch 4 is increased to 0.6 (other branches collectively account for 0.4); when S_obj ≤ μ, the sum of the weights of branches 1 and 2 is increased to 0.7. Simultaneously, combined with the confidence level of the keypoint heatmap (pre-output by the pose prediction head), the mask value for occlusion regions with a confidence level < 0.5 is reduced by 30%-50%, ensuring the mask focuses on reliable features.

[0048] Step 5: Dynamic Feature Fusion and Enhancement; Construct a dynamic fusion framework including temporal memory buffer and bidirectional information flow to improve the stability of long-term tracking. Quality-aware dynamic fusion models the features of the current frame, the previous frame, and the mask value through a weight prediction network, outputting adaptive fusion weights. When occluded, the weights of the previous frame features are increased to maintain continuity, and the weights of the current frame are increased to capture instantaneous dynamics during rapid motion. Cross-layer feature enhancement maps high-level semantic features to low-level feature dimensions through a channel adapter and combines cross-attention to calculate cross-layer semantic associations, achieving collaborative optimization of "semantic guidance-spatial localization". Bidirectional information flow establishes a bidirectional information exchange channel between adjacent layers. High-level features provide semantic guidance to low-level features, and low-level features supplement spatial details for high-level features, forming a closed-loop information flow. The memory enhancement mechanism selectively integrates historical features through a gating network, constructs an implicit temporal window, enhances the modeling ability of complex motion trajectories, and suppresses pose drift in long-term tracking.

[0049] The dynamic fusion framework consists of four sub-modules, implemented as follows:

[0050] Quality-aware dynamic fusion: The weighted prediction network consists of three 3×3 convolutional layers (channel count 64→32→2) and a softmax layer. Inputs include current frame features, previous frame features, and the mask mean. When the mask mean is <0.3 (indicating occlusion), the output weights for the previous frame features are w2 = 0.6-0.8; when the optical flow vector is >5 pixels / frame (indicating fast motion), the weights for the current frame features are w1 = 0.7-0.9; under normal scene conditions, w1 = 0.6 and w2 = 0.4. The fused features are generated by multiplying w1 by the current frame and w2 by the previous frame.

[0051] Cross-layer feature enhancement: High-level features (such as 1 / 64 resolution) are mapped to the channel dimension of low-level features (such as 1 / 4 resolution) through 1×1 convolution. The semantic association between the two is calculated through cross attention (using ScaledDot-ProductAttention). High-level semantic features with an association weight > 0.5 are injected into the low layer with a weight of 0.4 to achieve "semantic-guided spatial localization".

[0052] Two-way information flow: Two 1×1 convolutional channels are established between adjacent layers (such as 1 / 4 and 1 / 8 resolution): the high-level to low-level channel transmits semantic labels (such as "joint type"), and the low-level to high-level channel transmits spatial coordinates (such as "joint position offset"). The bidirectional flow of features between layers improves the localization accuracy by 15%.

[0053] Memory enhancement mechanism: The memory buffer stores the features of the most recent 5 frames. The gating network selects effective historical features by using the cosine similarity between the current frame and historical frames (a correlation is determined when the similarity is greater than 0.6). These features are then fused with the current frame features with a weight of 0.3:0.7 to suppress the cumulative error in long-term tracking (such as the drift of the robotic arm's motion trajectory).

[0054] Step 6: Attitude Output and Evaluation; Input the optimized features from the above steps into the attitude prediction head (such as heatmap regression + offset prediction), and output the target's 2D / 3D attitude parameters; Use indicators such as PCK (correct key point ratio), ADD (average displacement distance), median@pix (pixel-level median error), and median@mm (millimeter-level median error) to evaluate the tracking accuracy.

[0055] The pose prediction head employs a dual-branch structure of "heatmap regression + offset prediction": the heatmap branch outputs a keypoint probability heatmap (size and input features) through 3×3 convolution and sigmoid. Figure 1 The offset branch outputs sub-pixel offsets in the x / y directions. Peak points are extracted from the heatmap using non-maximum suppression (NMS), and the final joint coordinates are calculated by combining the offsets. These coordinates are then stitched together to form 2D / 3D pose parameters (3D parameters require additional camera intrinsics).

[0056] The evaluation phase employs a multi-metric comprehensive assessment: the PCK metric calculates the proportion where the distance between the predicted joint and the true value is less than a preset threshold (e.g., 0.1 × target diagonal length at the pixel level); the ADD metric evaluates the average displacement error of the 3D pose; and median@pix and median@mm are used to calculate the median of pixel-level and millimeter-level errors, respectively. On the industrial robotic arm dataset, all metrics outperform existing methods.

Claims

1. A target pose tracking method based on an enhanced Mamba attention network, characterized in that, The method includes the following steps: Step 1: Feature extraction and preliminary aggregation. Multi-scale features are extracted using the DLA backbone network. Progressive downsampling is completed through a 6-level resolution module. The tree-structured recursive aggregation framework is used to preserve shallow edge texture and deep semantic information. At the same time, the DLAUp module is used to achieve semantic guidance from deep to shallow layers. The multi-scale feature weight allocation is optimized through iterative aggregation using the IDAUp module. Step 2: Based on Mamba temporal dependency modeling, introduce Mamba SSM to construct a bidirectional adaptive Mamba architecture; wherein, hierarchical adaptive Mamba dynamically adjusts the state dimension and convolution kernel parameters according to the feature level, assigning higher dimensions to shallow features to preserve details, and expanding the convolution kernel to capture global dependencies for deep features; BiMamba constructs a bidirectional mapping of spatiotemporal features through forward and backward parallel processing paths, and integrates spatial neighborhood dependencies with temporal sequence associations; Step 3: Feature optimization with enhanced attention. Based on DLA, a spatial-channel dual attention module is introduced. The spatial attention module expands the receptive field through dilated convolution, calculates the importance weights of each position in the feature map, accurately locates key regions such as joints, and enhances the feature reasoning ability in occluded scenes. The channel attention module adopts joint pooling, namely average pooling + max pooling, and two fully connected layers to enhance the activation intensity of action semantic related channels. Step 4: Four-branch multi-scale adaptive attention mask generation. A four-branch multi-scale attention mask generator is constructed, combining channel attention and spatially deformable attention mechanisms. Four-branch feature extraction captures multi-scale information from local details to global semantics through 1×1 convolution, 3×3 standard convolution, dilated convolution I, and dilated convolution II, respectively. Dynamic mask modulation dynamically adjusts the weights of each branch based on a target scale-aware gating mechanism, and optimizes the mask strength by combining spatial consistency and confidence, guiding the model to focus on unoccluded key structures in occluded scenarios. Step 5: Dynamic Feature Fusion and Enhancement. A dynamic fusion framework incorporating temporal memory buffer and bidirectional information flow is constructed to improve the stability of long-term tracking. Quality-aware dynamic fusion models the current frame features, previous frame features, and mask values ​​using a weight prediction network, outputting adaptive fusion weights. Cross-layer feature enhancement maps high-level semantic features to low-level feature dimensions using a channel adapter and combines cross-attention to calculate cross-layer semantic associations. Bidirectional information flow establishes bidirectional information exchange channels between adjacent layers, with high-level features providing semantic guidance to low-level features and low-level features supplementing spatial details for high-level features, forming a closed-loop information flow. The memory enhancement mechanism selectively integrates historical features through a gating network to construct an implicit temporal window. Step 6: Attitude Output and Evaluation. Input the optimized features from the above steps into the attitude prediction head and output the target's 2D / 3D attitude parameters; use PCK, ADD, median@pix, and median@mm metrics to evaluate tracking accuracy.

2. The target pose tracking method based on an enhanced Mamba attention network according to claim 1, characterized in that, Step 2 specifically involves: A bidirectional adaptive architecture is constructed using the Mamba state-space model. Pre-trained Mamba-2.8B parameters are loaded during model initialization. The feature layers are divided into 0-5 levels, where 0 represents a shallow layer and 5 represents a deep layer. The parameters are dynamically adjusted according to the following rules: State dimension: set according to the formula d_state(i)=32×2^min(i,2), with 32, 64 and 128 for levels 0-2 respectively, and 128 for levels 3-5. Shallow layers need higher dimensions to preserve edge details, while deep semantic feature dimensions do not need to be increased further. Kernel size: Set according to d_conv(i)=min(4,2+i), 2×2 for levels 0-1, 3×3 for levels 2-3, and 4×4 for levels 4-5. Deep features require larger kernels to capture global dependencies. Nonlinear transformation strength: expand(i) = 2 + i, which is achieved by adjusting the channel expansion factor before the activation function. The higher the level, the stronger the feature transformation capability. The bidirectional processing path employs parallel computing logic: the forward path receives a chronologically ordered frame sequence x1 to x_t, and the backward path receives a reverse-order frame sequence x_t to x1. The output features of the two paths are concatenated along the channel dimension, doubling the feature dimension, then compressed back to the original dimension via a 1×1 convolution and activated by a sigmoid function with weights ranging from 0 to 1, achieving bidirectional fusion of spatiotemporal features. This architecture reduces the temporal modeling complexity from O(L) of the Transformer. 2 The inference speed is reduced to O(L), resulting in a more than 3x speedup when processing 1080p video at 30 frames per second.

3. The target pose tracking method based on an enhanced Mamba attention network according to claim 1, characterized in that, In step 3, after the DLA aggregates features, a spatial-channel dual attention module is added, which is implemented as follows: The spatial attention module includes a dilated convolution unit, a global average pooling unit, and a sigmoid activation unit. The dilated convolution unit is configured with a 3×3 dilated convolution with a dilation rate of 2. Through discontinuous sampling, it expands the receptive field from 3×3 to 7×7 and convolves the pre-processed feature map to capture the key structure of the target and the spatial relationship information of the neighborhood. The global average pooling unit compresses the multi-channel feature map output by the dilated convolution into a single-channel feature map, realizing spatial information aggregation and dimensional simplification, laying the foundation for weight generation. The sigmoid activation unit receives the single-channel feature map and generates a spatial weight map with the same size as the input feature map through the sigmoid function. In this weight map, the weight of the key target region is 2-3 times that of the background region, and the weight of the unoccluded region is automatically increased in the case of partial occlusion, thereby strengthening the key features, suppressing noise, guiding the model to focus on the effective region, and ensuring continuous and accurate pose tracking. The access channel attention module includes a global average pooling unit, a global max pooling unit, two fully connected network layers, and a sigmoid activation component. The global average pooling unit extracts overall information from the feature map, the global max pooling unit highlights local salient features, the two fully connected network layers respectively achieve 1 / 4 channel compression and restoration of the original number of channels, and the sigmoid activation component is used to generate channel weights. This module first performs two types of pooling in parallel on the feature map. After the concatenation result is filtered and optimized by the fully connected network, channel weights are generated by sigmoid. The weights of channels related to key target structures and actions are increased by more than 40%, while the weights of irrelevant channels such as background textures are reduced to below 0.3, thereby strengthening key semantic channels and suppressing interference.

4. The target pose tracking method based on an enhanced Mamba attention network according to claim 1, characterized in that, Step 4, the four-branch multi-scale adaptive attention mask generation, employs a parallel convolutional structure, with each branch configured as follows: Branch 1: 1×1 convolution, outputting a feature map with stride=1, capturing local information such as object edges and textures; Branch 2: 3×3 standard convolution, receptive field 3×3, extracting component structure; Branch 3: 3×3 convolution 1, receptive field 7×7, capturing the spatial relationships between components; Branch 4: 3×3 convolution 2, receptive field 17×17, to obtain scene context; The outputs of each branch are concatenated to a 64-channel output through a 1×1 convolution and then fused into a single-channel mask through a 3×3 convolution, with values ​​ranging from 0 to 1. The dynamic modulation logic is implemented using the bounding box area S_obj obtained from the target detection module: when S_obj is greater than the training set mean μ, the weight of branch 4 is increased to 0.6, and the weights of other branches are increased to 0.4; when S_obj ≤ μ, the sum of the weights of branches 1 and 2 is increased to 0.7; at the same time, combined with the confidence of the key point heatmap, the mask value of the occluded region with a confidence of <0.5 is reduced by 30%-50%, ensuring that the mask focuses on reliable features.

5. The target pose tracking method based on an enhanced Mamba attention network according to claim 1, characterized in that, The dynamic fusion framework in step 5 comprises four sub-modules, which are implemented as follows: Quality-aware dynamic fusion: The weight prediction network consists of 3 layers of 3×3 convolutions and Softmax, and takes the current frame features, the previous frame features, and the mask mean as input. When the mean mask value is less than 0.3, it is determined to be occlusion, and the feature weight of the previous frame is output as w2 = 0.6-0.8; when the optical flow vector is greater than 5 pixels / frame, it is determined to be fast motion, and the feature weight of the current frame is w1 = 0.7-0.9; under normal scene conditions, w1 = 0.6 and w2 = 0.

4. The fusion feature is generated by w1×current frame + w2×previous frame; Cross-layer feature enhancement: High-level features are mapped to the channel dimension of low-level features through 1×1 convolution, and the semantic association between the two is calculated through cross attention. High-level semantic features with an association weight > 0.5 are injected into the low layer with a weight of 0.

4. Two-way information flow: Two 1×1 convolutional channels are established between adjacent layers: the high-level to low-level channel transmits semantic labels, and the low-level to high-level channel transmits spatial coordinates. The bidirectional flow of features between layers improves the positioning accuracy by 15%. Memory enhancement mechanism: The memory buffer stores the features of the most recent 5 frames. The gating network determines that the current frame and the historical frame are related when the cosine similarity is greater than 0.

6. Valid historical features are selected and fused with the current frame features with a weight of 0.3:0.7 to suppress the cumulative error in long-term tracking.

Images