Three-dimensional human body posture estimation method fusing anatomical structure prior and state space modeling

Abstract

The invention discloses a three-dimensional human body posture estimation method fusing anatomical structure prior and state space modeling, and belongs to the technical field of computer vision. According to the method, a 2D human body posture key point sequence is obtained and is embedded into a high-dimensional feature space through a linear projection layer to obtain joint feature representation; the method comprises the following steps: constructing a backbone network containing a plurality of TriadFormer modules, wherein each TriadFormer module is integrated with a graph convolutional network GCN, a hierarchical part internal order state space model HIP-SSM and a limb pilot asymmetric local-global converter LimbFormer; capturing local spatial dependence through GCN, injecting anatomical structure prior by HIP-SSM and modeling chain kinematics dependence, and strengthening intra-limb collaboration and cross-limb global interaction through LimbFormer; and finally, a three-dimensional human body posture estimation result is obtained through mapping of a linear projection output head. According to the method, through multi-module collaborative modeling, the human anatomical structure and kinematics constraints are deeply fused, the precision and robustness of attitude estimation in a complex scene are improved, the calculation efficiency is considered, and the method is suitable for multiple fields such as human-computer interaction and motion analysis.

Description

Technical Field

[0001] This invention relates to the field of technology, specifically to a three-dimensional human pose estimation method that integrates anatomical prior knowledge with state space modeling. Background Technology

[0002] As 3D human pose estimation is one of the core research directions in the field of computer vision, its goal is to reconstruct the coordinate positions of human joints in 3D space from monocular images, videos, or 2D keypoint sequences. It is widely used in various scenarios such as human-computer interaction, motion analysis, rehabilitation training, and virtual simulation. With the development of deep learning technology, the accuracy of 2D human pose estimation has been significantly improved. However, the mapping from 2D information to 3D pose is essentially an ill-posed problem, with many challenges such as depth ambiguity, self-occlusion interference, and insufficient scene generalization ability. Especially in complex actions or non-laboratory environments, existing methods often struggle to balance estimation accuracy and robustness.

[0003] Current mainstream 3D human pose estimation methods mostly adopt the 2D-to-3DLifting framework, which first obtains the coordinates of human keypoints through a 2D pose estimation model, and then maps the low-dimensional 2D coordinates to 3D space through a neural network. Early methods mostly relied on fully connected networks or simple residual structures, which, although achieving end-to-end prediction, struggled to capture spatial constraints and long-range dependencies between joints. To address this issue, researchers began to introduce a fusion architecture of Graph Convolutional Networks (GCNs) and Transformer models. By leveraging the natural graph structure modeling ability of GCNs on the human skeleton to enhance local spatial features, and utilizing the self-attention mechanism of Transformers to capture global correlations, the accuracy of 3D pose estimation has been improved.

[0004] Chinese patent (publication number: CN118212689A) discloses a 3D human pose estimation method based on feature fusion of GCN and Transformer. This method extracts spatiotemporal features through multi-level feature layers, captures global pose features using spatial and temporal Transformers respectively, and extracts local pose features by combining GCN blocks and multi-scale Transformers. Finally, it regresses 3D key points by fusing features from all levels through a hierarchical feature fusion layer. This patent effectively makes up for the shortcomings of traditional Transformer in extracting spatial features of human pose and improves the completeness of feature representation. However, it still has significant shortcomings: it does not consider the inherent topological priors of human anatomy and does not adequately model the chain kinematic dependencies between joints, which can easily lead to joint position shifts and poor pose continuity in scenarios with rapid movement or large limb swings.

[0005] To address the structural modeling deficiencies in patent CN118212689A, Chinese patent (publication number: CN111353381B) proposes a 3D human pose estimation method for 2D images. This method strengthens the correlation between local joint features and global pose features by introducing a multi-scale feature extraction and fusion strategy, while also optimizing the model's adaptability to occluded scenes. While this patent improves the stability of pose estimation in complex scenes to some extent, it does not fundamentally solve the core problem: its use of a generalized feature fusion method fails to deeply embed the hierarchical relationships of human anatomical structures and kinematic constraints into the model architecture. This results in limited structural perception of human pose, and when handling complex movements, it still suffers from large 3D coordinate prediction errors and inaccurate pose reconstruction.

[0006] Therefore, there is an urgent need for a three-dimensional human pose estimation technology that deeply integrates prior knowledge of human anatomical structure with efficient feature modeling mechanisms to achieve synergistic optimization of local structural constraints, long-range kinematic chain modeling, and global interaction, thereby fundamentally improving the model's ability to represent complex poses and its estimation accuracy. Summary of the Invention

[0007] Based on the aforementioned technical problems, this application discloses a three-dimensional human pose estimation method that integrates anatomical structure priors and state space modeling, specifically including:

[0008] A 2D human pose key point sequence is obtained, and the 2D human pose key point sequence is embedded into a high-dimensional feature space through a linear projection layer to obtain joint feature representation;

[0009] The backbone network is constructed, including multiple TriadFormer modules. The TriadFormer modules include a graph convolutional network (GCN) module, a hierarchical part-order state space model (HIP-SSM), and a limb-leading asymmetric local-global transformer (LimbFormer).

[0010] The joint feature representation is input into the graph convolutional network (GCN) module to capture local spatial dependencies and limb constraints between joints and output local structural enhancement features.

[0011] The local structural enhancement features are input into the hierarchical intra-part order state space model HIP-SSM, anatomical priors are injected and chain kinematic dependencies are modeled, and structural awareness features are output.

[0012] The structurally aware features are input into the limb-leading asymmetric local-to-global transformer LimbFormer to enhance intralimb coordination and cross-limb global interaction, and output enhanced joint features.

[0013] The enhanced joint features are mapped to a three-dimensional coordinate space by a linear projection output head to obtain a three-dimensional human pose estimation result.

[0014] Preferably, the joint feature representation embeds the 2D human pose keypoint sequence into a high-dimensional feature space through a linear projection layer; the shape of the 2D human pose keypoint sequence is N×2, where N is the number of joints and N≥17, covering the key joints of the human torso, left and right upper limbs and left and right lower limbs; the linear projection layer contains a learnable weight matrix and bias terms, and maps the 2D coordinate vector from 2D to a D-dimensional high-dimensional feature space through matrix multiplication to obtain a joint feature representation of shape N×D, where D is the model hidden dimension and D≥160. The projection process also introduces learnable parameters to achieve an effective conversion from low-dimensional coordinate information to high-dimensional semantic features.

[0015] Preferably, the backbone network is composed of multiple TriadFormer modules stacked in a residual manner. Each TriadFormer module includes a graph convolutional network (GCN) module, a hierarchical intraorder state space model (HIP-SSM), and a limb-leading asymmetric local-global transformer (LimbFormer). Adjacent TriadFormer modules transfer features through residual connections. That is, the output features of the previous module are added to the output features of the current module, normalized by the LayerNorm layer, and then input to the next module. Through multi-layer stacking, the structure-aware joint features are gradually extracted.

[0016] Preferably, the Graph Convolutional Network (GCN) module is based on the natural graph structure of the human skeleton, with joints as nodes and limb connections as edges, constructing a fixed adjacency matrix. The module's graph convolution operation achieves local information fusion by aggregating the first-order neighborhood joint features of each joint. The specific calculation process is as follows: ,in For the first Output characteristics of each joint For the first The set of neighboring joints of a joint. The adjacency matrix is ​​the first... Line number The elements of the column represent joints. and The connection strength, The graph convolution weight matrix, This is a bias term; it efficiently captures the geometric relationships and limb constraints between adjacent joints through operations, enhancing the model's ability to perceive local topological structures.

[0017] Preferably, the hierarchical intra-partial state space model (HIP-SSM) includes feature preprocessing, topological prior injection, state update, and feature output, specifically:

[0018] The local structural enhancement features of the input are normalized by using the LayerNorm layer to eliminate interference caused by differences in feature scale.

[0019] Two types of learnable position codes are attached to each joint: location index and positional code. and depth index within the part The enhanced feature representation is obtained as follows: ,in The matrix is ​​embedded for different body parts, corresponding to five functional symmetrical units: torso, left / right arm, left / right leg, etc. For the depth embedding matrix within the region, This represents the maximum number of joints in a single body part, distinguishing the hierarchical positional relationships of different joints within that body part.

[0020] By outputting structure-aware features through state update and observation equations, the features are ensured to retain the topological information of the human skeleton.

[0021] Preferably, the state update equation of the hierarchical intra-part order state space model (HIP-SSM) is: ,in For the first The hidden state vector of each joint contains kinematic feature information of the joint. For the first Hidden predecessor states of each joint sequence The state transition matrix is ​​shared globally. For the first The input transformation matrix is ​​specific to each joint; it is output through the observation equation, with the following formula: ,in To integrate the structure-aware hidden states after topological priors. For the first The observation matrix of each joint, The output is the structure-aware feature.

[0022] Preferably, the limb-leading asymmetric local-to-global transformer (LimbFormer) includes a local modeling submodule, a cross-limb interaction submodule, and a global attention submodule. The local modeling submodule divides the 17 joints into five anatomical functional regions: spine, left upper limb, right upper limb, left lower limb, and right lower limb. Each region independently performs a local multi-head self-attention (PartMHSA) operation, with the following formula: ,in Parts The query, key, and value vectors obtained by linear mapping of internal joint features. , For the input feature dimension, To determine the number of attention heads, the local attention output is processed through residual connections and LayerNorm layers to obtain local enhancement features that strengthen intralimb synergy. .

[0023] Preferably, the cross-limb interaction submodule achieves cross-part information interaction through three steps: feature aggregation, graph convolution update, and feature broadcasting; and enhances local features for each part. Average pooling is performed to generate limb-level node features, using the following formula: ,in For the first The set of joints contained in each part. The number of joints in this region is given; subsequently, a five-node semantic graph and adjacency matrix are constructed. Satisfying diagonal self-connection and bidirectional connection between the spine node and four limb nodes, a single-layer graph convolution is performed after normalization: ,in The normalized adjacency matrix, For graph convolution, the weight matrix can be learned. As the initial limb-level node matrix, high-order dependency propagation between the spine and limbs is achieved through graph convolution; the updated limb-level node features are then used. The data is broadcast to each joint in the corresponding area and fused with the local enhancement features of that joint to obtain intermediate features with cross-limb perception capabilities.

[0024] Preferably, the global attention submodule employs an asymmetric attention mechanism to construct global associations; and transforms the original structural perception features through a linear mapping layer to generate a global query vector. , ,in For batch size, For the number of joints, To query the mapping weight matrix, ensure that the Query retains the original joint-level feature information; the Key and Value vectors are generated by linearly mapping the intermediate features output by the cross-limb interaction submodule, so that the key-value pair representation naturally integrates local limb collaboration patterns and cross-limb topological information; perform global self-attention calculation, the formula is: The output results are processed by residual connections and LayerNorm layers to obtain the enhanced joint features of the target.

[0025] Preferably, the linear projection output head consists of two fully connected network layers and an activation function. The first fully connected network layer maps the enhanced joint features from D-dimensional to 160-dimensional, introducing a non-linear transformation through the ReLU activation function to enhance feature representation. The second fully connected network layer maps the 160-dimensional features to 3-dimensional, corresponding to the x, y, and z coordinates in three-dimensional space. The loss function of the output head adopts the mean squared error loss (MSE), which optimizes the network parameters by minimizing the squared error between the predicted 3D coordinates and the true coordinates, and outputs a 3D human pose estimation result P with shape N×3, where each row vector corresponds to the 3D spatial coordinates of a joint.

[0026] Compared with the prior art, the technical solution of this application has the following technical effects:

[0027] This invention injects prior knowledge of human anatomy into the HIP-SSM module and adds dual encoding of location and depth within each joint, so that the evolution process of the state space model naturally follows biomechanical constraints, alleviating the problem of topological structure destruction caused by joint serialization. While maintaining linear computational complexity, it improves the structural consistency and stability of feature representation, making long-range kinematic chain modeling more in line with the real human movement law and avoiding the accumulation of errors along the joint link.

[0028] The asymmetric local-global attention mechanism of the LimbFormer module in this invention enhances intralimb collaboration and cross-limb interaction. By refining features through local multi-head self-attention within limbs and combining it with limb-level graph convolution to achieve global information transmission, it not only highlights the collaborative relationship of highly dynamic limbs but also avoids the weakening of local structural information by global attention. It accurately portrays the collaborative dependence of non-local joints in complex postures and improves the model's ability to represent complex movements.

[0029] Based on the local geometric constraints provided by the GCN module of the TriadFormer module in this invention, the HIP-SSM and LimbFormer modules complement each other, capturing both chain kinematic dependencies and cross-part topological relationships. This overcomes the shortcomings of simple module stacking in existing methods and achieves simultaneous improvement in estimation accuracy and generalization ability without significantly increasing computational complexity.

[0030] This invention uses end-to-end network design and residual stacking strategy to gradually extract structure-aware features, making the model more robust when dealing with long sequence actions, occluded scenes and complex posture changes. The deep integration of human anatomy and kinematic constraints reduces the model's sensitivity to the order of joint traversal and optimizes the stability of feature representation, providing more reliable 3D human posture estimation support for practical applications such as virtual reality and motion analysis.

[0031] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0032] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0034] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0035] Figure 1 : Overall network architecture diagram of the TriadFormer module integrating GCN, HIP-SSM and LimbFormer;

[0036] Figure 2 : Detailed structural flowchart of the HIP-SSM module including prior injection of anatomical structures and state updates;

[0037] Figure 3 : Schematic diagram of human skeletal joints sequenced according to kinematic chains and fused with topological priors;

[0038] Figure 4 : A schematic diagram of the internal structure of the LimbFormer module, which includes local MHSA and cross-limb interaction;

[0039] Figure 5 A schematic diagram illustrating the overall flow of the main internal structure of the LimbFormer module;

[0040] Figure 6 Visualizations comparing real and predicted 3D poses in different scenarios;

[0041] Figure 7 : Heatmap of feature response of HIP-SSM module and attention weight matrix of LimbFormer module; Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0043] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0044] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0045] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0046] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0047] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0048] Example 1

[0049] This embodiment mainly describes a three-dimensional human pose estimation method that integrates anatomical prior knowledge and state-space modeling, such as... Figure 1 As shown, it specifically includes:

[0050] A 2D human pose key point sequence is obtained, and the 2D human pose key point sequence is embedded into a high-dimensional feature space through a linear projection layer to obtain joint feature representation;

[0051] The joint feature representation is input into a backbone network consisting of multiple TriadFormer modules stacked in a residual manner. The TriadFormer module includes a graph convolutional network (GCN) module, a hierarchical intra-partial state space model (HIP-SSM), and a limb leader asymmetric local-global transformer (LimbFormer).

[0052] The graph convolutional network (GCN) module captures local spatial dependencies between joints and limb constraints, and outputs local structural enhancement features.

[0053] The hierarchical intra-particulate state space model (HIP-SSM) is used to inject anatomical priors and model chain kinematic dependencies, outputting structural awareness features.

[0054] The limb-leading asymmetric local-to-global transformer (LimbFormer) enhances intralimb coordination and cross-limb global interaction, outputting enhanced joint features.

[0055] The enhanced joint features are mapped to a three-dimensional coordinate space by a linear projection output head to obtain a three-dimensional human pose estimation result.

[0056] Furthermore, the 2D human pose keypoint sequence covers key joints of the human torso, left and right upper limbs, and left and right lower limbs. Its input form is an N×2 dimensional coordinate matrix, where N is the total number of joints. Each row of the matrix corresponds to the horizontal and vertical coordinates of a single joint in the 2D image plane. The linear projection layer contains a learnable weight matrix. With bias term The weight matrix The dimension is Bias term The dimension is , The hidden feature dimension is preset for the model; matrix multiplication and addition operations are used. By mapping 2D coordinate vectors from a 2D low-dimensional space to a D-dimensional high-dimensional feature space, a joint feature representation of shape N×D is obtained. This projection process achieves an effective transformation from raw coordinate information to high-dimensional semantic features through iterative optimization of learnable parameters;

[0057] Furthermore, the backbone network is composed of multiple TriadFormer modules stacked sequentially using residual connections. The number of modules stacked is determined based on actual training needs and application scenarios. For example... Figure 2 As shown, Figure 2 This demonstrates the complete structural flow of a single TriadFormer module: joint feature representation. The input GCN module performs local feature extraction. The local structural enhancement features output by the GCN module are processed by two linear transformation layers. One path is input into the HIP-SSM module for structural prior injection and kinematic dependency modeling, while the other path serves as a residual branch, directly adding and fusing the output features of the HIP-SSM module element-wise. The fused features are then input into the LimbFormer module for local and global interactive enhancement, outputting the feature processing result of the TriadFormer module. Adjacent TriadFormer modules use the same residual connection mechanism. The output features of the previous TriadFormer module are added element-wise with the output features of the current module, then normalized by the LayerNorm layer to eliminate feature scale differences before being input into the next TriadFormer module. This multi-layer stacking achieves progressive feature extraction.

[0058] Furthermore, the Graph Convolutional Network (GCN) module constructs a computational framework based on the natural graph structure of the human skeleton, where joints serve as nodes in the graph and the physiological connections of limbs serve as edges, thereby generating an initial adjacency matrix. Elements in the adjacency matrix =1 indicates the first The joint and the first Each joint has a direct connection to the limb. =0 indicates no direct limb connection. To avoid losing joint feature information during feature aggregation, the initial adjacency matrix is ​​adjusted accordingly. Add identity matrix Construct an adjacency matrix with self-loops. identity matrix The dimension is Its diagonal elements are all 1s, and its off-diagonal elements are all 0s. Based on an adjacency matrix with self-loops. Construct the degree matrix degree matrix For a diagonal matrix, its diagonal elements Equal to adjacency matrix with self-loop The sum of the elements in the i-th row represents the number of connected nodes (including itself) of the i-th joint. To address the feature scale imbalance caused by differences in the number of connected nodes among different joints, a self-loop adjacency matrix is ​​used. Perform double normalization; the normalization formula is as follows: ,in Degree matrix The inverse square root matrix. The feature aggregation operation formula of the GCN module is: ,in ∈ For graph convolution, the weight matrix can be learned. ∈ The bias term is ReLU, and the activation function is ReLU. This operation aggregates the first-order neighborhood joint features and the local features of each joint, outputting local structure enhancement features. This enhances the model's ability to perceive the local topological structure of the human body.

[0059] Furthermore, the structure and computational flow of the Hierarchical Partial In-Order State Space Model (HIP-SSM) are as follows: Figure 3 As shown, its working process is divided into four stages: feature preprocessing, anatomical structure prior injection, state update, and feature output. In the feature preprocessing stage, the local structure of the input is enhanced using a LayerNorm layer. Normalization is performed, and the calculation formula is as follows: This eliminates scale differences between different joint features, ensuring the stability of subsequent calculations. In the anatomical structure prior injection stage, two types of fixed indices are assigned to each joint: location indexes and site indexes. Intra-part depth index The part index The value range is {0, 1, 2, 3, 4}, corresponding to the five functional units: trunk, left upper limb, right upper limb, left lower limb, and right lower limb, respectively. (Depth index within the location) Joints are assigned hierarchical positions within their respective locations to differentiate the kinematic priority of different joints within the same location. A learnable positional code is appended to each joint based on two types of indices, and the enhanced feature representation formula is as follows: ,in For the part embedding matrix, For the depth embedding matrix within the region, This represents the maximum number of joints contained in a single part. The normalized feature of the j-th joint is used; during the state update phase, a chain-like propagation mechanism is used to model the kinematic dependencies between joints, and the state update equation is: ,in ∈ Let be the hidden state vector of the a-th joint, containing the kinematic feature information of that joint. ∈ This represents the preceding hidden state of the (a-1)th joint. ∈ This is a globally shared state transition matrix used to characterize the transmission of kinematic dependencies between joints. ∈ This is the input transformation matrix specific to the a-th joint, used to adapt to the feature input characteristics of different joints. In the feature output stage, it is obtained through the observation equation... The hidden state is mapped to a structure-aware feature, where ∈ To integrate the hidden state of structural perception prior to anatomical structure. ∈ Let be the observation matrix of the a-th joint. ∈ The output feature of the a-th joint is formed by combining the output features of all joints. This is the output of the HIP-SSM module.

[0060] Furthermore, the structure and computational flow of the limb-leading asymmetric local-to-global transformer (LimbFormer) are as follows: Figures 4-5 As shown, it comprises three functional units: a local modeling submodule, a cross-limb interaction submodule, and a global attention submodule. The local modeling submodule divides all joints into five independent functional parts according to anatomical structure: spine, left upper limb, right upper limb, left lower limb, and right lower limb. Each part independently performs Local Multi-Head Self-Attention (PartMHSA) operations. For the p-th part, the structural perception features of that part are obtained through three independent linear mapping layers. Perform dimensional transformation to generate query vectors Key vector AND value vector The mapping formulas are respectively , ,in The weight matrix is ​​a learnable matrix. The formula for local multi-head self-attention is: ,in For each dimension of attention head, To determine the number of attention heads, the computation results are processed through residual connections and LayerNorm layers to obtain local enhancement features. The cross-limb interaction submodule provides local enhancement features for each body part. Average pooling is performed to generate limb-level node features, calculated using the following formula: ,in For the first The set of joints contained in each part. The number of joints in this area is used to obtain a 5×D limb-level node matrix. A five-node semantic graph is constructed based on human anatomical structure, and its adjacency matrix is... The adjacency matrix satisfies the condition that all diagonal elements are 1 (self-loop connection), and the node corresponding to the spine is bidirectionally connected to the four nodes corresponding to the left upper limb, right upper limb, left lower limb, and right lower limb (corresponding elements in the adjacency matrix are 1), while other non-corresponding elements are 0. (This is related to the adjacency matrix.) After performing row normalization, we obtain Then, a single-layer graph convolution operation is performed. ,in ∈ The graph convolution can learn weight matrix, which is used to realize high-order dependency propagation between the spine and limbs, resulting in an updated limb-level node matrix. Update the limb-level node features Broadcast to each joint in the corresponding area, along with the local enhancement features of that joint. By performing element-by-element addition and fusion, intermediate features with cross-limb perception capabilities are obtained. The global attention submodule perceives features of the original structure through a linear mapping layer. Perform the transformation to generate a global query vector. ,in ∈ To query the mapping weight matrix; the key vector and value vector are derived from intermediate features. Generated through linear mapping, the mapping formulas are as follows: Here is the corresponding learnable weight matrix; the global self-attention calculation formula is: The computational results are processed by residual connection and LayerNorm layer to obtain enhanced joint features. . Figure 4 It demonstrates the feature flow relationship between local modeling, cross-limb interaction and global attention in the LimbFormer module, and clearly presents the complete process from local feature extraction to global feature fusion.

[0061] Furthermore, the linear projection output head consists of multiple fully connected networks connected in series, used to enhance joint features. Mapped to a three-dimensional coordinate space. The pre-connected network contains at least one linear transformation layer and an activation function; the weight matrix of the linear transformation layer has dimensions of... The bias dimension is , As an intermediate feature dimension, joint features are enhanced through linear transformation. Dimensional transformation and feature extraction are performed, with a non-linear transformation introduced through the ReLU activation function to enhance the model's ability to express complex features. The final fully connected layer contains a learnable weight matrix. With bias term ∈ Through matrix multiplication and addition operations The refined intermediate features are mapped to three-dimensional space, where This represents the output features of the preceding fully connected network. The output is a 3D human pose estimation result with an N×3 shape. Each element in the matrix corresponds to the x, y, and z coordinates of a joint in three-dimensional space, thus fully restoring the three-dimensional posture structure of the human body.

[0062] This implementation deeply integrates prior knowledge of human anatomy and kinematic constraints, enhances local topology awareness with the GCN module, models chain dependencies with the HIP-SSM module, optimizes local and global interactions with the LimbFormer module, and achieves progressive feature extraction with a residual stacked backbone network. This alleviates the problems of depth ambiguity and pose coherence, reduces the model's sensitivity to the order of joint traversal, and improves the structural consistency of pose estimation while maintaining efficient computational characteristics in complex action and occlusion scenarios.

[0063] Based on Example 1, this application conducted experiments and evaluations on Human3.6M, the largest indoor dataset for 3D human pose estimation. Following standard protocols, the model was trained on subjects 1, 5, 6, 7, and 8, and tested on subjects 9 and 11. MPJPE represents the mean joint distance error, P-MPJPE represents the mean joint distance error after rigid alignment, and MPJPE(GT) represents the input 2D joints originating from the real-world situation. The evaluation results are shown in Table 1.

[0064] Table 1. Performance comparison results of this invention with other studies on Human 3.6M.

[0065] As shown in Table 1, on the Human 3.6M dataset, the MPJPE value of TriadFormer in this invention is 47.8, the P-MPJPE value is 37.7, and the MPJPE(GT) value is 31.9; the error values ​​for each evaluation metric are the lowest. Therefore, TriadFormer in this invention achieves better prediction accuracy.

[0066] This invention compares its performance with other studies on the MPI-INF-3DHP dataset. PCK represents the percentage of correctly predicted joints (threshold 150mm), and AUC represents the area under the curve. The results are shown in Table 2.

[0067] Table 2. Performance comparison results of this invention with other studies on the MPI-INF-3DHP dataset.

[0068] As can be seen from Table 2, on the MPI-INF-3DHP dataset, TriadFormer in this invention also achieves the best performance in terms of the percentage of correctly predicted joints (PCK), area under the curve (AUC), and mean joint distance error (MPJPE).

[0069] To visually demonstrate the performance of the technology in different action scenarios, such as Figure 6 As shown, the figure includes pose estimation results for three different scenarios. The left side of the figure shows the input 2D human keypoint pose, and the right side shows the 3D pose structure reconstructed based on the proposed technical solution. It can be observed that under different movement amplitudes, different body orientations, and scenarios with complex limb pose changes, the reconstructed 3D pose maintains good consistency in overall skeleton topology, relative limb positions, and joint coherence. This demonstrates the effectiveness of the proposed technology in cross-limb information interaction and structural constraint modeling, thereby improving the stability of human pose reconstruction under complex movement conditions and highlighting the advantages of the LimbFormer module in cross-limb interaction modeling.

[0070] Combining the model's real-time performance with feature visualization results, such as Figure 7 As shown, the left subplot is a feature visualization heatmap of the HIP-SSM module. Different colors represent feature response intensity. It can be observed that the feature responses of the trunk and limb joints are significantly higher than other areas, and the joint features of the same limb show a continuous distribution, proving that the anatomical prior has been successfully embedded in the feature representation, and the chain dependency relationship between joints has been effectively modeled. The right subplot is a visualization of the attention weights of the LimbFormer module. The red areas represent joint pairs with higher attention weights, showing strong attention interactions between spinal joints and limb joints, and between adjacent joints of the same limb, verifying the effectiveness of the asymmetric local-global attention mechanism.

[0071] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A three-dimensional human pose estimation method that integrates anatomical prior knowledge and state-space modeling, characterized in that, include: A 2D human pose key point sequence is obtained, and the 2D human pose key point sequence is embedded into a high-dimensional feature space through a linear projection layer to obtain joint feature representation; The backbone network is constructed, including multiple TriadFormer modules. The TriadFormer modules include a graph convolutional network (GCN) module, a hierarchical part-order state space model (HIP-SSM), and a limb-leading asymmetric local-global transformer (LimbFormer). The joint feature representation is input into the graph convolutional network (GCN) module to capture local spatial dependencies and limb constraints between joints and output local structural enhancement features. The local structural enhancement features are input into the hierarchical intra-part order state space model HIP-SSM, anatomical priors are injected and chain kinematic dependencies are modeled, and structural awareness features are output. The structurally aware features are input into the limb-leading asymmetric local-to-global transformer LimbFormer to enhance intralimb coordination and cross-limb global interaction, and output enhanced joint features. The enhanced joint features are mapped to a three-dimensional coordinate space by a linear projection output head to obtain a three-dimensional human pose estimation result.

2. The three-dimensional human pose estimation method integrating anatomical prior and state space modeling as described in claim 1, characterized in that, The joint feature representation embeds the 2D human pose key point sequence into a high-dimensional feature space through a linear projection layer; The 2D human pose keypoint sequence has a shape of N×2, where N is the number of joints and N≥17, covering the key joints of the human torso, left and right upper limbs and left and right lower limbs; the linear projection layer contains a learnable weight matrix and bias terms, which maps the 2D coordinate vector from 2D to a D-dimensional high-dimensional feature space through matrix multiplication, resulting in a joint feature representation of shape N×D, where D is the model hidden dimension and D≥1024. The projection process also introduces learnable parameters to achieve an effective conversion from low-dimensional coordinate information to high-dimensional semantic features.

3. The three-dimensional human pose estimation method according to claim 1, which integrates anatomical prior and state space modeling, is characterized in that... The backbone network consists of multiple TriadFormer modules stacked in a residual manner. Each TriadFormer module includes a graph convolutional network (GCN) module, a hierarchical intraorder state space model (HIP-SSM), and a limb-leading asymmetric local-global transformer (LimbFormer). Features are transferred between adjacent TriadFormer modules through residual connections. That is, the output features of the previous module are added to the output features of the current module, normalized by the LayerNorm layer, and then input to the next module. Through multi-layer stacking, structurally aware joint features are gradually extracted.

4. The three-dimensional human pose estimation method according to claim 3, which integrates anatomical prior and state space modeling, is characterized in that... The Graph Convolutional Network (GCN) module is based on the natural graph structure of the human skeleton, with joints as nodes and limb connections as edges, constructing a fixed adjacency matrix. The module's graph convolution operation achieves local information fusion by aggregating the first-order neighborhood joint features of each joint. The specific calculation process is as follows: ,in For the first Output characteristics of each joint For the first The set of neighboring joints of a joint. The adjacency matrix is ​​the first... Line number The elements of the column represent joints. and The connection strength, The graph convolution weight matrix, This is a bias term; it efficiently captures the geometric relationships and limb constraints between adjacent joints through operations, enhancing the model's ability to perceive local topological structures.

5. The three-dimensional human pose estimation method integrating anatomical prior and state space modeling according to claim 3, characterized in that, The hierarchical intra-order state space model (HIP-SSM) includes feature preprocessing, topological prior injection, state update, and feature output, specifically: The local structural enhancement features of the input are normalized by using the LayerNorm layer to eliminate interference caused by differences in feature scale. Two types of learnable position codes are attached to each joint: location index and positional code. and depth index within the part The enhanced feature representation is obtained as follows: ,in The matrix is ​​embedded for different body parts, corresponding to five functional symmetrical units: torso, left / right arm, left / right leg, etc. For the depth embedding matrix within the region, This represents the maximum number of joints in a single body part, distinguishing the hierarchical positional relationships of different joints within that body part. By outputting structure-aware features through state update and observation equations, the features are ensured to retain the topological information of the human skeleton.

6. The three-dimensional human pose estimation method according to claim 5, which integrates anatomical prior and state space modeling, is characterized in that... The state update equation for the hierarchical intraorder state-space model HIP-SSM is: ,in For the first The hidden state vector of each joint contains kinematic feature information of the joint. For the first Hidden predecessor states of each joint sequence The state transition matrix is ​​shared globally. For the first Each joint has its own dedicated input transformation matrix; The formula, output from the observation equation, is as follows: ,in To integrate the structure-aware hidden states after topological priors. For the first The observation matrix of each joint, The output is the structure-aware feature.

7. The three-dimensional human pose estimation method integrating anatomical prior and state space modeling as described in claim 1, characterized in that, The LimbFormer, a limb-leading asymmetric local-to-global transformer, includes a local modeling submodule, a cross-limb interaction submodule, and a global attention submodule. The local modeling submodule divides the 17 joints into five anatomical functional regions: spine, left upper limb, right upper limb, left lower limb, and right lower limb. Each region independently performs a local multi-head self-attention (PartMHSA) operation, with the following formula: ,in Parts The query, key, and value vectors obtained by linear mapping of internal joint features. , For the input feature dimension, To determine the number of attention heads, the local attention output is processed through residual connections and LayerNorm layers to obtain local enhancement features that strengthen intralimb synergy. .

8. The three-dimensional human pose estimation method according to claim 7, which integrates anatomical prior and state space modeling, is characterized in that... The cross-limb interaction submodule achieves cross-body information interaction through three steps: feature aggregation, graph convolution update, and feature broadcasting; and enhances local features for each body part. Average pooling is performed to generate limb-level node features, using the following formula: ,in For the first The set of joints contained in each part. The number of joints in this region is given; subsequently, a five-node semantic graph and adjacency matrix are constructed. Satisfying diagonal self-connection and bidirectional connection between the spine node and four limb nodes, a single-layer graph convolution is performed after normalization: ,in The normalized adjacency matrix, For graph convolution, the weight matrix can be learned. The initial limb-level node matrix is ​​used, and higher-order dependency propagation between the spine and limbs is achieved through graph convolution. Updated limb-level node features The data is broadcast to each joint in the corresponding area and fused with the local enhancement features of that joint to obtain intermediate features with cross-limb perception capabilities.

9. The three-dimensional human pose estimation method according to claim 8, which integrates anatomical prior and state space modeling, is characterized in that... The global attention submodule employs an asymmetric attention mechanism to construct global associations; it transforms the original structural perception features through a linear mapping layer to generate a global query vector. , ,in For batch size, For the number of joints, To query the mapping weight matrix, ensure that the Query retains the original joint-level feature information; the Key and Value vectors are generated by linearly mapping the intermediate features output by the cross-limb interaction submodule, so that the key-value pair representation naturally integrates local limb collaboration patterns and cross-limb topological information; perform global self-attention calculation, the formula is: The output results are processed by residual connections and LayerNorm layers to obtain the enhanced joint features of the target.

10. The key design of the three-dimensional human pose estimation method that integrates anatomical prior and state space modeling as described in claim 1 is the main component APTB (Asymmetric Part-wise Topology) in LimbFormer. The Block's linear projection output head consists of two fully connected network layers and activation functions. The first fully connected network layer maps the enhanced joint features from D-dimensional to 160-dimensional, introducing a non-linear transformation through the ReLU activation function to enhance feature representation. The second fully connected network layer maps the 160-dimensional features to 3-dimensional, corresponding to the x, y, and z coordinates in three-dimensional space. The loss function of the output head adopts the mean squared error loss (MSE), which optimizes the network parameters by minimizing the squared error between the predicted 3D coordinates and the true coordinates, and outputs a 3D human pose estimation result P with shape N×3, where each row vector corresponds to the 3D spatial coordinates of a joint.

Images