Method and system for 3d human pose estimation based on context and anatomical interaction

Abstract

The present application belongs to the field of computer vision, and provides a three-dimensional human pose estimation method and system based on context and anatomical interaction, comprising: acquiring a monocular image and extracting a two-dimensional coordinate sequence of human joint points in the image coordinate system in the image; based on anatomical structure constraints, the two-dimensional coordinate sequence and the original image are fused to generate initial high-dimensional features that fuse visual semantics and anatomical priors; the initial high-dimensional features are input into an interactive fusion module, the local anatomical structure and the global context semantics are cooperatively modeled, and the dynamic joint dependency relationship under the current pose is captured to obtain enhanced features after deep fusion; the enhanced features are subjected to nonlinear transformation and dimension mapping, and the three-dimensional spatial coordinates of the human joint points are output. This method achieves advanced estimation accuracy and generalization ability on public benchmarks, effectively solving the ambiguity and irrationality problems in monocular three-dimensional pose estimation.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a three-dimensional human pose estimation method and system based on context and anatomical interaction. Background Technology

[0002] 3D human pose estimation aims to reconstruct a 3D skeleton conforming to human anatomy from monocular images or videos. It is a fundamental task in computer vision, with significant applications in augmented reality, action recognition, and human-computer interaction. Current research methods mainly follow two paradigms: one is end-to-end direct coordinate regression, which, while direct, relies on large amounts of expensive 3D labeled data; the other is the "2D-to-3D upscaling" method, which first uses a mature 2D pose detector to obtain 2D joints, and then upscales them to 3D space through geometric priors or learning models. The latter has become mainstream due to its effective use of abundant 2D labeled data and pre-trained models, but it is still limited by the inherent depth ambiguity of monocular projection.

[0003] Early fully convolutional networks (FCNs) located joints through 3D heatmap regression, but their independent modeling of each joint made their predictions susceptible to occlusion and often resulted in anatomically inaccurate poses. Subsequently, graph convolutional networks (GCNNs) explicitly introduced prior knowledge of human structure by modeling joints as graph nodes and encoding the human body's topology using a predefined adjacency matrix, thus improving the geometric plausibility of poses. However, the receptive field of traditional GCNNs is usually limited to local neighborhoods, making it difficult to model dependencies between distant joints such as the left and right hands and feet, and leading to feature homogenization in deep networks.

[0004] In recent years, Transformer models based on self-attention mechanisms have been widely used in this field, capable of modeling interactions between arbitrary joint pairs and effectively capturing long-range dependencies. However, pure Transformer architectures often ignore inherent anatomical constraints of the human body, potentially outputting biomechanically unreasonable poses, and their deep structures are often accompanied by high computational complexity. To address this, some studies have attempted to construct hybrid models combining Transformers and graph convolutional networks to simultaneously achieve global contextual understanding and local structure preservation. However, existing hybrid methods often fail to achieve deep adaptive fusion of anatomical priors and visual semantics, or exhibit insufficient stability under complex poses.

[0005] While video sequence-based methods can leverage temporal information to improve performance, they suffer from limitations such as high computational cost and sensitivity to frame rate. Therefore, efficient and robust single-frame image 3D pose estimation methods remain crucial for resource-constrained real-world applications. The core challenge of current single-frame methods lies in effectively coordinating the use of global environmental semantics within the image with local anatomical constraints in the human body model to simultaneously address multiple challenges, including depth blur, severe occlusion, and pose accuracy. Summary of the Invention

[0006] The purpose of this invention is to provide a three-dimensional human pose estimation method and system based on context and anatomical interaction to solve the above-mentioned problems.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a three-dimensional human pose estimation method based on context and anatomical interaction, comprising: Acquire a monocular image and extract the two-dimensional coordinate sequence of human joints in the image coordinate system; Based on anatomical constraints, the two-dimensional coordinate sequence and the original image are fused to generate initial high-dimensional features that integrate visual semantics and anatomical priors. The initial high-dimensional features are input into the interactive fusion module, which collaboratively models the local anatomical structure and global contextual semantics, and captures the dynamic joint dependencies under the current pose, to obtain the enhanced features after deep fusion. The enhanced features are subjected to nonlinear transformation and dimension mapping to output the three-dimensional spatial coordinates of the human joints.

[0008] Furthermore, the step of acquiring a monocular image and extracting the two-dimensional coordinate sequence of human joints in the image coordinate system includes: A pre-trained two-dimensional human pose detection model is used to automatically detect and output the two-dimensional coordinate sequence of each joint point of the human body in the image pixel coordinate system from the input monocular image; the pre-trained model is a pose detector based on deep learning.

[0009] Furthermore, based on anatomical constraints, the two-dimensional coordinate sequence and the original image are fused to generate initial high-dimensional features that integrate visual semantics and anatomical priors, including: A lightweight convolutional neural network is used as the feature extraction backbone to encode the original image and obtain multi-scale visual feature maps containing different spatial resolutions. Simultaneously, based on the pixel coordinates of each joint point in the two-dimensional coordinate sequence, a two-dimensional Gaussian distribution is generated on a two-dimensional spatial grid of a preset size, centered on the coordinate point and with a preset standard deviation, to obtain a Gaussian heatmap representing the confidence of the position of each joint point. Through cross-modal feature fusion operations, the multi-scale visual feature maps and Gaussian heatmaps are integrated to obtain image context semantic features containing global and local semantics of the image. Image contextual semantic features are input into a learnable gating function, which outputs a soft attention weight matrix of the same dimension as a predefined fixed anatomical adjacency matrix. This soft attention weight matrix is ​​then multiplied element-wise with the fixed anatomical adjacency matrix, which encodes the physical connections between human joints based on physiological structure in binary form. Through this multiplication operation, the predefined fixed connections are adaptively weighted based on image content, thereby generating a dynamic hybrid anatomical mask with continuous element values. The two-dimensional coordinate sequence is mapped to a high-dimensional latent space through a learnable linear projection layer to obtain high-dimensional coordinate features. A spatial gating unit is constructed, which uses the hybrid anatomical mask as a control signal to perform spatial dimension gating modulation on the high-dimensional coordinate features. Specifically, it enhances feature channels that are strongly correlated with anatomical structures and suppresses irrelevant or weakly correlated feature responses to achieve feature sparsity and semantic filtering, and finally outputs the initial high-dimensional features.

[0010] Furthermore, the initial high-dimensional features are input into the interactive fusion module, and dynamic joint dependencies in the current pose are captured by collaboratively modeling local anatomical structures and global contextual semantics, resulting in deeply fused enhanced features, including: A static anatomical perception module is constructed, which consists of a graph convolutional network branch based on a fixed anatomical adjacency matrix A and a residual fully connected layer branch in parallel. The graph convolutional network branch transmits information on the human body topology and applies local kinematic chain constraints. The residual fully connected layer branch provides a global receptive field by performing nonlinear transformation on the input features, capturing nonlocal joint dependencies; the outputs of the two branches are added together and then fused with the input features through residual connections, thereby achieving a preliminary integration of local anatomical constraints and global contextual information; A hybrid local-global fusion module is constructed, which includes parallel anatomical perception paths and self-attention paths. The anatomical perception path maintains the perception capability of a fixed anatomical topology by integrating graph convolutional networks and residual fully connected layers, while the self-attention path captures the contextual dependencies between any global joint pairs through a multi-head self-attention mechanism. The intermediate features calculated by the anatomical perception path are multiplied by the first learnable scalar parameter to obtain the attention flow from joint to space, and then injected into the self-attention path. The intermediate features calculated by the self-attention path are multiplied by the second learnable scalar parameter to obtain the attention flow from space to joint, and then injected into the anatomical perception path. The anatomical perception path integrates its own features with the space-to-joint attention flow and processes them to obtain an output result; the self-attention path integrates its own features with the joint-to-space attention flow and processes them to obtain an output result; finally, the output results of the two paths are fused to obtain an enhanced feature representation.

[0011] Furthermore, capturing the dynamic joint dependencies in the current pose includes: Based on the fusion features, calculate the Euclidean distance between all joint feature vectors, specifically for the first joint in the batch. Any two joints of a sample and Its distance is ; Based on distance, dynamically select the joint with the smallest distance for each joint. Each joint is considered as its neighbor, and a dynamic adjacency matrix is ​​constructed accordingly. If the joint It is a joint of One of the neighbors, Otherwise, it is 0. This matrix represents the dynamic joint associations that transcend fixed anatomical structures under the current input posture. Using the dynamic adjacency matrix Combining Chebyshev graph convolution with dynamic graph convolution to aggregate dynamic neighborhood information, its first... The output of the convolution is calculated as follows: ,in Based on The normalized graph Laplacian matrix is ​​calculated. For the first Chebyshev polynomials The weight matrix is ​​a learnable matrix; the fused features are obtained by performing a nonlinear transformation on the fused features through a residual fully connected layer in parallel with the convolution operation of the dynamic graph. The sum of the multi-level outputs of the dynamic graph convolution is added to the output of the residual fully connected layer, and this is used as the feature after dynamic perception.

[0012] Furthermore, the step of performing nonlinear transformation and dimensional mapping on the enhanced features to output the three-dimensional spatial coordinates of the human joint points includes: The dynamically perceived features are input into a feature enhancement module consisting of multiple residual fully connected layers for nonlinear transformation. Finally, the feature dimensions are mapped to three dimensions through a linear projection layer, and the three-dimensional spatial coordinates of each joint point are output.

[0013] Furthermore, the network is trained and optimized using a dynamic weight scheduling strategy and a hybrid loss function combining coordinate error and orientation constraints, including: Using a hybrid loss function The network is trained, where The average positional error per joint, For skeletal cosine loss, For a predefined set of bone edges, and These are the actual and predicted skeleton vectors, respectively. For fixed weights, Based on training rounds The dynamically adjusted weights and their scheduling strategy are as follows: ,in As the initial weights, Hyperparameters for controlling the adjustment rate.

[0014] Secondly, the present invention provides a three-dimensional human pose estimation system based on context and anatomical interaction, comprising: The data acquisition module is used to acquire monocular images and extract the two-dimensional coordinate sequence of human joints in the image coordinate system. The high-dimensional feature acquisition module is used to fuse the two-dimensional coordinate sequence and the original image based on anatomical structure constraints to generate initial high-dimensional features that integrate visual semantics and anatomical priors. The fusion enhancement module is used to input the initial high-dimensional features into the interactive fusion module, and obtain the enhanced features after deep fusion by collaboratively modeling the local anatomical structure and global contextual semantics and capturing the dynamic joint dependencies under the current pose. The output module is used to perform nonlinear transformation and dimension mapping on the enhanced features and output the three-dimensional spatial coordinates of the human joints.

[0015] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the three-dimensional human pose estimation method based on context and anatomical interaction.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the three-dimensional human pose estimation method based on context and anatomical interaction.

[0017] Compared with the prior art, the present invention has the following technical effects: This invention acquires monocular images and extracts 2D joint coordinates. Based on a hybrid anatomical mask module and an anatomical guided encoder, it generates initial features that fuse visual semantics and anatomical constraints. A static anatomical perception module initially fuses local and global features, and a dual-path bidirectional attention flow mechanism enables deep interaction between the anatomical perception path and the self-attention path, collaboratively modeling local anatomical structures and global contextual semantics. A dynamic feature perception module then captures pose-related joint dependencies, ultimately regressing and outputting 3D coordinates. This method employs a hybrid loss function optimized with dynamic weight scheduling, effectively addressing depth blur, severe occlusion, and anatomical inconsistencies in single-frame estimation. It improves estimation accuracy while ensuring physiological plausibility of the pose, and the model is efficient and suitable for real-time applications.

[0018] This invention fully utilizes the adaptive fusion of anatomical priors and visual semantics, the bidirectional interaction of local constraints and global dependencies, and the adaptive construction of dynamic graph structures, which significantly improves the accuracy and robustness of pose estimation in complex scenarios. At the same time, the model has a compact structure, high inference efficiency, and is easy to deploy and apply in practical systems. Attached Figure Description

[0019] Figure 1 This is an overall flowchart of a single-frame image three-dimensional human pose estimation system method based on context and anatomical interaction according to the present invention.

[0020] Figure 2 This is a schematic diagram of the context-anatomy embedding module (S2) in this invention.

[0021] Figure 3 This is a schematic diagram of the static anatomical sensing module (S3) in this invention.

[0022] Figure 4 This is a diagram of the architecture of the hybrid local-global fusion module (S4) in this invention.

[0023] Figure 5 This is a schematic diagram of the working principle of the dynamic feature perception module (S5) in this invention. Detailed Implementation

[0024] The present invention will be further described below with reference to the accompanying drawings: Example 1, please refer to Figure 1This invention provides a three-dimensional human pose estimation method based on context and anatomical interaction, including: Acquire a monocular image and extract the two-dimensional coordinate sequence of human joints in the image coordinate system; Based on anatomical constraints, the two-dimensional coordinate sequence and the original image are fused to generate initial high-dimensional features that integrate visual semantics and anatomical priors. The initial high-dimensional features are input into the interactive fusion module, which collaboratively models the local anatomical structure and global contextual semantics, and captures the dynamic joint dependencies under the current pose, to obtain the enhanced features after deep fusion. The enhanced features are subjected to nonlinear transformation and dimension mapping to output the three-dimensional spatial coordinates of the human joints.

[0025] This invention provides a method for estimating 3D human pose from a single frame image based on context and anatomical interaction. Its core lies in adaptively fusing the visual semantics of the original image with prior knowledge of human biomechanics through a learnable hybrid anatomical masking mechanism, and collaboratively modeling local joint kinematic constraints and global spatial dependencies in a dual-path interaction architecture, thereby achieving robust and accurate reconstruction of 3D human pose from a single frame image.

[0026] Example 2: This invention provides a three-dimensional human pose estimation method based on context and anatomical interaction, including: S1: Acquire a monocular image and extract the two-dimensional coordinate sequence of the human joints in the image, specifically: A pre-trained, deep learning-based 2D human pose detection model is used to automatically analyze the input monocular RGB image. The model outputs a 2D coordinate sequence containing N key human joints; these coordinates are located in the image pixel coordinate system, providing a foundation for subsequent 2D-to-3D upscaling.

[0027] S2: The two-dimensional coordinate sequence and the original image are processed through a context-anatomical embedding block to generate initial high-dimensional features that fuse visual semantics and anatomical constraints. See [link to relevant documentation]. Figure 2 Specifically: A lightweight convolutional neural network is used to extract multi-scale visual feature maps from the original image; Based on the two-dimensional coordinate sequence obtained from S1, a Gaussian heatmap is generated for each joint on a preset two-dimensional grid. The calculation formula is as follows:

[0028] in, It is a joint Normalized coordinates, The standard deviation is used. Through cross-modal fusion, the visual feature map and Gaussian heatmap are integrated to form a joint-level contextual semantic descriptor. .

[0029] A fixed binary adjacency matrix is ​​predefined. This involves encoding the physiological connections between human joints. A learnable gating function is designed to utilize contextual semantic features. As input, output the soft attention weight matrix. Compare this with a fixed matrix. Element-wise multiplication yields the soft dissection mask. :

[0030] in, This indicates element-wise multiplication. It is the Sigmoid activation function. These are learnable weights. Further, they are processed using a multilayer perceptron. Generate dynamic gating coefficients The final hybrid anatomy mask Calculate using the following formula:

[0031] The two-dimensional coordinate sequence is mapped to a high-dimensional space through linear projection to obtain the features. Construct spatial gating units to hybrid anatomical masks. To control the signal, high-dimensional features are modulated. This process can be described as follows:

[0032] in, For learnable scaling factor, As a spatial gating unit, it is responsible for using the mask. The relevant features are enhanced and the irrelevant features are suppressed, and the initial high-dimensional features are finally output.

[0033] S3: The initial high-dimensional features are processed by the static anatomical perception module to achieve a preliminary fusion of local kinematic constraints and global dependencies. See [link to documentation]. Figure 3 Specifically: Construct a module containing two parallel branches: one branch is based on a fixed anatomical adjacency matrix. Perform Chebyshev graph convolution to capture local kinematic constraints; Another branch employs a residual multilayer perceptron module to capture global dependencies. The features output from S2 are input into this module, and the processing flow is as follows: Intermediate features are obtained through graph convolution. ; The normalized input is processed through residual MLP branching. The calculation for this branch is as follows:

[0034] The outputs of the two branches are fused with the input features through residual connections, resulting in the final output features. :

[0035] S4: Through a hybrid local-global fusion module, a dual-path bidirectional attention flow mechanism is used to interactively fuse features from the anatomical perception path and the self-attention path to collaboratively model local anatomical structures and global contextual semantics. This step is performed by the hybrid local-global fusion module. See also Figure 4 The specific implementation method is as follows: This module is designed with two parallel and complementary processing paths: the anatomical perception path and the self-attention path. Anatomical perception path integrates the ResMLP-Spatial module with a fixed anatomical adjacency matrix The graph convolutional network module provides the ability to transform non-local features across joints, while the graph convolutional network module explicitly enforces kinematic constraints between local joints by passing information on the graph structure. Together, they are responsible for extracting and refining local structural information that conforms to anatomical logic from the input features. The self-attention path employs a standard multi-head self-attention mechanism to capture global dependencies that are not limited by physiological connectivity; For the input features, a query is generated through a learnable linear projection. ,key ,value Matrix, where , , This is the weight matrix. (The remaining text appears to be incomplete and lacks context.) , , Split into After the head, each attention head The calculation follows the formula:

[0036] in This is the head dimension. The outputs of all heads are concatenated and linearly transformed to form the path output. This mechanism enables any joint to interact directly with all other joints, thereby modeling global spatial context semantics; To achieve effective complementarity of information between the two paths, this invention introduces a bidirectional attention flow mechanism for interaction; The features output from the previous step S3 After layer normalization, the normalized features are used as input for this module. The anatomical perception path processes the normalized features to generate its intermediate feature representation. The features normalized by the self-attention path are used to generate intermediate feature representations through multi-head self-attention computation. ; Construct and inject bidirectional attention flow: on the one hand, dissect the intermediate features of the perceptual pathway. via a learnable scalar parameter Scaling to obtain the joint-to-space attention flow And inject it into the self-attention path to give physiological rationale to global context modeling; intermediate features of the self-attention path via another learnable scalar parameter Scaling to obtain the attention flow from space to joints And inject it into the anatomical perception pathway to guide the integration of local structural perception into global scene understanding; After receiving attention flow information from the other path, the two paths respectively complete feature integration and updating; Anatomical sensory pathway combination Then, the updated features are output by constructing a joint descriptor and transforming it. The process can be represented by the following formula:

[0037] Self-attention path combination Then, the updated features are output via linear projection. The process can be represented by the following formula:

[0038] Output features of the two paths and They are added together and then refined through a fusion submodule; This submodule includes layer normalization, residual connections, and MLP transformations along different dimensions, which can be specifically represented by the following formula:

[0039]

[0040] in This indicates an MLP module that shares parameters along the joint dimension.

[0041] Final output features It deeply integrates the precise constraints of local anatomical structures with the rich information of global contextual semantics, achieving effective collaborative modeling of the two.

[0042] S5: Through the dynamic feature perception module, dynamic joint dependencies related to the current input pose are captured based on dynamic graph convolution and residual fully connected layers. See [link to module]. Figure 5 Specifically: This step is performed by the dynamic feature perception module, whose core function is to adaptively construct a dynamic joint association topology based on the features of the current input pose, and then aggregate information to capture pose-specific dependencies beyond fixed anatomical structures. The module first bases its design on the features output by S4. To construct a dynamic adjacency matrix; For each sample in the batch, calculate any two joints. and The Euclidean distance between the feature vectors measures the similarity of joints in the current high-dimensional feature space, and can be represented by the following formula:

[0043] For each joint Dynamically select the joint with the smallest distance from all other joints. Each joint serves as its dynamic neighbor; Based on this Nearest neighbor relationships are defined by a dynamic binary adjacency matrix. ,in If and only if the joint It is a joint of The nearest neighbor is used; otherwise, the value is 0. This matrix... It encodes dynamic joint associations that are triggered by the current posture and may not be constrained by fixed physiological connections; During the feature aggregation phase, modules collaboratively utilize dynamic topology and static dissection priors; By using a fixed adjacency matrix Derived normalized Laplace matrix The representation employs Chebyshev graph convolution to operate on a graph structure that integrates dynamic associations and basic anatomical constraints, in order to aggregate multi-hop neighborhood information. No. The contribution of order convolution is ,in It is the first Chebyshev polynomials These are learnable weight coefficients; A Residual Multilayer Perceptron (ResMLP) Spatial module processes the input features. Perform parallel nonlinear transformations to enhance its expressive power; The multi-level aggregation results of dynamic graph convolution are combined with the MLP-enhanced features through residual connections, and Dropout regularization is applied to output the final feature representation, which is calculated by the following formula:

[0044] Through the above process, this module can effectively capture the specific joint dependencies generated in complex postures such as crossed arms, and output refined features that respect the basic anatomical structure and adapt to specific posture changes. This provides crucial information for subsequent coordinate regression.

[0045] S6: Enhance the features after dynamic perception processing and map and output the coordinates of 3D human joint points, specifically: This step is the decoding and output stage of the network, responsible for mapping the feature information, which has been fully refined and fused by the aforementioned modules, into the final three-dimensional spatial coordinates. This process consists of a feature enhancement module and an output projection layer in sequence. First, the features output by the dynamic feature perception module are... Input to the feature enhancement module; This module typically consists of multiple stacked residual fully connected layers (i.e., multilayer perceptrons, MLPs). Each residual block performs a nonlinear transformation to further enhance the discriminative power and representation depth of the features. Its basic operation can be expressed as the following formula:

[0046] in For the first The input features of each layer are processed through several layers of non-linear enhancement to obtain a highly refined feature representation. Subsequently, this highly refined feature representation is fed into a simple linear projection layer, which reduces the feature dimension from... Directly map to 3D to obtain the predicted 3D coordinates of each joint. Its basic operation can be expressed as the following formula:

[0047] in and These are the learnable weights and bias parameters. This output... This represents the final 3D human pose estimated by the network based on a single frame of input image.

[0048] S7: The network defined by the above method is trained and optimized using a dynamic weight scheduling strategy and a hybrid loss function combining coordinate error and orientation constraint. Specifically: This step defines the optimization objectives and strategies for network training. Through a composite loss function and its dynamic adjustment mechanism, it collaboratively optimizes the accuracy of 3D coordinate prediction and the anatomical rationality of pose. The training employs a hybrid loss function. Loss due to coordinate error and direction constraint loss The weighted summation constitutes, i.e. ,in For fixed weighting coefficients, To follow the training rounds Dynamically adjusted weighting coefficients.

[0049] Coordinate error loss The Mean Per Joint Position Error (MPJPE) is used to directly measure the predicted 3D joint coordinates. with actual coordinates The Euclidean distance between them is calculated using the following formula:

[0050] This loss term ensures the accuracy of the network output in absolute spatial location.

[0051] Directional constraint loss The aim is to improve the anatomical plausibility of poses by constraining the direction of skeletal vectors. A set of human skeletal edges is defined. For each connecting joint and bone edge Calculate its true skeletal vector The negative cosine similarity between the edges is calculated by averaging over all edges and batches of samples. The formula is as follows:

[0052] This loss term drives the predicted bone orientation to align with the true orientation, effectively suppressing non-physiological postures such as abnormal joint angles. To achieve a balance between coordinate accuracy and anatomical rationality during training, this invention employs a dynamic weight scheduling strategy to adjust the weights of the orientation constraint loss. Adjustments were made. This strategy enabled... From initial value Initially, with each training round... The value increases monotonically with the increase of , and its change follows the following scheduling formula:

[0053] in, To control the hyperparameter of the weight growth rate, this strategy assigns a higher weight to the coordinate error loss in the early stages of training, prioritizing the accuracy of basic spatial positioning. As training progresses, the weight of the orientation constraint loss is gradually increased, guiding the network to further optimize the anatomy of the pose based on the already obtained more accurate coordinates. Ultimately, driven by optimizers such as Adam, the entire network minimizes the aforementioned dynamically weighted hybrid loss. Perform end-to-end training and optimization.

[0054] This invention constructs a multi-stage, multi-modal information deep fusion processing framework for estimating 3D human pose from a single image. Traditional methods often lose the visual semantics of the image itself or ignore the inherent anatomical constraints of the human body when regressing from 2D to 3D coordinates, resulting in unnatural or inaccurate predicted poses. This invention addresses this problem by introducing two main threads: "context" and "anatomy," and designing dedicated modules to allow them to continuously interact. First, visual context features are extracted from the original image and combined with 2D joint coordinates to generate initial fusion features with anatomical prior guidance. Next, a parallel static anatomical perception module and a global self-attention mechanism are used to model local kinematic chain constraints and global spatial dependencies, respectively. The most crucial step is the design of a bidirectional attention flow mechanism, allowing paths representing local anatomical information and paths representing global context to mutually inject information, achieving collaborative optimization. Then, the system can dynamically discover new dependencies between joints based on the features of the current pose, transcending fixed physiological connections to better handle complex poses. Finally, the entire network is trained using a hybrid loss function with dynamically adjusted weights. While optimizing coordinate accuracy, the constraints on the rationality of bone orientation are gradually strengthened, thereby outputting a 3D pose that is both accurate and conforms to human anatomy.

[0055] The specific method of this invention is as follows: The first step, feature initialization and fusion, involves extracting the 2D coordinate sequence of human joints from the input RGB image using a pre-trained 2D pose detection model (such as HRNet, CPM, etc.). Simultaneously, a lightweight convolutional neural network (CNN) is used to extract multi-scale visual features from the image. The joint coordinates are converted into Gaussian heatmaps and fused with the visual features to obtain semantically rich image context features. A fixed binary anatomical adjacency matrix is ​​defined to encode physiological connections such as "wrist to elbow". A learnable gating function is used to generate a soft attention weight matrix with the context features as input, which is then multiplied element-wise with the fixed anatomical matrix to obtain a "hybrid anatomical mask". This mask adaptively adjusts the strength of each anatomical connection based on the image content. Finally, this mask is used to control a spatial gating unit to modulate the high-dimensional coordinate features, enhancing features related to anatomical structures and outputting the initial fused features.

[0056] The second step involves parallel and interactive extraction of local and global features: the initial features are input into a static anatomical perception module. This module has two branches: one branch is based on a graph convolutional network (GCN), which passes information over a fixed anatomical adjacency matrix, enforcing kinematic constraints on local limbs (such as the forearm); the other branch is a residual multilayer perceptron (MLP), which provides a global receptive field through fully connected layers, capturing latent dependencies between non-local joints (such as the left hip and right shoulder). The outputs of the two branches are fused through residual connections, achieving preliminary local-global information integration.

[0057] Subsequently, the features are fed into the core hybrid local-global fusion module. This module designs two parallel paths: an anatomical perception path (integrating GCN and MLP to maintain awareness of fixed anatomical structures) and a self-attention path (employing a multi-head self-attention mechanism to model dependencies between any global joint pairs). To achieve deep interaction, this scheme introduces a bidirectional attention flow mechanism: intermediate features calculated by the anatomical perception path are multiplied by a learnable scalar and injected into the self-attention path as a "joint-to-space" attention flow, injecting anatomical rationality into the global context; conversely, intermediate features of the self-attention path are similarly processed and injected into the anatomical perception path as a "space-to-joint" attention flow, integrating local structure awareness into global scene understanding. After receiving information from each other, the two paths update their own features respectively, and finally, the outputs of the two paths are added and fused to obtain features that deeply integrate precise anatomical constraints and rich contextual semantics.

[0058] The third step, dynamic dependency perception and output: The features obtained through the above interactions are fed into the dynamic feature perception module. This module first calculates the Euclidean distance of all joint features in high-dimensional space, dynamically selects the K most similar joints for each joint as its "dynamic neighbors," and constructs a dynamic adjacency matrix accordingly. This matrix can capture specific dependencies beyond fixed anatomical structures under the current pose (e.g., the association between the hands when "arms crossed"). The module uses this dynamic matrix, combined with a fixed anatomical matrix, to perform graph convolution operations, while simultaneously using an MLP for feature transformation in parallel. Finally, it outputs refined features that respect the basic anatomy and adapt to the specific pose through residual connections. Finally, these features are passed through a feature enhancement module consisting of multiple residual MLP layers and directly mapped to the three-dimensional spatial coordinates of each joint by a linear projection layer.

[0059] Step 4, Training Strategy: The network is trained end-to-end using a hybrid loss function. This loss function consists of two parts: coordinate error loss (MPJPE, Mean Per Joint Position Error), which directly minimizes the Euclidean distance between the predicted and true coordinates to ensure positional accuracy; and orientation constraint loss, which constrains bone orientation by minimizing the negative cosine similarity between the predicted and true bone vectors, thereby improving the anatomical rationality of the pose. A dynamic weight scheduling strategy is employed during training: initially, the coordinate error is primarily optimized, while later the weight of the orientation constraint loss is gradually increased. This guides the network to further optimize the pose rationality based on more accurate coordinates, thus making the training process more stable and effective.

[0060] This invention significantly improves the accuracy and plausibility of 3D pose estimation. Through designs such as "hybrid anatomical masking" and "bidirectional attention flow," the model can simultaneously utilize rich visual semantics from images and constraints from prior knowledge of human anatomical structures. This makes the predicted 3D pose not only closer to the true value in absolute coordinates (with lower MPJPE error), but also more visually natural and plausible, effectively reducing non-physiological errors such as abnormal joint angles and disproportionate limbs.

[0061] Secondly, it enhances robustness to complex poses and occlusion. The "dynamic feature perception module" can adaptively establish dynamic relationships between joints based on the features of the input pose, rather than rigidly relying on fixed physiological connections. This enables the model to better understand and handle complex poses such as crossed arms, sitting postures, and dance movements, as well as situations where some joints are occluded, because dynamic relationships can capture new functional or spatial connections between joints in these scenarios.

[0062] Finally, the training efficiency and convergence of the model were optimized. The adopted "dynamic weight scheduling" training strategy embodies a curriculum-based learning approach. It allows the model to focus on learning basic coordinate regression tasks in the early stages of training, while gradually introducing more complex anatomical rationality constraints in later stages. This phased and progressive optimization avoids the instability or convergence difficulties that may result from multiple strong constraints acting simultaneously, enabling the entire network to converge more smoothly and effectively to a state with better performance.

[0063] In another embodiment of the present invention, a three-dimensional human pose estimation system based on context and anatomical interaction is provided, which can be used to implement the above-mentioned three-dimensional human pose estimation method based on context and anatomical interaction. Specifically, the system includes: The data acquisition module is used to acquire monocular images and extract the two-dimensional coordinate sequence of human joints in the image coordinate system. The high-dimensional feature acquisition module is used to fuse the two-dimensional coordinate sequence and the original image based on anatomical structure constraints to generate initial high-dimensional features that integrate visual semantics and anatomical priors. The fusion enhancement module is used to input the initial high-dimensional features into the interactive fusion module, and obtain the enhanced features after deep fusion by collaboratively modeling the local anatomical structure and global contextual semantics and capturing the dynamic joint dependencies under the current pose. The output module is used to perform nonlinear transformation and dimension mapping on the enhanced features and output the three-dimensional spatial coordinates of the human joints.

[0064] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0065] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a three-dimensional human pose estimation method based on context and anatomical interaction.

[0066] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the three-dimensional human pose estimation method based on context and anatomical interaction in the above embodiments.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A three-dimensional human pose estimation method based on context and anatomical interaction, characterized in that, include: Acquire a monocular image and extract the two-dimensional coordinate sequence of human joints in the image coordinate system; Based on anatomical constraints, the two-dimensional coordinate sequence and the original image are fused to generate initial high-dimensional features that integrate visual semantics and anatomical priors. The initial high-dimensional features are input into the interactive fusion module, which collaboratively models the local anatomical structure and global contextual semantics, and captures the dynamic joint dependencies under the current pose, to obtain the enhanced features after deep fusion. The enhanced features are subjected to nonlinear transformation and dimension mapping to output the three-dimensional spatial coordinates of the human joints.

2. The three-dimensional human pose estimation method based on context and anatomical interaction according to claim 1, characterized in that, The step of acquiring a monocular image and extracting the two-dimensional coordinate sequence of human joints in the image coordinate system includes: A pre-trained two-dimensional human pose detection model is used to automatically detect and output the two-dimensional coordinate sequence of each joint point of the human body in the image pixel coordinate system from the input monocular image; the pre-trained model is a pose detector based on deep learning.

3. The three-dimensional human pose estimation method based on context and anatomical interaction according to claim 1, characterized in that, The process, based on anatomical constraints, fuses the two-dimensional coordinate sequence with the original image to generate initial high-dimensional features that integrate visual semantics and anatomical priors, including: A lightweight convolutional neural network is used as the feature extraction backbone to encode the original image and obtain multi-scale visual feature maps containing different spatial resolutions. Simultaneously, based on the pixel coordinates of each joint point in the two-dimensional coordinate sequence, a two-dimensional Gaussian distribution is generated on a two-dimensional spatial grid of a preset size, centered on the coordinate point and with a preset standard deviation, to obtain a Gaussian heatmap representing the confidence of the position of each joint point. Through cross-modal feature fusion operations, the multi-scale visual feature maps and Gaussian heatmaps are integrated to obtain image context semantic features containing global and local semantics of the image. Image contextual semantic features are input into a learnable gating function, which outputs a soft attention weight matrix of the same dimension as a predefined fixed anatomical adjacency matrix. This soft attention weight matrix is ​​then multiplied element-wise with the fixed anatomical adjacency matrix, which encodes the physical connections between human joints based on physiological structure in binary form. Through this multiplication operation, the predefined fixed connections are adaptively weighted based on image content, thereby generating a dynamic hybrid anatomical mask with continuous element values. The two-dimensional coordinate sequence is mapped to a high-dimensional latent space through a learnable linear projection layer to obtain high-dimensional coordinate features. A spatial gating unit is constructed, which uses the hybrid anatomical mask as a control signal to perform spatial dimension gating modulation on the high-dimensional coordinate features. Specifically, it enhances feature channels that are strongly correlated with anatomical structures and suppresses irrelevant or weakly correlated feature responses to achieve feature sparsity and semantic filtering, and finally outputs the initial high-dimensional features.

4. The three-dimensional human pose estimation method based on context and anatomical interaction according to claim 1, characterized in that, The initial high-dimensional features are input into the interactive fusion module, which captures the dynamic joint dependencies in the current pose by collaboratively modeling local anatomical structures and global contextual semantics, resulting in enhanced features after deep fusion, including: A static anatomical perception module is constructed, which consists of a graph convolutional network branch based on a fixed anatomical adjacency matrix A and a residual fully connected layer branch in parallel. The graph convolutional network branch transmits information on the human body topology and applies local kinematic chain constraints. The residual fully connected layer branch provides a global receptive field by performing nonlinear transformation on the input features, capturing nonlocal joint dependencies; the outputs of the two branches are added together and then fused with the input features through residual connections, thereby achieving a preliminary integration of local anatomical constraints and global contextual information; A hybrid local-global fusion module is constructed, which includes parallel anatomical perception paths and self-attention paths. The anatomical perception path maintains the perception capability of a fixed anatomical topology by integrating graph convolutional networks and residual fully connected layers, while the self-attention path captures the contextual dependencies between any global joint pairs through a multi-head self-attention mechanism. The intermediate features calculated by the anatomical perception path are multiplied by the first learnable scalar parameter to obtain the attention flow from joint to space, and then injected into the self-attention path. The intermediate features calculated by the self-attention path are multiplied by the second learnable scalar parameter to obtain the attention flow from space to joint, and then injected into the anatomical perception path. The anatomical perception path integrates its own features with the space-to-joint attention flow and processes them to obtain an output result; the self-attention path integrates its own features with the joint-to-space attention flow and processes them to obtain an output result; finally, the output results of the two paths are fused to obtain an enhanced feature representation.

5. The three-dimensional human pose estimation method based on context and anatomical interaction according to claim 4, characterized in that, The capture of dynamic joint dependencies in the current pose includes: Based on the fusion features, calculate the Euclidean distance between all joint feature vectors, specifically for the first joint in the batch. Any two joints of a sample and Its distance is ; Based on distance, dynamically select the joint with the smallest distance for each joint. Each joint is considered as its neighbor, and a dynamic adjacency matrix is ​​constructed accordingly. If the joint It is a joint of One of the neighbors, Otherwise, it is 0. This matrix represents the dynamic joint associations that transcend fixed anatomical structures under the current input posture. Using the dynamic adjacency matrix Combining Chebyshev graph convolution with dynamic graph convolution to aggregate dynamic neighborhood information, its first... The output of the convolution is calculated as follows: ,in Based on The normalized graph Laplacian matrix is ​​calculated. For the first Chebyshev polynomials The weight matrix is ​​a learnable matrix; the fused features are obtained by performing a nonlinear transformation on the fused features through a residual fully connected layer in parallel with the convolution operation of the dynamic graph. The sum of the multi-level outputs of the dynamic graph convolution is added to the output of the residual fully connected layer, and this is used as the feature after dynamic perception.

6. The three-dimensional human pose estimation method based on context and anatomical interaction according to claim 6, characterized in that, The process of performing nonlinear transformation and dimensional mapping on the enhanced features to output the three-dimensional spatial coordinates of the human joints includes: The dynamically perceived features are input into a feature enhancement module consisting of multiple residual fully connected layers for nonlinear transformation. Finally, the feature dimensions are mapped to three dimensions through a linear projection layer, and the three-dimensional spatial coordinates of each joint point are output.

7. The three-dimensional human pose estimation method based on context and anatomical interaction according to claim 6, characterized in that, The network is trained and optimized using a dynamic weight scheduling strategy and a hybrid loss function that combines coordinate error and orientation constraints, including: Using a hybrid loss function The network is trained, where The average positional error per joint, For skeletal cosine loss, For a predefined set of bone edges, and These are the actual and predicted skeleton vectors, respectively. For fixed weights, Based on training rounds The dynamically adjusted weights and their scheduling strategy are as follows: ,in As the initial weights, Hyperparameters for controlling the adjustment rate.

8. A three-dimensional human pose estimation system based on context and anatomical interaction, characterized in that, include: The data acquisition module is used to acquire monocular images and extract the two-dimensional coordinate sequence of human joints in the image coordinate system. The high-dimensional feature acquisition module is used to fuse the two-dimensional coordinate sequence and the original image based on anatomical structure constraints to generate initial high-dimensional features that integrate visual semantics and anatomical priors. The fusion enhancement module is used to input the initial high-dimensional features into the interactive fusion module, and obtain the enhanced features after deep fusion by collaboratively modeling the local anatomical structure and global contextual semantics and capturing the dynamic joint dependencies under the current pose. The output module is used to perform nonlinear transformation and dimension mapping on the enhanced features and output the three-dimensional spatial coordinates of the human joints.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the three-dimensional human pose estimation method based on context and anatomical interaction as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the three-dimensional human pose estimation method based on context and anatomical interaction as described in any one of claims 1 to 7.

Images