A method and system for predicting human posture

Abstract

This invention belongs to the field of computer vision and machine learning technology, specifically relating to a method and system for human pose prediction. The invention decomposes historical human pose sequences into multiple potential motion patterns; it uses a noise encoder to extract exogenous variables representing motion intention, environmental context, and unobserved control signals; for each motion pattern, it learns a pattern-specific causal directed acyclic graph by combining the exogenous variables and optimizes it; it combines the refined causal graphs into a unified dynamic causal graph representation; finally, this dynamic causal graph representation is used as a structured condition, injected into each step of the back-diffusion process of the denoising diffusion implicit model through a cross-attention mechanism and time-aware feature modulation, iteratively denoising from random noise using the historical sequence as a condition to generate future human pose sequences. This invention significantly improves the accuracy and physical plausibility of long-term pose prediction through dynamic combination and causal constraints.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and machine learning technology, and specifically relates to a method and system for predicting human pose. Background Technology

[0002] Human pose prediction is a fundamental task in computer vision, aiming to predict pose changes over a future time domain based on observed human pose sequences. It has broad application prospects in fields such as autonomous driving, human-machine collaboration, and virtual reality. Current mainstream methods can be broadly categorized as follows: Methods based on recurrent neural networks treat pose prediction as a sequence-to-sequence learning problem, but their autoregressive structure leads to error accumulation over time steps; methods based on graph convolutional networks capture spatial dependencies between joints by modeling the human skeleton as a graph structure, but typically employ fixed or weakly adaptive graph structures; methods based on Transformers utilize self-attention mechanisms to model long-range spatiotemporal dependencies, achieving good mid-term prediction results; and methods based on diffusion models generate diverse future motion trajectories through iterative denoising processes, offering advantages in handling long-term uncertainties.

[0003] However, existing methods generally suffer from a common core technical problem: a lack of ability to express dynamic biomechanical causal mechanisms that adapt to the evolution of movement stages. Specifically, on the one hand, most existing methods are essentially correlation-driven, learning statistical co-occurrence patterns in historical movement sequences. When the prediction time domain is extended or the movement pattern changes, these statistical correlations often fail, leading to physically unreasonable kinematic drift phenomena in predicted postures, such as foot slippage and joint distortion, resulting in poor generalization ability. On the other hand, a few methods attempt to introduce causal modeling, typically assuming that human movement follows a fixed, static causal graph structure. However, human movement exhibits significant non-stationarity and stage transition characteristics. For example, walking includes different stages such as the support phase and the swing phase, and the active and passive relationships, constraints and drives between joints change significantly. A single static causal graph cannot adapt to such dynamic changes, nor can it express the cyclic feedback and multi-joint synergistic effects in movement, leading to severe long-term prediction distortion.

[0004] Therefore, how to construct a method that can dynamically evolve with the movement phase to truly reflect the causal driving relationship between joints is a core technical problem that urgently needs to be solved in current human posture prediction. Summary of the Invention

[0005] To address the problems existing in the prior art, the purpose of this invention is to provide a human posture prediction method and system. Through dynamic combination and causal constraints, it can dynamically evolve with the movement stage to truly reflect the causal driving relationship between joints, significantly improving the accuracy and physical rationality of long-term posture prediction.

[0006] The technical solution of this invention is: A method for predicting human posture includes the following steps: Historical human posture sequences are collected and decomposed into multiple potential motion patterns. The historical human posture sequence is a series of observed human posture data, and each frame of posture consists of the position coordinates of multiple joints of the human body in three-dimensional space. Historical human posture sequences are input into a noise encoder to extract exogenous variables that represent potential contextual information. Potential contextual information includes motion intention, environmental context, and unobserved control signals. Environmental context refers to external conditions in the physical environment in which the observed human body is located that can affect the motion trajectory. For each motion pattern, a pattern-specific causal directed acyclic graph is learned by combining exogenous variables. The causal directed acyclic graph is then optimized by a counterfactual refinement strategy based on structural intervention, resulting in multiple refined causal directed acyclic graphs. The adjacency matrix and corresponding causal representation of each refined causal directed acyclic graph are mapped to key vector and query vector respectively. The attention weight of each mode-specific causal directed acyclic graph is calculated by scaling dot product attention mechanism. The attention weight is used as a weighting coefficient to linearly combine all mode-specific causal directed acyclic graphs to obtain dynamic causal graph. The dynamic causal graph is then flattened and projected onto the feature space of a preset dimension through a linear layer to obtain the dynamic causal graph representation. The dynamic causal graph representation is used as a structured condition. Through cross-attention mechanism and time-aware feature modulation, it is injected into each step of the back diffusion process of a denoising diffusion implicit model. With the historical human posture sequence as the condition, iterative denoising starts from random noise to generate the future human posture sequence.

[0007] Preferably, the method for determining the plurality of potential motion patterns includes the following steps: The historical human posture sequence is divided into multiple overlapping short-time motion segments using a sliding window with a preset length and step size. A truncated approximation Dirichlet process mixture model is adopted, and a lightweight one-dimensional convolutional neural network is used as a global clustering network to assign each short-time motion segment to a corresponding latent motion pattern, wherein each latent motion pattern is characterized by a Gaussian distribution. The posterior probability of each segment belonging to each motion mode is determined by Bayes' formula, and negligible components are pruned according to mixed weights, ultimately resulting in multiple potential motion modes determined adaptively by the data.

[0008] Preferably, the method for extracting the exogenous variable includes the following steps: Obtain multiple motion patterns determined after motion pattern decomposition and their corresponding segment allocation information; Based on the segment allocation information, extract the posture subsequence corresponding to the current motion mode from the historical human posture sequence; Flatten the posture subsequence into a one-dimensional vector; The one-dimensional vector is input to the noise encoder, which is a multilayer perceptron containing three fully connected layers, each followed by a ReLU activation function. The multilayer perceptron is used to encode the one-dimensional vector into low-dimensional exogenous variables and output the exogenous variables. The exogenous variables are used to characterize the latent context information, which includes motion intention, environmental context, and unobserved control signals. During the training phase, the noise encoder adds Gaussian noise sampled from a standard multivariate Gaussian distribution to the output of the multilayer perceptron to introduce randomness and improve generalization ability; during the inference phase, the Gaussian noise is not added to obtain deterministic output.

[0009] Preferred methods for learning a pattern-specific causal directed acyclic graph include the following steps: For the p Each motion pattern defines an adjacency matrix. To encode the causal structure of its directed acyclic graph, where non-zero elements Indicates the first j The variable for the first i Each variable exerts a direct causal effect; Each causal variable is modeled as a nonlinear function of its parent node variable and the corresponding exogenous input, and a continuous acyclic penalty is introduced to constrain each mode-specific graph to a directed acyclic graph. The continuous acyclic penalty is determined according to the following formula: , in, H ( A P ) is the first p The continuous non-cyclic penalty value of a causal directed acyclic graph of a motion pattern; No. p Adjacency matrix of each motion pattern; Represents the Hadamard product; d The variable dimension is the total number of nodes in the causal graph. , N Indicates the number of joints in the human body. T This indicates the length of the historical human posture sequence.

[0010] Preferably, the causal directed acyclic graph is optimized using a counterfactual refinement strategy based on structural intervention, specifically including: For a given number p A pattern-specific structural causal model, applying an intervention to the target variable that sets it to a constant value; Based on the topological order of the causal directed acyclic graph, the intervention effect is propagated through a graph neural network, and the counterfactual value of each variable is calculated. Define descendant loss and non-descendant loss, wherein the descendant loss is used to ensure that descendant nodes should change under intervention, and the non-descendant loss is used to ensure that non-descendant nodes should remain unchanged under intervention, and construct counterfactual loss based on the weighted sum of the descendant loss and non-descendant loss; The causal directed acyclic graph is refined with the goal of minimizing the counterfactual loss function.

[0011] Preferably, the attention weights of the pattern-specific causal directed acyclic graph are determined according to the following formula: , In the formula, Attention weights for pattern-specific causal directed acyclic graphs; For the first p The query vector corresponding to a causal directed acyclic graph; For the first p The key vectors corresponding to each causal directed acyclic graph; The scaling factor for stabilizing the gradient. For the attention dimension.

[0012] A preferred method for iteratively denoising random noise to generate future human pose sequences includes the following steps: The historical human posture sequence is encoded as historical motion features, the dynamic causal graph representation is projected onto the same feature dimension, and the two are spliced ​​together to form conditional memory. Sample initial noisy motion features from standard Gaussian noise; For each back-diffusion step within the total number of steps, perform the following: The noisy motion features of the current step are used as a query and cross-attention calculation is performed with the conditional memory to obtain the motion features of causal perception. A sinusoidal time-step embedding is generated based on the current diffusion step number. The sinusoidal time-step embedding is mapped to scaling and shift parameters through a multilayer perceptron. The scaling and shift parameters are then used to modulate the motion features of the causal perception. The modulated features are input into the noise prediction network of the denoising diffusion implicit model to predict the noise of the current step, and the noisy motion features are updated according to the predicted noise to obtain the noisy motion features of the next step. Repeat the above back-diffusion steps until all T steps are completed, and decode the final noisy motion features into a future human posture sequence.

[0013] Preferably, the future human posture sequence is determined by future human posture features, which are determined according to the following formula: , In the formula, For future human posture characteristics; Let be the noisy future motion feature at step t; For the spread noise scheduling coefficient; This is the cumulative noise dispatch coefficient; For noise prediction networks; X A sequence of historical human postures; Z Represented as a dynamic cause-effect graph; t This represents the current diffusion time step.

[0014] Preferably, it also includes an optimization strategy step, which comprises two stages: Phase 1: Causal discovery and counterfactual graph refinement phase, which involves jointly training the motion pattern decomposition, the exogenous variable extraction, and the dynamic causal graph fusion process. The pattern-specific causal directed acyclic graph is optimized using counterfactual loss and continuous noncyclic penalty. After convergence in this phase, the trained causal parameters are frozen. The second stage is the causal conditional diffusion generation stage. Under the guidance of the frozen causal correlation parameters, only the denoised diffusion implicit model is trained, and a causal consistency loss function is introduced. The causal consistency loss function is used to constrain the alignment between the predictions generated conditioned on the original dynamic causal graph representation and the counterfactual predictions generated conditioned on the dynamic causal graph representation after counterfactual intervention.

[0015] A human posture prediction system, comprising: The motion pattern decomposition module is used to decompose historical human posture sequences into multiple potential motion patterns using a Dirichlet process mixture model. The noise encoder module is used to extract exogenous variables representing potential contextual information from the historical human posture sequence; The causal learning and counterfactual refinement module is used to learn and optimize multiple refined causal directed acyclic graphs for each motion pattern, in conjunction with the exogenous variables. The dynamic graph fusion module is used to map the adjacency matrix and corresponding causal representation of each refined causal directed acyclic graph to key vectors and query vectors, respectively. It calculates the attention weight of each mode-specific causal directed acyclic graph through a scaling dot product attention mechanism. The attention weight is used as a weighting coefficient to linearly combine all mode-specific causal directed acyclic graphs to obtain a dynamic causal graph. The dynamic causal graph is then flattened and projected onto a high-dimensional space to obtain a dynamic causal graph representation. The conditional diffusion generation module is used to inject the structured conditions represented by the dynamic causal graph into each step of the denoising diffusion implicit model through cross-attention mechanism and time-aware feature modulation, and iteratively denoise to generate future human posture sequences.

[0016] Compared with the prior art, the human posture prediction method and system of the present invention have the following beneficial effects: This invention first decomposes historical pose sequences into multiple potential motion patterns and extracts exogenous variables to represent the underlying context. Then, for each pattern, a specific causal directed acyclic graph (DAG) is learned and optimized using a counterfactual refinement strategy based on structural intervention, ensuring that the learned dependencies reflect genuine causal effects rather than spurious statistical associations. Next, an attention-based weighted fusion mechanism combines multiple refined causal graphs into a unified dynamic causal graph representation. This representation naturally expresses cross-pattern recurrent feedback and multi-joint synergistic effects, overcoming the expressive capacity bottleneck of static DAGs. Finally, this dynamic causal graph representation is used as a structured condition, and cross-attention mechanisms and temporally aware feature modulation are applied. In each step of the backdiffusion process of the denoising diffusion implicit model, the future posture sequence is generated iteratively from random noise, conditioned on the historical posture sequence. Thus, through the above working principle, the present invention not only relies on the statistical distribution of historical observations during the generation process, but is also explicitly guided by dynamic biomechanical causal constraints. This results in an effective reduction of the average position error per joint in the long-term prediction time domain, a significant improvement in prediction accuracy, and effective suppression of physical artifacts such as foot slippage and joint twisting. The generated posture sequence has been effectively improved in terms of joint coherence, kinematic constraint preservation, and fidelity of end-body details. Furthermore, the causal structure can be dynamically adjusted according to the movement pattern, providing interpretability for model decision-making. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method in an embodiment of the present invention; Figure 2 This is a block diagram illustrating the method execution structure in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the principle of the counterfact generation process in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0020] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0021] See Figures 1 to 3 As shown, in order to dynamically reflect the causal driving relationship between joints during the movement phase and improve the accuracy and physical rationality of human posture prediction, this embodiment provides a human posture prediction method, including the following steps: Historical human posture sequences are collected and decomposed into multiple potential motion patterns. The historical human posture sequence is a series of observed human posture data, and each frame of posture consists of the position coordinates of multiple joints of the human body in three-dimensional space.

[0022] Historical human posture sequences are input into a noise encoder to extract exogenous variables representing potential contextual information. This potential contextual information includes motion intention, environmental context, and unobserved control signals. Environmental context refers to external conditions in the physical environment of the observed human body that can influence the motion trajectory. Unobserved control signals refer to potential or hidden factors that cannot be directly obtained from historical posture data (joint coordinates) but profoundly affect future motion trajectories.

[0023] For each motion pattern, a pattern-specific causal directed acyclic graph is learned by combining exogenous variables. The causal directed acyclic graph is then optimized by a counterfactual refinement strategy based on structural intervention, resulting in multiple refined causal directed acyclic graphs.

[0024] The adjacency matrix and corresponding causal representation of each refined causal directed acyclic graph are mapped to key vectors and query vectors, respectively. The attention weights of each mode-specific causal directed acyclic graph are calculated through a scaling dot product attention mechanism. The attention weights are used as weighting coefficients to linearly combine all mode-specific causal directed acyclic graphs to obtain a dynamic causal graph. The dynamic causal graph is then flattened and projected onto a feature space of a preset dimension through a linear layer to obtain the dynamic causal graph representation.

[0025] The dynamic causal graph representation is used as a structured condition. Through cross-attention mechanism and time-aware feature modulation, it is injected into each step of the back diffusion process of a denoising diffusion implicit model. With the historical human posture sequence as the condition, iterative denoising starts from random noise to generate the future human posture sequence.

[0026] The specific implementation process of the above steps is as follows (see Figure 2 and Figure 3 (as shown) Given a segment of length T Historical posture sequence abbreviated as X ,in N The number of joints in the human body. D =3 represents the three-dimensional coordinate dimension, for each pose. Depend on N The three-dimensional joint positions corresponding to the number of joints in an individual's body, i.e. The goal is to predict future lengths of... pose sequence Learn a conditional generation mapping .

[0027] Phase 1: Motion pattern decomposition.

[0028] Human movement is inherently non-stationary, with different time periods corresponding to different movement phases, each exhibiting unique inter-joint interaction patterns. Learning a single global causal structure from the entire sequence can lead to incompatible dependencies, limiting the expressive power of causal modeling. Therefore, this invention first employs a Dirichlet Process Mixture Model (DPM) to decompose the observed motion sequence into multiple potential motion patterns.

[0029] For input historical attitude sequence Using a length of L =8. A sliding window with a step size of 1 divides it into multiple overlapping segments. i Each segment is defined as ,in Each segment The dynamic coherence of short-term motion is preserved as an independent observation sample for DPM clustering.

[0030] To classify motion sequences into potential motion patterns, this invention employs a truncated approximation Dirichlet process mixture model (DPM) with a maximum number of components. ,parameter The mixed weights are sampled from a symmetric Dirichlet distribution:

[0031] , in, satisfy Movement segments Potential component allocation m i Sampling from categorical distribution The fragment itself follows a Gaussian distribution of the corresponding component. Fragment i Belongs to components k The posterior probability is given by Bayes' theorem:

[0032] , In the formula, For the first The motion segment belongs to the first The posterior probability of each component; For a given fragment Mixed weights And the mean and covariance of all components. Under the conditions, fragment Assigned to component The conditional probability; For the first The mixed weight of a component represents the proportion of that component in the whole; For fragments In the Probability density function values ​​under a Gaussian distribution; For the first The mean vector of the Gaussian components, i.e., the centroid; For the first The covariance matrix of a Gaussian component; Normalization factor, i.e., fragment In all The sum of weighted probability densities under each component; To truncate the approximate maximum number of components, we set it to 30.

[0033] To efficiently obtain posterior assignments, this invention employs a lightweight one-dimensional convolutional neural network as the global clustering network. The network uses two one-dimensional convolutional layers (kernel size 3, with 32 and 64 channels respectively), each followed by ReLU activation and batch normalization. After global average pooling to compress the time dimension, a 64-unit fully connected layer (ReLU activation) further processes the features, and finally, a softmax output layer generates a soft-assigned probability vector. Each element Approximate Bayesian posterior probability The network and DPM parameters are jointly trained via amortized variational inference.

[0034] After DPM training converges: 0. Each sports mode p By a Gaussian distribution Describe and associate it with the set of fragments assigned to it. .in, For the first Gaussian distributions corresponding to each motion pattern; The mean vector (centroid) of this Gaussian distribution describes the average pose of the motion segment in this mode; Let be the covariance matrix of the Gaussian distribution, which describes the degree of linkage between joints in this mode. To be assigned to the A collection of short motion segments of a motion pattern; For the first A short segment of motion; To select the component with the highest posterior probability, if equal to Then the fragment is classified into the pattern. ; For fragments Belongs to components The posterior probability (by The output is the network mentioned earlier.

[0035] 0. Components with negligible mixed weights ( ,in The number of surviving components after pruning is the final number of motion patterns. .

[0036] for T =30 frames of human motion sequence, sliding window L =8, Step size 1, Number of segments (in, This represents the total number of short-duration motion segments; Here, is the total number of frames in the historical pose sequence. ; Here is the length of the sliding window. ; stride Here is the step size for each window slide. stride ; (For rounding down), parameter The expected number of patterns meets , p Automatically determined by data (e.g.) p=8). This initialization produces dense, data-driven priors that capture pattern-specific dependencies without strong manual biases. The initial graphs do not have to be acyclic; they serve as flexible starting points and are further refined into efficient DAGs by NOTEARS.

[0037] Second stage: Exogenous variable extraction (noise encoder).

[0038] Causal motion generation depends not only on observable joint trajectories but also on latent exogenous factors such as motion intent, environmental context, and unobserved control signals. To capture these factors, this invention introduces a noise encoder that maps the observation sequence to exogenous variables (i.e., extracts exogenous variables that characterize latent contextual information). ;in, This is a set of exogenous variables, i.e., latent contextual information; For the first The joint in the first The exogenous variables corresponding to the frame; For joint indexing, ; For frame index, ; This represents the total number of joints in the human body. This represents the total number of frames in the historical pose sequence.

[0039] Given an input sequence of historical poses First, flatten it into dimensions. The vector is passed through a noisy encoder containing a multilayer perceptron (MLP). This MLP consists of three fully connected layers, each followed by a ReLU activation. Gaussian noise is injected to introduce randomness and improve generalization:

[0040] , in, This is a set of exogenous variables, i.e., the extracted potential contextual information; It is a multilayer perceptron, with three fully connected layers (512 → 256 → 128, ReLU); Given the input historical pose sequence, dimension ; The noise is Gaussian, with a mean of zero and a covariance of the identity matrix. : Standard multivariate Gaussian distribution, where 0 is the zero-mean vector. It is the unit covariance matrix.

[0041] During training Sample during each forward propagation, and set during inference. To obtain deterministic output. This design introduces randomness into the causal learning process, enabling the model to take into account unobservable sources of variation, rather than forcibly attributing all variability to observable poses. This makes the learned causal model more expressive and better matches the inherent uncertainty of human movement.

[0042] Phase 3: Pattern-specific causal learning and counterfactual refinement.

[0043] Given observation sequence X Exogenous variables U and initialization graph set This invention learns a set of pattern-specific structural causal models (SCM) for each motion pattern, i.e. .

[0044] For the p Each pattern, adjacency matrix Encoding Correspondence Diagram The causal structure of (the directed acyclic graph corresponding to the p-th motion pattern). Among them, non-zero elements... Representing variables For variables Imposing a direct causal effect. Each causal variable is modeled as a nonlinear function of its parent variable and its corresponding exogenous input:

[0045] , in, For the first In the first motion mode The values ​​of the causal variables; For the first Nodes in each pattern The structural mechanism function is a non-linear function (implemented by the subsequent GNN). For nodes The set of values ​​that all parent nodes can take, i.e., the graph All points in Nodes; For the first Nodes in the diagram The set of parent node indices; For the first Nodes in each pattern The corresponding exogenous input.

[0046] No. The causal generation process of an SCM is described as follows: , in, Let p be the complete causal representation vector of the p-th pattern, with dimension p. ; For identity matrix, dimension ; For the first Adjacency matrix of each pattern; transpose; It is an inverse matrix, used to store exogenous variables. The propagation of the cause-effect graph through directed edges (parent → child → grandchild) is equivalent to writing the node-by-node calculation of Formula 3 in matrix form all at once. For all nodes, the exogenous input vectors are used. The adjacency matrix is ​​the first i The line encodes all pointers to nodes. i Causal edge weights; For element-wise multiplication, The effect is to retain only the nodes. The value of the parent node is multiplied by zero for non-parent nodes; This is the vector of current values ​​for all nodes; for Learnable parameters; Weighted values ​​and parameters of the parent node The input is a nonlinear function.

[0047] For element-wise multiplication, This statement ensures that... according to The causal structure is generated through encoding, rather than through unconstrained aggregation.

[0048] To ensure causal identifiability within each homogeneous motion state, this invention constrains each specific pattern graph into a directed acyclic graph through continuous acyclic penalties: , in, H ( A P ) is the first p The continuous non-cyclic penalty value of a causal directed acyclic graph of a motion pattern; No. p Adjacency matrix of each motion pattern; Represents the Hadamard product; d The variable dimension is the total number of nodes in the causal graph. , N Indicates the number of joints in the human body. T This indicates the length of the historical human posture sequence.

[0049] While each subgraph remains acyclic to ensure tractable causal discovery, subsequent attention-based fusion (stage four) linearly combines multiple DAGs across modes, which naturally produces efficient directed loops and higher-order hypergraph dependencies at the sequence level, capturing feedback loops and multi-joint synergies without violating NOTEARS identifiability guarantees.

[0050] Learning graph structures solely from observational data does not guarantee that the learned dependencies reflect true causal effects. To refine the learned structural equations (SCMs) and improve causal fidelity, this invention introduces a counterfactual generation strategy based on structural intervention. This intervention is performed within the learned latent space, thus constituting a soft intervention: it is used to regularize the structural equations, enforce cross-modal motion dynamic decoupling, and suppress spurious co-activation between latent joint representations.

[0051] For a given specific pattern SCM, the target variable Intervene and set it to a constant value. c , recorded as ,like Figure 3 As shown.

[0052] Intervention Then, following the topological propagation of the effect according to the causal graph, the counterfactual value of each variable is calculated as follows: , in, For pattern Medium variables The counterfactual value (the horizontal bar above the head indicates the "counterfactual world after intervention"); A constant value set for intervention, that is, forcibly setting the target variable. Change it to this value; If the current variable is the target variable being intervened in, directly assign it a value. ; If the current variable is not the target of intervention, then through the structural mechanism... Recalculate; For variables The set of counterfactual values ​​of all parent nodes (note that the values ​​of the parent nodes themselves have also been changed by intervention, so use counterfactual values ​​instead of the original values); For the first Variables in the graph The set of parent node indices; For variables The corresponding inferred exogenous noise (take the corresponding part from the output of the noise encoder); The same structural mechanism function as before the intervention is implemented by GNN.

[0053] This invention uses graph convolutional networks (GNNs) in... above The adjacency matrix is The network has three graph convolutional layers with 256, 128, and 64 channels respectively, each using ReLU activation, and a final node-level linear layer. Exogenous input. It is concatenated with the features of the parent node before the final prediction.

[0054] Based on the aforementioned counterfactual generation process, evaluation metrics are defined to enforce causal fidelity. Let The set of all variables. Intervention node. In the figure The descendants are defined as follows:

[0055] }, in, For nodes In the figure The set of descendants in; The set of all variables; Let be a variable node in the graph; The target node to be intervened.

[0056] The descendant loss and the non-descendant loss are defined as follows: , in, For the first The descendant loss of each pattern; For the first Non-descendant loss of each pattern; The size of the descendant set (number of descendant nodes) is used as the normalization denominator; The size of the non-descendant set. This represents the set difference, which is the entire set minus its descendants; For variables The change before and after the intervention (counterfactual value minus original value); The L2 norm square of the change measures the degree to which the variable is affected by the intervention.

[0057] The counterfactual loss, which reflects the idea that offspring should change under intervention rather than remain stable, is defined as a weighted sum: , in, : No. Counterfactual losses in each pattern; The weight of offspring loss; a larger value emphasizes that "offspring must change accordingly." : The weight of non-descendant loss; a smaller value indicates that "non-descendant invariance" is a soft constraint. , We separately control for the importance of changes in offspring and non-offspring. To prevent offspring loss from approaching zero (indicating that offspring did not change under intervention), we further introduce a margin-based regularization term:

[0058] , in, : No. Marginal penalty of each pattern; When the loss of offspring is below a threshold A penalty is only applied when the threshold is exceeded (to prevent the intervention from being completely blocked); the penalty is zero when the threshold is exceeded. This is the marginal threshold. The total counterfactual loss is:

[0059] , in, For the first Total counterfactual loss for each pattern; : The weight of the marginal regularization term.

[0060] Phase 4: Fusion of higher-order causal graphs and generation of conditional diffusion.

[0061] Different movement patterns correspond to different causal mechanisms. Using only a single-pattern-specific graph during inference will discard complementary structural information and fail to capture the combined characteristics of real human movement. Therefore, this invention will... A learned causal graph They are integrated into a unified, high-order dynamic causal graph representation. .

[0062] Each optimized graph is defined by its adjacency matrix. It indicates. Among them, Measure from node The causal strength. To adaptively combine these graphs, this invention introduces a learnable attention mechanism. For each motion pattern First, a two-story It will express causality Mapped to dimensional query vector At the same time, key vectors are derived from the flattened adjacency matrix. and the value vector of the encoded graph structure . No. The attention weights for each graph are calculated using scaled dot product attention:

[0063] , In the formula, The attention weights for a causal directed acyclic graph; For the firstp The query vector corresponding to a causal directed acyclic graph; For the first p The key vectors corresponding to each causal directed acyclic graph; The scaling factor for stabilizing the gradient. For the attention dimension.

[0064] The fused dynamic cause-effect graph is defined as follows: It aggregates the causal structure of all motion patterns. Although each Individually, they are acyclic, but their weighted combinations naturally produce richer structures: if the pattern coding And the pattern coding The fused DCG in and Positive values ​​at all points indicate effective bidirectional feedback. Similarly, joints appearing as common parent nodes in multiple subgraphs generate higher-order synergistic effects in the aggregation matrix. To obtain a fixed-dimensional representation for conditional generation, the DCG (dimension ...) is... Flattening and projecting through a linear layer yields a higher-order causal graph representation. ,in This is the latent dimension. This representation serves as a dynamic structural prior for diffusion-based predictions, encoding how different joints interact across time within the current motion context.

[0065] In the diffusion generation section, this invention uses the aforementioned denoising diffusion implicit model (DDIM) to generate... (Dynamic cause-effect graph representation) represents the future motion under certain conditions. The DDIM model represents uncertainty, while... Applying dynamic structural priors guides the denoising process toward predictions consistent with causality and biomechanics. For simplicity, let's use... Represents the observed attitude sequence, Represents a future sequence. This represents the causal representation after fusion. The latent space is obtained by embedding it through a three-layer MLP. The forward diffusion process is scheduled according to variance. Gradually towards Add Gaussian noise:

[0066] , In the formula, This represents the conditional distribution for forward diffusion; For the first t Noise-added future motion embedding; For the first Embedded in the future of movement; For the first The noise variance scheduling parameter of each step controls how much noise is added in each step; This is a scaling factor to ensure that signal energy is maintained while noise is added; Gaussian distribution, mean is The covariance is .

[0067] Accordingly, the formula for direct sampling from a noisy distribution is: , In the formula, Embedding of raw, un-noiseed future motion; For cumulative noise dispatch coefficient, ,in ; From 0 to t The cumulative signal scaling factor of the step, as t Increases while decreasing; To accumulate noise variance, as t It increases to approximately 1.

[0068] To make the conditional denoising process reproducible, this invention employs explicit injection via a cross-attention module and time-aware FiLM modulation. .make Indicates noise reduction backbone from Noisy motion features extracted from, among which L For the number of future movement tokens, Hidden dimension. Observation sequence. X Encoded as And concatenate it with the projected causal token to form conditional memory:

[0069] , In the formula, This is conditioned memory, specifically the key and value in cross-attention; For historical posture sequence X Embedding, dimensional , The number of tokens in the historical movement; As a causal token, by Projected onto the denoised feature space; To be a learnable projection matrix, The causal representation of dimension is mapped to dimension. For dynamic cause-effect graph representation, dimensions . For splicing along the direction of travel; The total number of condition tokens after concatenation (historical tokens plus 1 causal token); To hide the dimension.

[0070] in, The fused causal representation is mapped to the denoised feature space. At each back-diffusion step, the noisy motion features... As a query, conditional memory As key and value:

[0071] in For a learnable projection matrix, For the attention dimension. Causal perception of motion features is calculated using scaled dot product attention:

[0072] For the first Causal perceptual motion characteristics of the diffusion step; The query matrix is ​​obtained by linear projection of the current noisy motion features; The key matrix is ​​determined by conditional memorization. Obtained by linear projection; For value matrices, also determined by conditional memorization. Obtained by linear projection; Conditioned memory, that is ; There are three learnable projection matrices; For attention dimension; This is a scaling factor to prevent the softmax gradient from vanishing due to excessively large dot product values; To query the similarity matrix with all keys, each element represents the relevance between a query token and a condition token; Normalize along the dimension of the bond to obtain the attention weight distribution; : The number of future movement tokens.

[0073] Then Projection Back Dimensions are modulated using time-aware FiLM. Sine time-step embedding. Scaling and shift parameters are generated using MLP:

[0074] In the formula, For the first t The sinusoidal time step embedding of the diffusion step encodes the current denoising progress; For a dedicated multilayer sensor, the time step embedding is mapped to modulation parameters; The scaling parameter adjusts the intensity of the causal sensing feature channel by channel; The shift parameter is used to adjust the offset of the causal sensing features channel by channel; To hide the dimension, the dimension of the parameter is the same as the number of feature channels.

[0075] The final conditional features are obtained in the following way: In the formula, : No. t The final motion characteristics after injecting causal conditions; Linear projection, from Dimensional projection back dimension; Channel-wise multiplication, broadcast along the token dimension; Scaling modulation; : Shift modulation; Residual connections directly add back the original noisy features to prevent causal modulation from overwriting the original motion information.

[0076] The denoising network predicts noise from the injected features, meaning this design enables the causal representation to interact directly with the noisy pose token, while the FiLM parameters adaptively adjust the condition strength at different diffusion steps.

[0077] In the reverse denoising process, the model uses the observation sequence and dynamic causal representation Embedding for conditionally predicting future motion in the previous step: , In the formula, Learnable inverse denoising distribution For network parameters; : No. Motion embedding after denoising step (compared to the first step) (Cleaner steps) : No. The current noisy motion embedding of the step; Historical observation attitude sequence; : Dynamic cause-effect graph representation; The denoised mean value predicted by the network is: 、 、 and time step t For input; : No. t The noise variance scheduling parameter of the step determines the step size of the denoising step; Gaussian distribution, mean is The covariance is ; For the sake of brevity, the following will be Recorded as The predicted future posture is reconstructed as follows: , In the formula, From the first t The future posture is predicted by directly jumping steps; Inverse scaling factor, restores the original scale of the signal; : No. t Noisy motion embedding of steps; The predicted noise output by the noise prediction network; The noise removal factor determines how much predicted noise is subtracted from the current noisy embedding. : The abbreviation of .

[0078] The diffusion target is defined by the standard noise prediction loss: , In the formula, Training loss of the diffusion model; Mathematical expectation, the average of three random sources; Sampled from the true future pose distribution; : The actual Gaussian noise (real noise) added during forward diffusion; : Diffusion time step of random sampling; Noise in network prediction; The mean square error between the actual noise and the predicted noise; From the truth and noise Constructed noisy embedding.

[0079] Optimize strategies.

[0080] Jointly optimizing DPM clustering, NOTEARS causal discovery, counterfactual graph refinement, and continuous diffusion denoising directly within a single end-to-end objective introduces unstable gradients because discrete graph structure learning and generative denoising operate on different optimization landscapes. Therefore, CaMoG employs a two-stage training paradigm that decouples causal discovery from diffusion generation.

[0081] The first stage involves causal discovery and counterfactual graph refinement. First, the motion sequence is divided into latent motion patterns using DPM. For each pattern, an initial causal graph is estimated using NOTEARS optimization. Then, a counterfactual GNN is applied... Intervention and forced causal fidelity preservation are used to refine the discovered graph. Counterfactual causal representations are obtained by intervening in the original causal graph. Counterfactual fidelity loss Only at this point is it used to refine the causal graph and obtain the fused dynamic causal representation. After the first phase convergence, the discovered causal graph, graph fusion module, and causal encoder are frozen.

[0082] The second stage is causal conditional diffusion generation. After the causal module is frozen, the denoising network of DDIM is trained using Z_CaMoG as a non-updated structural condition. In addition to the standard diffusion loss, the generation process is regularized using counterfactual causal variables generated by the frozen causal module. Specifically, using... As an alternative condition, the diffusion model generates counterfactual future postures. , and the original conditions Generate standard predictions To enforce causal consistency, the two predictions should remain aligned because the underlying structural dynamics should remain unchanged in response to interventions that do not affect the outcome of the motion. The causal consistency loss is defined as:

[0083] , In the formula, Loss of causal consistency; Using counterfactual causation The counterfactual future posture generated by the conditions; Represented by original cause and effect Standard future pose generated under certain conditions; The mean square error between the two forces the prediction to remain consistent before and after the intervention.

[0084] The optimization objective for the diffusion generation stage is: , In the formula, ( The overall training objective of the diffusion generation phase; Standard spread noise prediction loss; : Weights of diffusion loss; : Weight of causal consistency loss.

[0085] This phased design prevents gradient conflicts between discrete graph structure learning and continuous denoising generation, while allowing the learned causal representations to guide the backdiffusion process. Overall, the first stage learns stable and causally faithful structures, while the second stage learns expressive generative models conditioned on these fixed structures.

[0086] Based on the above method, this invention provides a human posture prediction system for executing the aforementioned human posture prediction method, comprising: a motion pattern decomposition module, a noise encoder module, a causal learning and counterfactual refinement module, a dynamic graph fusion module, and a conditional diffusion generation module. Specifically, the motion pattern decomposition module decomposes historical human posture sequences into multiple potential motion patterns using a Dirichlet process mixture model; the noise encoder module extracts exogenous variables representing potential contextual information from the historical human posture sequences; the causal learning and counterfactual refinement module, for each motion pattern, learns and optimizes multiple refined causal directed acyclic graphs (DAGs) by combining the exogenous variables; the dynamic graph fusion module combines the multiple refined DAGs into a unified dynamic causal graph representation using an attention-based weighted fusion mechanism; and the conditional diffusion generation module, using the dynamic causal graph representation as structured conditions, injects it into each step of the backdiffusion process of the denoising diffusion implicit model through a cross-attention mechanism and time-aware feature modulation, iteratively denoising and generating future human posture sequences.

[0087] Compared with the prior art, the present invention has the following six significant technical effects.

[0088] First, it significantly improves the accuracy of long-term human pose prediction. Extensive experiments on four public benchmark datasets—Human3.6M, AMASS, 3DPW, and HuMMan—demonstrate that the present invention reduces the mean joint position error (MPJPE) by approximately 5.62% and the foot slip error (FSE) by approximately 9.10% in the long-term prediction domain compared to state-of-the-art methods. The performance advantage is particularly pronounced in the ultra-long prediction time domain. For example, on Human3.6M, the MPJPE at 2000ms is 216.62mm, a reduction of approximately 2.6% compared to the second-best method, ALIEN's, 222.48mm. The accuracy advantage continues to expand as the prediction time domain lengthens.

[0089] Second, it significantly improves the physical plausibility and visual realism of the generated poses. The substantial reduction in foot sliding error indicates that this invention effectively suppresses common artifacts caused by unreasonable foot sliding on the ground. Qualitative visualization results on the AMASS ACCAD subset, 3DPW, and HuMMan datasets show that in complex actions such as walking and putting down a box, the pose sequences generated by this invention are significantly superior to existing baseline methods in terms of joint coherence, kinematic constraint preservation, and fidelity of end-body details, especially after 700ms.

[0090] Third, it achieves dynamic adaptability and interpretability of the causal structure. By automatically discovering motion patterns and learning a dedicated causal graph for each pattern through DPM, the model can clearly reveal the driving relationships between key joints in different motion stages, providing an interpretable technical foundation for understanding model decision-making, fault diagnosis, and verification in safety-critical scenarios.

[0091] Fourth, the computational efficiency is competitive. Although causal learning and diffusion generation are introduced, global graph learning is decomposed into local subgraph learning through sliding window decomposition (L=8), keeping the number of nodes in each subgraph within a manageable range of N×L, making NOTEARS optimization tractable. As shown in Table 3, the inference latency is 148.1ms, the number of parameters is 28.3M, and the GPU memory usage is 7.4GB, all of which remain at reasonable levels. The accuracy far exceeds that of lightweight non-diffusion methods, and the inference efficiency is better than some diffusion baselines.

[0092] Fifth, it exhibits strong generalization ability across datasets. Whether in indoor controlled environments (Human3.6M), large-scale diverse motion capture (AMASS), outdoor real-world scenarios (3DPW), or large-scale complex datasets containing 1000 subjects, 500 actions, and 400K sequences, the various metrics of this invention outperform most existing methods, demonstrating the good versatility of the dynamic causal graph combination mechanism.

[0093] Sixth, it pioneers a new technological direction for mechanism-aware generative modeling. This invention combines dynamic causal graphs with conditional diffusion models for human pose prediction, promoting a paradigm shift from "purely statistical correlation-driven" to "causal mechanism awareness," and providing a referable technical path for future research integrating causal reasoning and generative artificial intelligence.

[0094] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for predicting human posture, characterized in that, Includes the following steps: Historical human posture sequences are collected and decomposed into multiple potential motion patterns. The historical human posture sequence is a series of observed human posture data, and each frame of posture consists of the position coordinates of multiple joints of the human body in three-dimensional space. Historical human pose sequences are input into a noise encoder to extract exogenous variables that represent potential contextual information; Potential contextual information includes motion intention, environmental context, and unobserved control signals. Environmental context refers to external conditions in the physical environment in which the observed human body is located that can affect the motion trajectory. For each motion pattern, a pattern-specific causal directed acyclic graph is learned by combining exogenous variables. The causal directed acyclic graph is then optimized by a counterfactual refinement strategy based on structural intervention, resulting in multiple refined causal directed acyclic graphs. The adjacency matrix and corresponding causal representation of each refined causal directed acyclic graph are mapped to key vector and query vector respectively. The attention weight of each mode-specific causal directed acyclic graph is calculated by scaling dot product attention mechanism. The attention weight is used as a weighting coefficient to linearly combine all mode-specific causal directed acyclic graphs to obtain dynamic causal graph. The dynamic causal graph is then flattened and projected onto the feature space of a preset dimension through a linear layer to obtain the dynamic causal graph representation. The dynamic causal graph representation is used as a structured condition. Through cross-attention mechanism and time-aware feature modulation, it is injected into each step of the back diffusion process of a denoising diffusion implicit model. With the historical human posture sequence as the condition, iterative denoising starts from random noise to generate the future human posture sequence.

2. The human posture prediction method according to claim 1, characterized in that, The method for determining the multiple potential motion patterns includes the following steps: The historical human posture sequence is divided into multiple overlapping short-time motion segments using a sliding window with a preset length and step size. A truncated approximation Dirichlet process mixture model is adopted, and a lightweight one-dimensional convolutional neural network is used as a global clustering network to assign each short-time motion segment to a corresponding latent motion pattern, wherein each latent motion pattern is characterized by a Gaussian distribution. The posterior probability of each segment belonging to each motion mode is determined by Bayes' formula, and negligible components are pruned according to mixed weights, ultimately resulting in multiple potential motion modes determined adaptively by the data.

3. The human posture prediction method according to claim 1, characterized in that, The method for extracting exogenous variables includes the following steps: Obtain multiple motion patterns determined after motion pattern decomposition and their corresponding segment allocation information; Based on the segment allocation information, extract the posture subsequence corresponding to the current motion mode from the historical human posture sequence; Flatten the posture subsequence into a one-dimensional vector; The one-dimensional vector is input to the noise encoder, which is a multilayer perceptron containing three fully connected layers, each followed by a ReLU activation function. The multilayer perceptron is used to encode the one-dimensional vector into low-dimensional exogenous variables and output the exogenous variables. The exogenous variables are used to characterize the latent context information, which includes motion intention, environmental context, and unobserved control signals. During the training phase, the noise encoder adds Gaussian noise sampled from a standard multivariate Gaussian distribution to the output of the multilayer perceptron to introduce randomness and improve generalization ability; during the inference phase, the Gaussian noise is not added to obtain deterministic output.

4. The human posture prediction method according to claim 1, characterized in that, Learning a pattern-specific causal directed acyclic graph involves the following steps: For the p Each motion pattern defines an adjacency matrix. To encode the causal structure of its directed acyclic graph, where non-zero elements Indicates the first j The variable for the first i Each variable exerts a direct causal effect; Each causal variable is modeled as a nonlinear function of its parent node variable and the corresponding exogenous input, and a continuous acyclic penalty is introduced to constrain each mode-specific graph to a directed acyclic graph. The continuous acyclic penalty is determined according to the following formula: ， in, H ( A P ) is the first p The continuous non-cyclic penalty value of a causal directed acyclic graph of a motion pattern; No. p Adjacency matrix of each motion pattern; Represents the Hadamard product; d The variable dimension is the total number of nodes in the causal graph. , N Indicates the number of joints in the human body. T This indicates the length of the historical human posture sequence.

5. The human posture prediction method according to claim 4, characterized in that, The causal directed acyclic graph is optimized using a counterfactual refinement strategy based on structural intervention, specifically including: For a given number p A pattern-specific structural causal model, applying an intervention to the target variable that sets it to a constant value; Based on the topological order of the causal directed acyclic graph, the intervention effect is propagated through a graph neural network, and the counterfactual value of each variable is calculated. Define descendant loss and non-descendant loss, wherein the descendant loss is used to ensure that descendant nodes should change under intervention, and the non-descendant loss is used to ensure that non-descendant nodes should remain unchanged under intervention, and construct counterfactual loss based on the weighted sum of the descendant loss and non-descendant loss; The causal directed acyclic graph is refined with the goal of minimizing the counterfactual loss function.

6. The human posture prediction method according to claim 1, characterized in that, The attention weights of the pattern-specific causal directed acyclic graph are determined according to the following formula: ， In the formula, Attention weights for pattern-specific causal directed acyclic graphs; For the first p The query vector corresponding to a causal directed acyclic graph; For the first p The key vectors corresponding to each causal directed acyclic graph; The scaling factor for stabilizing the gradient. For the attention dimension.

7. The human posture prediction method according to claim 1, characterized in that, A method for generating future human pose sequences by iteratively denoising random noise includes the following steps: The historical human posture sequence is encoded as historical motion features, the dynamic causal graph representation is projected onto the same feature dimension, and the two are spliced ​​together to form conditional memory. Sample initial noisy motion features from standard Gaussian noise; For each back-diffusion step within the total number of steps, perform the following: The noisy motion features of the current step are used as a query and cross-attention calculation is performed with the conditional memory to obtain the motion features of causal perception. A sinusoidal time-step embedding is generated based on the current diffusion step number. The sinusoidal time-step embedding is mapped to scaling and shift parameters through a multilayer perceptron. The scaling and shift parameters are then used to modulate the motion features of the causal perception. The modulated features are input into the noise prediction network of the denoising diffusion implicit model to predict the noise of the current step, and the noisy motion features are updated according to the predicted noise to obtain the noisy motion features of the next step. Repeat the above back-diffusion steps until all T steps are completed, and decode the final noisy motion features into a future human posture sequence.

8. The human posture prediction method according to claim 1, characterized in that, The future human posture sequence is determined by future human posture features, which are determined according to the following formula: ， In the formula, For future human posture characteristics; Let be the noisy future motion feature at step t; For the spread noise scheduling coefficient; This is the cumulative noise dispatch coefficient; For noise prediction networks; X A sequence of historical human postures; Z Represented as a dynamic cause-effect graph; t This represents the current diffusion time step.

9. The human posture prediction method according to claim 1, characterized in that, It also includes an optimization strategy step, which comprises two stages: Phase 1: Causal discovery and counterfactual graph refinement phase, which involves jointly training the motion pattern decomposition, the exogenous variable extraction, and the dynamic causal graph fusion process. The pattern-specific causal directed acyclic graph is optimized using counterfactual loss and continuous noncyclic penalty. After convergence in this phase, the trained causal parameters are frozen. The second stage is the causal conditional diffusion generation stage. Under the guidance of the frozen causal correlation parameters, only the denoised diffusion implicit model is trained, and a causal consistency loss function is introduced. The causal consistency loss function is used to constrain the alignment between the predictions generated conditioned on the original dynamic causal graph representation and the counterfactual predictions generated conditioned on the dynamic causal graph representation after counterfactual intervention.

10. A human posture prediction system, characterized in that, include: The motion pattern decomposition module is used to decompose historical human posture sequences into multiple potential motion patterns using a Dirichlet process mixture model. The noise encoder module is used to extract exogenous variables representing potential contextual information from the historical human posture sequence; The causal learning and counterfactual refinement module is used to learn and optimize multiple refined causal directed acyclic graphs for each motion pattern, in conjunction with the exogenous variables. The dynamic graph fusion module is used to map the adjacency matrix and corresponding causal representation of each refined causal directed acyclic graph to key vectors and query vectors, respectively. It calculates the attention weight of each mode-specific causal directed acyclic graph through a scaling dot product attention mechanism. The attention weight is used as a weighting coefficient to linearly combine all mode-specific causal directed acyclic graphs to obtain a dynamic causal graph. The dynamic causal graph is then flattened and projected onto a high-dimensional space to obtain a dynamic causal graph representation. The conditional diffusion generation module is used to inject the structured conditions represented by the dynamic causal graph into each step of the denoising diffusion implicit model through cross-attention mechanism and time-aware feature modulation, and iteratively denoise to generate future human posture sequences.

Images