A human motion prediction method based on spectral-geometric joint representation

Abstract

The present application relates to human motion prediction, in particular to a human motion prediction method based on spectral-geometric joint representation, comprising: in the frequency domain, using a dynamic frequency denoising module to respectively denoise, node dimension feature normalize and space-time information extract the key node sequence and bone vector sequence converted to the frequency domain, respectively obtaining the frequency domain features of the key node sequence and the bone vector sequence, and taking the original representation corresponding to the bone vector sequence feature as a bone vector prediction sequence; obtaining the human skeleton key node sequence of continuous multiple frames, aggregating through a hyperbolic space-time perception enhancement module, and adding and fusing the aggregated features and the features processed by the dynamic frequency denoising module to obtain a joint node prediction sequence; and weighting and fusing the bone vector prediction sequence and the joint node prediction sequence to obtain the fusion position of each joint node. The present application can ensure that the denoising process does not destroy the action hierarchy structure, thereby significantly improving the accuracy and robustness of human motion prediction.

Description

Technical Field

[0001] This invention relates to human motion prediction, and in particular to a human motion prediction method based on spectral-geometric joint representation. Background Technology

[0002] Human Motion Prediction (HMP) is an important research area in computer vision and machine learning, aiming to predict future human behavior based on historical human motion sequences. HMP has wide applications in security monitoring, human-computer interaction, and autonomous driving, and is closely related to research on human motion recognition. Traditional methods typically preprocess historical human pose sequences and input them into a trained prediction model to learn deep representations of motion.

[0003] In motion prediction, joint trajectory prediction is often affected by accumulated noise, such as sensor errors, pose estimation errors, or small deviations propagating over time. Traditional denoising strategies, such as fixed-time smoothing or uniform filtering, tend to oversuppress motion details, resulting in the loss of key joint dynamic information. On the other hand, many hierarchical representation learning methods embed relational structures into Euclidean space, but the polynomial volume growth of Euclidean space can lead to geometric distortion of hierarchical data. Research shows that hyperbolic spaces with exponential volume growth are more suitable for preserving tree-like or hierarchical relationships, achieving more compact and realistic embeddings in multi-level semantic organization tasks.

[0004] However, most current hierarchical modeling methods have two limitations: First, most methods adopt graph convolution or attention architectures based on Euclidean assumptions, such as LTD, PGBIG, DDGCN, and GCNext. Although they enhance spatiotemporal modeling through residuals, dynamics, or multi-scale graphs, they are still limited to flat spaces, making it difficult to effectively represent hierarchical dependencies between joints. Furthermore, they exhibit oversmoothing of information and decreased distinguishability under structural or temporal changes. Second, existing hierarchical representations typically treat structural learning and noise suppression separately rather than as a unified process. Noise or estimation errors in lower-level joints may propagate upwards, weakening the semantic consistency of higher levels.

[0005] Furthermore, existing methods neglect the frequency domain noise remaining after DCT transformation. Unfiltered noise interferes with graph convolution feature aggregation, limiting prediction stability. In real-world motion data, noise such as sensor jitter, calibration drift, or subtle hand tremors is unavoidable. This noise introduces temporal inconsistencies and skeletal structure perturbations, thereby disrupting the hierarchical relationships of human movements. Traditional denoising methods, such as low-pass filters, use fixed parameters to suppress high-frequency noise, but they cannot dynamically adjust the filtering range to adapt to different signal characteristics, easily losing crucial motion information. Simultaneously, the hierarchical nature of human movements, including the interaction between local joint movements and whole-body dynamics, is also easily interfered with by the accumulation of low-frequency noise, impairing the integrity and accuracy of motion sequence features. Summary of the Invention

[0006] To address the problems existing in current technologies and improve the accuracy and robustness of human motion prediction, this invention proposes a human motion prediction method based on spectral-geometric joint representation. The process of human motion prediction includes:

[0007] The sequence of key human skeleton nodes and bone vector sequence in multiple consecutive frames are obtained and transformed into the frequency domain based on discrete cosine transform.

[0008] In the frequency domain, the dynamic frequency denoising module is used to denoise, normalize the node dimension features and extract spatiotemporal information on the features transformed to the frequency domain, respectively, to obtain the frequency domain features of the key node sequence and the bone vector sequence. Then, the inverse discrete cosine transform is used to transform them back to the original representation, and the original representation corresponding to the bone vector sequence features is used as the bone vector prediction sequence.

[0009] The sequence of key human skeleton nodes in multiple consecutive frames is obtained and aggregated by the hyperbolic spatiotemporal awareness enhancement module. The aggregated features are then fused with the features processed by the dynamic frequency denoising module to obtain the joint prediction sequence.

[0010] The bone vector prediction sequence and the joint prediction sequence are weighted and fused to obtain the fused position of each joint.

[0011] This invention proposes a Spectral-Geometric Representation Network (SGRNet), which combines dynamic frequency domain denoising and hierarchical relationship modeling within a unified framework. The network consists of two parts: a Dynamic Frequency Denoising (DFD) module and a Hyperbolic Spatiotemporal Awareness Enhancement (HSAE) module. The DFD module suppresses frequency domain noise through dynamic threshold adjustment while preserving key frequency features through global and local filtering. The HSAE module utilizes negative curvature hyperbolic geometry to mitigate the exponential distance expansion problem of long-range information propagation and designs a learnable K-adjacency matrix to enhance local joint dependencies and overall skeletal consistency. The DFD and HSAE modules work synergistically to ensure that the denoising process does not disrupt the hierarchical structure of movements, thereby significantly improving the accuracy and robustness of human motion prediction. Unlike existing technologies, this invention learns pose features simultaneously in the frequency domain and hyperbolic space: in the frequency domain, the denoising level is dynamically adjusted to reduce the impact of noise; in hyperbolic space, a three-dimensional spatiotemporal adjacency graph is constructed based on HSAE to further enhance model robustness and predictive performance. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the framework of a human motion prediction method based on spectral-geometric joint representation according to the present invention;

[0013] Figure 2 This is a schematic diagram of the fusion module structure of the present invention;

[0014] Figure 3 This is a schematic diagram of the framework structure of the dynamic frequency denoising module of the present invention;

[0015] Figure 4 A schematic diagram of the x-axis coordinate trajectory of Joint5 (left toe) for two different movements;

[0016] Figure 5 This is a schematic diagram of the framework structure of the hyperbolic spatiotemporal perception enhancement module of the present invention;

[0017] Figure 6 This is a schematic diagram of the shell-based sampling strategy in the Poincaré sphere of the present invention;

[0018] Figure 7 This is a visual comparison of the ablation experiments using the DFD module of this invention;

[0019] Figure 8 Visualization of the general prediction results for the "walking" action on the Human3.6M dataset;

[0020] Figure 9 These are the key analytical results of the method of this invention in ablation experiments;

[0021] Figure 10This is a schematic diagram of the elements in the i-th row and j-th column of the local multi-scale spatiotemporal adjacency matrix of this invention, excluding k=0. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] This invention proposes a human motion prediction method based on spectral-geometric joint representation, specifically including:

[0024] The sequence of key human skeleton nodes and bone vector sequence in multiple consecutive frames are obtained and transformed into the frequency domain based on discrete cosine transform.

[0025] In the frequency domain, the dynamic frequency denoising module is used to denoise, normalize the node dimension features and extract spatiotemporal information on the features transformed to the frequency domain, respectively, to obtain the frequency domain features of the key node sequence and the bone vector sequence. Then, the inverse discrete cosine transform is used to transform them back to the original representation, and the original representation corresponding to the bone vector sequence features is used as the bone vector prediction sequence.

[0026] The sequence of key human skeleton nodes in multiple consecutive frames is obtained and aggregated by the hyperbolic spatiotemporal awareness enhancement module. The aggregated features are then fused with the features processed by the dynamic frequency denoising module to obtain the joint prediction sequence.

[0027] The bone vector prediction sequence and the joint prediction sequence are weighted and fused to obtain the fused position of each joint.

[0028] This invention aims to address two major problems in existing human motion prediction methods: first, insufficient frequency domain denoising results in noise remaining in low-frequency components, thus affecting the accuracy of motion prediction; second, most existing graph convolutional networks (GCNs) are based on Euclidean space, making it difficult to effectively capture the hierarchical relationships between human joints, especially under complex spatiotemporal variations. To address these issues, this invention proposes a spectral geometric representation network (SGRNet). It introduces a dynamic frequency denoising (DFD) module to dynamically adjust the denoising threshold, suppressing noise in the frequency domain while preserving important motion information. Furthermore, it employs a hyperbolic spatiotemporal enhancement (HSAE) module, utilizing the negative curvature of hyperbolic space to more effectively model the spatiotemporal relationships of human joints, overcoming the limitations of Euclidean space in handling complex hierarchical structures. Through this approach, this invention significantly improves the accuracy and robustness of human motion prediction, demonstrating superior performance in long-step prediction tasks.

[0029] like Figure 1 As shown, this invention proposes a motion prediction network framework based on spectral geometric representation. This network framework consists of multiple functional modules, mainly including joint branch, bone vector branch, and fusion module. These modules work together to achieve higher accuracy and robustness in motion prediction. The specific implementation process includes the following stages:

[0030] (I) First stage:

[0031] In the keypoint branch, the input human motion sequence is first encoded through the frequency domain branch. The frequency domain branch uses Discrete Cosine Transform (DCT) to convert the input time series into a frequency domain representation. The specific formula is as follows:

[0032]

[0033] in, It is the discrete cosine transform matrix. This represents the input motion sequence. This is a frequency domain representation. Subsequently, multiple Dynamic Frequency Denoising (DFD) modules perform frequency domain denoising, node feature normalization, and spatiotemporal information extraction. The DFD module adaptively adjusts the denoising intensity for different motion modes using dynamically learned denoising thresholds to effectively remove low-frequency noise while preserving key motion signals. The key formulas are as follows:

[0034]

[0035] in, It is a binary mask matrix. It is the dynamic noise reduction threshold. and These are global and local learning filters, respectively. Using the above formulas, the DFD module preserves frequency information while denoising.

[0036] Meanwhile, the hyperbolic branch of the joint branch uses the Hyperbolic Spatiotemporal-Aware Enhancement (HSAE) module to aggregate joint information in hyperbolic space. Hyperbolic space, with its negative curvature properties, can better capture the hierarchical relationships in the joint position sequence. This makes hyperbolic space more suitable than Euclidean space for processing hierarchical structural information between human joints.

[0037] As an optional implementation, this embodiment provides a method for constructing a discrete cosine transform matrix. If the number of observation frames is T, the sequence of key human skeletal nodes or bone vector sequences is transformed to the frequency domain based on the discrete cosine transform matrix. The elements of the discrete cosine transform matrix are represented as follows:

[0038]

[0039] in, Let represent the element in the i-th row and j-th column of the discrete cosine transform matrix, where ; Let Kronecker function be the function of i = j. ,otherwise .

[0040] (II) Second Stage:

[0041] Unlike joint branching, bone vector branching uses only frequency domain branches for motion sequence encoding. Bone vectors represent the linear relationships between joints and lack a clear hierarchical structure; therefore, bone vector branching does not have hyperbolic branches and focuses on modeling using frequency domain branches. The frequency domain encoding formula is the same as that of joint branching:

[0042]

[0043] in, This represents the input bone vector sequence. This is the frequency domain representation of the bone vector. After frequency domain processing, the bone vector branch is processed using only multiple dynamic frequency denoising modules, and the features obtained after denoising are used to generate bone vector predictions.

[0044] (III) Third Stage:

[0045] like Figure 2 The fusion module generates separate prediction outputs for the joint point branches and bone vector branches. These outputs are then fused to obtain the final motion prediction result. This invention is applicable to any skeleton structure. The three-dimensional shape of each joint is used as the feature of that node. A joint point is selected as the starting position, and the prediction extends from this starting position to the joints of the limbs and head. The bone vector in this invention represents the direction from one joint point to another. The fusion process is shown in the formula:

[0046]

[0047] in, This represents the sequence of fused joint positions in the (T+1)th frame, predicted based on T observation frames. This is the predicted sequence of key points for frame T+1; The bone vector prediction sequence for frame T+1; These are learnable weights used to balance the contributions between joint outputs and bone vector outputs.

[0048] Apart from the keypoints at the starting position, the prediction result of the current node is corrected using the prediction result of the parent node of the current keypoint, as shown below:

[0049]

[0050] in, This indicates the corrected position of the k-th joint. This represents the predicted value of the k-th bone vector in the bone vector prediction sequence of frame T+1; This represents the predicted value of the parent node of the k-th joint in the joint prediction sequence of frame T+1. Let represent the fusion position of the parent node of the k-th joint in the joint fusion position sequence of the (T+1)-th frame predicted based on T observation frames; This represents the predicted value of the k-th joint in the joint prediction sequence of frame T+1. , and its dual weights , It is a learnable parameter used to dynamically adjust the fusion ratio.

[0051] by Figure 2 For example, after fusing all the joints to obtain their 3D coordinates, the coordinates of the starting joint are first determined. This joint has no parent node and cannot be corrected. For the joint with the number 8, if its fused joint position is J8 and its bone vector is B8 (this bone vector points from its parent node to the current node), then the final coordinates obtained based on the parent node position correction are F8. For the joint with the number 9, its parent node is the joint with the number 8, its fused joint position is J9 and its bone vector is B9 (this bone vector points from the joint with the number 8 to the joint with the number 9), then its final coordinates obtained based on the parent node's final coordinates F8 are F9.

[0052] Through the fusion process of the above two stages, a more accurate motion prediction result is finally obtained, which fully combines the features of joints and bone vector branches, thereby achieving accurate prediction of future posture.

[0053] Next, this embodiment provides a specific implementation process for a dynamic frequency denoising module to process data.

[0054] Due to factors such as sensor inaccuracy, obstruction, or small-amplitude movements, motion sequences may inherently contain noise. When these noise signals are transformed to the frequency domain, the noise may affect the distribution of coefficients.

[0055] Starting with the formula for the DCT transform, we analyze possible noise sources. Given a discrete pose sequence with additional noise... The observed sequence is Its DCT coefficients are:

[0056]

[0057] in These are real signal components. It is the noise term in the frequency domain:

[0058]

[0059] Due to the characteristics of DCT, the energy of high-frequency components tends to approach zero, and the relative contribution of noise projected into the low-frequency band may become non-negligible. That is, noise in human motion sequences can be reflected in the low-frequency region after DCT transformation, which may reduce prediction accuracy. However, the existence of noise cannot be objectively and quantitatively determined solely from the derivation of formulas and theorems.

[0060] Therefore, in addition to theoretical derivation, this embodiment also performs noise analysis on the Human3.6M dataset. Two typical types of real noise were observed: (1) sensor jitter, characterized by zero-mean frame-level perturbation and weak inter-joint consistency; (2) calibration drift, manifested as low-frequency deviation inconsistent with pelvic movement or global trajectory. Figure 4 The image shows the x-axis coordinate trajectories of Joint 5 (left toe) for two different movements: Walking Together and Smoking. The left plot depicts the time evolution (red: original signal, blue: smooth curve), while the right plot shows the amplitude spectrum of the Fast Fourier Transform (FFT).

[0061] For WalkingTogether, the first 25 frames exhibit noticeable sensor jitter around frame 10, followed by a sharp jump around frame 25. The corresponding spectrum shows a strong low-frequency energy peak (0–0.05 Hz) indicating calibration drift, accompanied by several mid-frequency peaks (0.05–0.15 Hz) representing local oscillations.

[0062] In contrast, the Smoking sequence showed only slight short-term sensor jitter, but significant calibration drift was present. The time trajectory remained smooth, but shifted slowly after frame 20, and the FFT spectrum was dominated by extremely low-frequency components (<0.03 Hz), confirming long-term bias and calibration drift. These observations collectively validate the coexistence of sensor jitter and calibration drift in real motion capture data.

[0063] like Figure 3This invention proposes a Dynamic Frequency Denoising (DFD) module for adaptive noise suppression and spectral feature enhancement modeling of temporal representations of human motion sequences in the frequency domain. The module uses Discrete Cosine Transform (DCT) as the basis for temporal coding, mapping the input human pose sequence to the frequency domain representation space. While maintaining the original feature dimensional structure, it explicitly characterizes the frequency composition information of human motion. The frequency domain features after DCT transformation typically exhibit energy concentration in the low-frequency region. Based on this, the DFD module further constructs a dynamic frequency filtering mechanism based on a learnable threshold, addressing noise frequency components introduced during motion modeling due to perturbations, jitter, or local anomalies from the perspective of spectral distribution. Simultaneously, the module combines a global spectral branch and a local denoised spectral branch to achieve joint modeling of the original and purified frequency domain information, thereby improving the temporal representation's ability to characterize key motion patterns and its robustness.

[0064] Specifically, in this module, the input human pose time sequence is first assumed to be represented in the frequency domain after undergoing discrete cosine transform. ,in, This represents the spectral coefficient distribution of the original time-series signal in the frequency domain, encompassing the response intensity of human motion at different frequency dimensions. To accurately describe the energy magnitude of each frequency component, a spectral power matrix P is further constructed, defined as:

[0065]

[0066] in, This is used to characterize the energy intensity of spectral coefficients at various locations in the frequency domain representation. Since effective human motion patterns typically correspond to relatively stable dominant frequency components, while noise disturbances often exhibit irrelevant or anomalous spectral responses, this invention introduces a learnable threshold. For the power spectrum The frequency components are dynamically identified to adaptively distinguish between effective motion frequencies and noise frequencies.

[0067] Furthermore, by using the spectral power matrix With learnable threshold Perform element-by-element comparisons to generate the corresponding binary mask matrix. Its definition is as follows:

[0068]

[0069] in, This represents a frequency domain filtering mask used to retain frequency components with energy above a threshold and suppress lower-energy noise responses. The threshold is... During model training, adaptive updates are performed through backpropagation of the loss function, which enables dynamic adjustment of the screening boundary based on the spectral statistical characteristics of different motion modes, giving the module the ability to perform frequency domain denoising for different motion types.

[0070] Obtaining the mask matrix Then, it is combined with the original frequency domain representation. Element-wise multiplication yields the denoised frequency domain representation. Specifically, it is expressed as:

[0071]

[0072] Among them, symbols This represents element-wise multiplication. Through the above operation, the frequency responses below the threshold in the original frequency domain representation are effectively weakened, while the main motion frequencies above the threshold are preserved, thus achieving dynamic filtering of frequency domain noise components. Unlike traditional static frequency truncation strategies, this denoising process does not rely on a preset fixed frequency band for hard pruning, but rather adaptively determines the effective frequency region based on the spectral characteristics of the input action, thus enabling more flexible adaptation to the temporal variation patterns of different action sequences.

[0073] To further enhance the complementary expressive ability of spectral information before and after denoising, this invention designs a global spectrum modeling branch and a local denoised spectrum modeling branch based on dynamic frequency selection. Specifically, on the one hand, the original frequency domain representation... Input is fed into the global filtering branch, and processed through learnable global filtering parameters. We perform weighted modeling to extract overall motion structure information in the complete frequency spectrum space; on the other hand, we dynamically denoise the frequency domain representation. Input is fed into the local filtering branch, and learned local filtering parameters are used. Weighted modeling is applied to enhance the representation of key dominant frequency components and denoised local spectral responses. Finally, the outputs of the two branches are fused to obtain a comprehensive spectral representation. Its expression is:

[0074]

[0075] in, This represents the global filter weights applied to the original frequency domain representation. This represents the local filtering weights applied to the denoised frequency domain representation.

[0076] In addition, such as Figure 3After fusing the original spectral branch and the denoised spectral branch, the resulting features are normalized and reorganized. Then, using a residual connection method, the enhanced frequency domain representation is superimposed on the input frequency domain features. This preserves the basic information of the original time-series spectrum while further improving the stability and integrity of the frequency domain representation. Compared to traditional single-path frequency domain filtering methods, this invention, by simultaneously preserving the original spectral representation and the denoised spectral representation and performing joint modeling, effectively alleviates the information abruptness and spectral discontinuity problems caused by spectrum truncation. This suppresses noise interference, enhances the frequency response of key actions, and improves the robustness and discriminative ability of human motion time-series representation.

[0077] To verify the necessity and effectiveness of this module, this invention selected different denoising schemes. In signal processing, frequency domain denoising is a widely used noise suppression strategy. For example, classic examples such as Kalman filtering and Wiener filtering suppress noise by assuming linear or stationary statistical models and adjusting the filter coefficients accordingly. These techniques are effective under stable noise distributions, but may struggle to cope with the inherent complex spectral variations of human motion. As shown in Table 1, this invention designed a control experiment with clean motion sequences injected with synthetic noise, as follows:

[0078] 1. Frequency Domain Noise Injection. First, use DCT to transform the clean sequence to the frequency domain, then use... Gaussian noise is added to simulate high-frequency jitter or irregular spectral fluctuations. Narrowband spikes are also included (by injecting local spectral perturbations at selected frequencies).

[0079] 2. Temporal perturbation. The clean sequence in the temporal domain is directly modified by (i) adding a random offset of 3px to the selected joints (simulating bone estimation error) and (ii) repeating for 5 consecutive frames (simulating temporal redundancy).

[0080] Table 1. Robustness test of waiting sequences under extreme noise.

[0081]

[0082] The results consistently demonstrate that the proposed DFD module mitigates frequency and time domain perturbations, reducing MPJPE by a significant margin compared to the baseline without DFD. This verifies that DFD effectively suppresses noise energy under well-defined noise conditions while preserving meaningful motion signals.

[0083] Next, this embodiment provides a specific implementation process for a hyperbolic spatiotemporal perception enhancement module to process data.

[0084] The objective of this invention is to determine the effectiveness of hyperbolic space in the field of human motion prediction. The designed experiments include both quantitative and qualitative analyses. The following two formulas are used to measure the pairwise distance reconstruction error (Stress) between the original space and the embedded space, and to scalarize the degree of preservation of local pairwise distances after embedding (Distortion), respectively.

[0085]

[0086]

[0087] in and These represent the pairwise distances of the original and the embedded, respectively. Lower values ​​indicate that the embedded more faithfully preserves the global geometry. The distortion value is the number of points; a smaller distortion value indicates better local geometric consistency between the embedded manifold and the original manifold. This invention evaluates eight representative actions of Human3.6M (walking, walking together, smoking, making a phone call, eating, greeting, waiting, and sitting). The shape of each input sequence is... This represents 128 sequences, 50 frames, and 22 coordinates. The results, shown in Table 2, quantitatively validate that hyperbolic space preserves distance geometry and skeletal hierarchy more effectively than Euclidean space.

[0088] Table 2. Quantitative comparison of Euclidean and hyperbolic embeddings under eight typical actions.

[0089]

[0090] The metrics for Stress and Distortion analyze the original space, the embedded space, and the degree of length preservation after embedding. A more intuitive way to demonstrate the advantages of hyperbolic embedding is needed. The experimental design begins with the root node of the skeleton (pelvis), dividing the joints into concentric shells based on the graphical distance from the joint to the root node (shortest jump depth). Based on the distance between concentric shells, the sampling from positive to negative samples can be represented. )like Figure 6 As shown, for clear comparison, based on A negative sampling mechanism is designed to divide the region into several areas before conducting quantitative experiments. Represents the sampling range; (limit): ; (Medium to long distance): ; (Medium-to-medium distance): ; (Near the root): ; (Uniform distance): From Uniform sampling ; : from the same depth but different kinematic branches .

[0091] Distance distributions under Euclidean and Poincaré metrics were calculated, and separability was assessed using the scale-invariant indices Cohen'd and ROC-AUC (Receiver Operating Characteristic-AOC) to ensure comparability across spaces. Each experiment was repeated five times with a random seed, and the mean ± standard deviation is reported. As shown in Table 3, separability gradually decreases as the negative shell moves inward (R2–R4), which is expected due to the decreasing depth contrast. Crucially, hyperbolic embeddings maintain significantly higher margins across all mechanisms, not only in the extreme S1 vs. Sm cases, but also in intermediate (R2–R3) and structurally challenging (R6) conditions. This robustness stems from the exponential expansion of Poincaré geometry near the boundaries: even small Euclidean differences in deeper shells correspond to disproportionately larger geodesic distances, as shown in the following formula:

[0092]

[0093] This amplifies the hierarchical differences that Euclidean geometry tends to compress. Therefore, hyperbolic space provides a better geometric prior for modeling the tree-like kinematic dependencies of human motion, preserving long-range relationships and enhancing structural discriminability beyond shallow contrasts.

[0094] Table 3 Separability under different negative sample sampling mechanisms

[0095]

[0096] like Figure 5 This invention proposes a Hyperbolic Spatiotemporal-Aware Enhancement (HSAE) module for jointly modeling the hierarchical structure and spatiotemporal dynamic dependencies of joint sequences during human motion prediction. Based on the Poincaré Ball Model, this module constructs a hyperbolic feature representation space. It utilizes the negative curvature of the hyperbolic space to characterize the naturally occurring tree-like topology, hierarchical dependencies, and non-Euclidean geometric features of the human skeleton. By combining a sliding time window mechanism and a multi-order adjacency aggregation strategy, it achieves synergistic enhancement of spatial correlation and temporal evolution information within local time intervals of the observed sequence, thereby improving the structural modeling capability of human motion sequence representation and the accuracy of future posture prediction.

[0097] Specifically, this invention first employs the Poincaré sphere model. As a hyperbolic representation space, it is defined as:

[0098]

[0099] in, The constant negative curvature parameter represents hyperbolic space. The hyperbolic space represents the Euclidean norm. Compared to Euclidean space, hyperbolic space can express data structures with hierarchical and long-range dependencies with lower distortion, making it more suitable for representing parent-child dependencies, local driving patterns, and overall topological structures of the human skeleton. In this space, geometric operations on node features require transformations between hyperbolic and tangent spaces using exponential and logarithmic mappings to ensure that linear transformations, neighborhood aggregations, and nonlinear activations can be performed while maintaining hyperbolic geometric consistency.

[0100] In the feature mapping stage, let the first... Layer input features are First through the origin The exponential mapping at a given point projects the Euclidean space features onto the hyperbolic space, yielding the corresponding hyperbolic feature representation. ,Right now:

[0101]

[0102] in, This represents the exponential mapping operation from tangent space to hyperbolic space. Through this mapping, the original human pose features are transformed from Euclidean representation to feature representation that satisfies hyperbolic geometric constraints, thus providing a foundation for subsequent hyperbolic graph convolution operations.

[0103] In graph structure modeling, considering that traditional methods typically construct spatial adjacency matrices based solely on the natural connectivity of a single-frame skeleton and simply stack them along the time dimension to form a discrete temporal graph, which is insufficient to fully express the dynamic interaction relationships between joints across time intervals, this invention further extends single-frame spatial adjacency relationships to spatiotemporal joint adjacency modeling. Specifically, let the single-frame skeleton adjacency matrix be... ,in, This represents the number of joints; it is further extended to a three-dimensional spatiotemporal adjacency matrix covering all observation frames. ,in, Indicates the length of the observation sequence. Indicates the first Frame and the Joint connections between frames. This construction allows the model to represent the spatial associations and temporal dependencies between different time frames within a unified graph structure. Three-dimensional spatiotemporal adjacency matrix. Represented as:

[0104]

[0105]

[0106]

[0107] Furthermore, in the spatial dimension, to avoid weakening the expressive power of local key motion patterns by uniformly processing joint dependencies at different distances, this invention introduces a K-order adjacency matrix decomposition mechanism to hierarchically divide spatial adjacency relationships according to the number of graph hops. Specifically, for any node... and If its shortest path distance Then the corresponding Adjacency matrix The Middle If an element is 1, it is 0; otherwise, it is 1. Take as the identity matrix This is used to preserve node self-connections. A multi-scale structure is constructed by concatenating the adjacency submatrices corresponding to different hop counts. adjacency matrix In this structure, the low-order adjacency submatrix mainly describes the fine-grained dependencies between local joints, while the high-order adjacency submatrix is ​​used to model the global structural connections between distant joints, thereby achieving a joint expression of the local motion drive and overall linkage relationship of the human skeleton.

[0108] By combining the time sliding window mechanism and the spatial K-order adjacency decomposition mechanism, this invention further constructs the final local multi-scale spatiotemporal adjacency matrix. In this matrix, the elements in the i-th row and j-th column represent the adjacency matrices of each node at different orders, such as... Figure 10 Then, normalization is performed on each order to obtain the normalized spatiotemporal adjacency matrix. When performing hyperbolic graph convolution operations on neighborhood nodes of each order within the window, the hyperbolic graph convolution operation includes cascaded multi-layer operations. The operations of the (l+1)th layer include:

[0109] The window features of the previous layer, i.e., the l-th layer of Euclidean space. By projecting the exponential map onto hyperbolic space, we obtain... ,Right now:

[0110]

[0111] In this embodiment, the coordinates of each joint point within the window are used as the window features of the first layer of hyperbolic graph convolution, and the output of the previous layer of hyperbolic graph convolution is used as the window features of the current layer of hyperbolic graph convolution.

[0112] Based on this, the module performs hyperbolic graph convolution operations on the neighborhood nodes of each order within the window, and then... Layer feature representation is defined as:

[0113]

[0114] in, This represents the Möbius addition operation in hyperbolic space. Represents the hyperbolic activation function, and the aggregation transformation term of the l-th layer. Its definition is:

[0115]

[0116] in, This represents a logarithmic mapping from hyperbolic space to tangent space; Let be the weight matrix of the l-th layer in the hyperbolic graph convolution. Let be the bias vector of the l-th layer in the hyperbolic graph convolution, and be the learnable parameters; Let represent the normalized local spatiotemporal neighbor matrix with hop count k; This represents the Möbius matrix multiplication operation. Specifically, the hyperbolic features are first transformed to the tangent space through a logarithmic mapping. Within the tangent space, a linear transformation and weighted aggregation based on the normalized adjacency matrix are performed. Then, the aggregation result is remapped back to the hyperbolic space through an exponential mapping. Finally, the hyperbolic activation function and residual Möbius addition are combined to update the output features of the current layer. Through this process, the module can achieve the cooperative propagation of multi-scale spatial dependencies and local temporal dynamics under hyperbolic geometric constraints.

[0117] Compared to traditional Euclidean graph convolution methods that only perform feature propagation within a fixed spatial topology, the HSAE module proposed in this invention simultaneously introduces a hyperbolic representation space, a sliding time window mechanism, and a multi-order adjacency aggregation strategy. This enables the model to more accurately describe the hierarchical dependencies between different joints, local action-driven patterns, and cross-time dynamic interactions during human movement. Specifically, the sliding window mechanism enhances the ability to model continuous dynamics within local time intervals, the K-order adjacency matrix strengthens the expressive power of multi-scale spatial relationships, and the hyperbolic space representation improves the fitting ability to skeleton tree structures and nonlinear topological relationships. Through these joint modeling mechanisms, the HSAE module effectively enhances the spatiotemporal representation of human motion sequences, improves the effectiveness of node information aggregation and propagation, and thus achieves more accurate predictions of future human postures.

[0118] This invention uses Mean Per Joint Position Error (MPJPE) as an evaluation metric in human motion prediction tasks to measure the deviation of joint positions between the predicted pose and the actual pose. MPJPE is a commonly used performance evaluation standard in the field of human motion prediction, and it can intuitively reflect the error level of the model in 3D human joint position prediction. To ensure fairness in the comparison between different methods, this invention refers to two commonly used MPJPE calculation methods in existing research and reports them independently: the first method only counts the error of each prediction frame within the prediction time range, that is, from the first frame... Frame to the The prediction error of the first frame is denoted as MPJPE (TestFrames); the second method simultaneously calculates the overall time range error, including historical observation frames and future prediction frames, i.e., from the first frame... Frame to the The total error of a frame is denoted as MPJPE (All Frames).

[0119] Specifically, in the above calculation process, M represents the total number of human joints, and M represents the number of observation frames, i.e., the length of the historical human motion sequence input to the model. This represents the number of prediction frames, i.e., the length of the future time steps the model needs to predict. Indicates the first Frame number Predicted 3D position of each joint Indicates the corresponding actual three-dimensional position. This represents the squared Euclidean distance between the predicted and actual joint positions, used to characterize the prediction error of the joint at the current time step. Based on this, MPJPE (Test Frames) can be represented as:

[0120]

[0121] MPJPE (All Frames) can be represented as:

[0122]

[0123] Among them, MPJPE is mainly used to measure the model's actual predictive ability within the future prediction interval, and can more directly reflect the model's extrapolation performance for unknown human motion states; while MPJPE comprehensively considers the overall error between historical observation frames and future prediction frames, and can reflect the consistency of the model's overall reconstruction and prediction of human motion sequences from the perspective of the entire sequence. By using the above two evaluation methods simultaneously, this invention can more comprehensively evaluate the performance of the proposed method in short-term prediction, medium- and long-term prediction, and overall sequence modeling, thus providing a unified basis for objective comparison between different methods.

[0124] Table 4 shows the performance comparison results of the method of this invention and various existing human motion prediction methods on the Human3.6M dataset. This table uses both MPJPE and MPJPE evaluation methods to statistically analyze the prediction errors of different methods at multiple prediction time steps, comprehensively reflecting the performance of each method in short-term and medium-to-long-term human motion prediction tasks.

[0125] As shown in Table 4, on the Human3.6M dataset, the prediction errors of all comparative methods generally show a gradual upward trend as the prediction time step increases, indicating that the difficulty of human motion dynamic modeling increases significantly with the extension of the prediction time domain. Under the MPJPE (All Frames) evaluation mode, the method of this invention achieved prediction errors of 9.0, 20.5, 44.2, 55.0, 73.8, 88.1, 100.2, and 108.1 at 80ms, 160ms, 320ms, 400ms, 560ms, 720ms, 880ms, and 1000ms, respectively. Compared with existing comparative methods, it achieved better results at most time steps, especially demonstrating stronger error control capabilities in the medium- and long-term prediction stages. Under the MPJPE (Test Frames) evaluation method, the proposed method achieved prediction scores of 6.6, 12.1, 23.8, 29.5, 40.2, 49.7, 58.1, and 63.8, respectively. It outperformed existing comparative methods at multiple prediction times starting at 320ms, further demonstrating the strong stability and superiority of the proposed method under different evaluation criteria. These results indicate that the proposed method can more effectively model the spatiotemporal dependencies in human motion sequences, achieving higher accuracy in predicting future human poses on the Human3.6M dataset.

[0126] Table 4 Experimental results on the Human 3.6M dataset

[0127] A suffix "†" indicates MPJPE (TEST FRAMES); otherwise, it indicates MPJPE (ALL FRAMES).

[0128] Table 5 shows the performance comparison results of the method of this invention and various existing human motion prediction methods on the AMASS-BMLrub dataset. The table uses MPJPE as the evaluation metric, and statistically analyzes the human motion prediction errors of different methods at multiple prediction time steps to evaluate the performance of each method under different prediction duration conditions.

[0129] As shown in Table 5, the prediction errors of all compared methods generally increase with the increase of the prediction time step, indicating that human motion prediction is more difficult to model in the medium to long term. On the AMASS-BMLrub dataset, the method of this invention achieved the best results at multiple prediction time steps of 160ms, 320ms, 400ms, 560ms, 720ms, and 880ms, with prediction errors of 18.7, 33.2, 39.4, 49.2, 56.0, and 61.3, respectively. Meanwhile, it maintained performance comparable to existing state-of-the-art methods at other time steps. These results demonstrate that the method of this invention can more effectively model the spatiotemporal dependencies in human motion sequences and maintain low errors in the medium to long term prediction stage, exhibiting strong prediction accuracy and stability.

[0130] Table 5 Experimental results of the AMASS-BMLrub dataset

[0131]

[0132] Table 6 shows the performance comparison results of the method of the present invention and various existing human motion prediction methods on the 3DPW dataset. This table also uses MPJPE as the evaluation metric, comparing the prediction errors of different methods at multiple prediction time steps to verify the generalization ability and prediction performance of the method of the present invention in complex real-world scene data.

[0133] As shown in Table 6, on the 3DPW dataset, the method of this invention achieved optimal results at multiple prediction time steps, including 160ms, 320ms, 400ms, 560ms, 720ms, 880ms, and 1000ms, with prediction errors of 21.5, 37.2, 43.5, 53.7, 61.0, 66.7, and 70.6, respectively. Its overall performance is superior to existing comparative methods. Given the more complex real-world scene interference and stronger data distribution variations in the 3DPW dataset, the fact that the method of this invention can still maintain leading performance at multiple prediction time steps indicates its stronger modeling ability for the spatiotemporal dynamics of human movement. This effectively improves the prediction accuracy of future human postures in complex scenes and further verifies the good robustness and cross-dataset generalization ability of the method of this invention.

[0134] Table 6 Experimental results on the 3DPW dataset

[0135] Table 7 shows the results of the proposed method for conventional and extended long-term prediction on the Human3.6M dataset. The evaluation metric is MPJPE, in millimeters. To adapt to long-term prediction tasks, the proposed method re-optimizes the model during training, thereby improving the model's performance in the extended prediction interval. However, this retraining also leads to some changes in performance in the conventional prediction interval (0~1000ms). Data for other methods in the table are from DD-GCN.

[0136] As can be seen from the experimental results in Table 7, the method of this invention has achieved low errors in the conventional prediction intervals (560ms and 1000ms), namely 76.51mm and 109.06mm, respectively, which is superior to existing methods. Meanwhile, it also maintains leading performance in the extended prediction interval (1200ms~2000ms), achieving prediction errors of 119.49mm, 123.16mm, 127.96mm, 133.67mm, 140.83mm, and 146.62mm at 1200ms, 1300ms, 1400ms, 1600ms, 1800ms, and 2000ms, respectively, showing a significant advantage over existing methods in most time steps. This result fully demonstrates that thanks to the sliding window mechanism adopted by the HSAE module in the time dimension, the node aggregation operation can slide sequentially along the time axis, thereby better preserving motion information between multiple frames, significantly reducing the accumulated error in extended prediction, and improving the accuracy and robustness of the model in long-term prediction tasks.

[0137] Table 7 Experimental results of standard and extended predictions on the H3.6M dataset.

[0138]

[0139] Table 8 shows the comparison results of the proposed method with several existing human motion prediction methods in terms of model complexity. The evaluation metrics include the number of model parameters, computational cost (FLOPs), inference time, and memory usage. To measure the efficiency of the framework in detail, the proposed method divides the model into a "Joint" branch and a "Bone" branch. The "Joint" branch contains 0.31M parameters and 33.3M FLOPs, while the "Bone" branch contains 0.14M parameters and 8.7M FLOPs. The inference time is measured after the two branches are fused.

[0140] As shown in Table 8, the method of this invention outperforms the comparative methods in terms of parameter count, computational cost, inference time, and memory usage. Specifically, the total parameter count is 0.45M, FLOPs are 42M, inference time is 4.26ms, and memory usage is 2.1G, all of which are optimal in the table. Experimental results demonstrate that even after dividing the model into "Joint" and "Bone" parts, the method of this invention maintains a lightweight design and exhibits a significant advantage in inference speed, meeting the real-time requirements for efficient human motion prediction.

[0141] Table 8 Comparison of Model Complexity

[0142]

[0143] Figure 7 This paper presents the ablation experiment visualization results of the proposed Dynamic Frequency Domain Denoising (DFD) module in a human motion prediction task. To intuitively present the denoising effect of the DFD module, we visualized the prediction results as a spectrum in the frequency domain and compared them with the prediction results without DFD processing and the actual joint trajectory (Ground Truth, GT).

[0144] As can be observed from the spectral distribution in the figure, after denoising using the DFD module, the frequency domain characteristics are closer to the true spectrum, the spectral energy distribution conforms to the characteristics of Discrete Cosine Transform (DCT), the amplitude of the main motion frequency components is enhanced, while the noise frequency components are significantly weakened. The corresponding spatial domain joint prediction visualization results show that without the DFD module, the predicted posture exhibits significant perturbations and large joint position deviations, especially in medium- to long-term predictions where step errors are more pronounced. However, after introducing the DFD module, the predicted posture is smoother in the overall skeleton structure, the joint motion trajectory is highly consistent with the true posture, and the perturbation amplitude is significantly reduced.

[0145] Furthermore, the bottom of the figure illustrates the difference between the original spectrum and the spectrum after DFD denoising. Different colors are used to indicate the magnitude of the prediction error, clearly showing that the DFD module can suppress high-frequency noise and enhance key motion frequency components, thereby simultaneously improving the accuracy and stability of human posture prediction in both the frequency and spatial domains. In summary, the DFD module of this invention plays a significant role in removing prediction noise and preserving key motion frequency information.

[0146] Figure 8This paper presents the visualization results of the proposed method for predicting poses of the "walking" action in the Human3.6M dataset. To more intuitively demonstrate the prediction effect of the proposed method, we compare the predicted future motion sequences with the actual joint trajectories and the prediction results of the existing open-source method siMLPe. The areas marked with red boxes in the figure are the key comparison parts. It can be clearly observed that the proposed method is closer to the actual motion state in the long-term prediction stage, and the joint motion trajectory is smoother and conforms to the natural movement law of the human body. Compared with siMLPe, the proposed method is more accurate in predicting leg, arm, and overall skeletal coordination, which fully verifies that the proposed method can generate high-fidelity, stable, and natural human motion sequences in both short-term and medium-to-long-term conventional prediction tasks.

[0147] Figure 9 The key analytical results of the method of the present invention in ablation experiments are presented, including two sub-figures: the upper figure shows the ablation analysis of the number of DFD modules, and the lower figure shows the comparative analysis of different denoising methods, which aims to verify the design effectiveness of the dynamic frequency domain denoising (DFD) module and its impact on the accuracy of human motion prediction.

[0148] Figure 9 The above figure, through ablation experiments on the number of DFD modules, shows that at the 1000ms prediction time, MPJPE (All Frames) first decreases and then increases with the number of modules. The model achieves optimal prediction performance when the network contains 48 DFD modules, and the number of network parameters in this configuration is 0.451M, indicating that the number of modules has a significant impact on prediction accuracy while maintaining lightweight design.

[0149] Figure 9 The figure below verifies the specific design of the DFD module. In the experiment, the dynamic frequency denoising module proposed in this invention (green curve) was compared with traditional Gaussian filtering (blue curve) and a fixed threshold. The DFD module (yellow curve) was compared. To facilitate comparison of different methods, the vertical axis uses a standardized index of MPJPE per frame divided by the number of frames. The results show that the dynamic frequency denoising module of this invention can effectively suppress prediction noise through adaptive threshold adjustment. Compared with Gaussian filtering and fixed threshold schemes, it achieves lower errors at multiple prediction time steps, fully verifying the key role and effectiveness of the DFD module in human motion sequence prediction.

[0150] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for predicting human motion based on spectral-geometric joint representation, characterized in that, The process of predicting human motion includes: The sequence of key human skeleton nodes and bone vector sequence in multiple consecutive frames are obtained and transformed into the frequency domain based on discrete cosine transform. In the frequency domain, the dynamic frequency denoising module is used to denoise, normalize the node dimension features and extract spatiotemporal information on the features transformed to the frequency domain, respectively, to obtain the frequency domain features of the key node sequence and the bone vector sequence. Then, the inverse discrete cosine transform is used to transform them back to the original representation, and the original representation corresponding to the bone vector sequence features is used as the bone vector prediction sequence. The sequence of key human skeleton nodes in multiple consecutive frames is obtained and aggregated by the hyperbolic spatiotemporal awareness enhancement module. The aggregated features are then fused with the features processed by the dynamic frequency denoising module to obtain the joint prediction sequence. The bone vector prediction sequence and the joint prediction sequence are weighted and fused to obtain the fused position of each joint.

2. The human motion prediction method based on spectral-geometric joint representation according to claim 1, characterized in that, If the number of observation frames is T, the sequence of key nodes or bone vectors of the human skeleton is transformed to the frequency domain based on the discrete cosine transform matrix. The elements in the discrete cosine transform matrix are represented as follows: in, Let represent the element in the i-th row and j-th column of the discrete cosine transform matrix, where ; Let Kronecker function be the function of i = j. ,otherwise .

3. A human motion prediction method based on spectral-geometric joint representation according to claim 1 or 2, characterized in that, Multiple cascaded dynamic frequency denoising modules are used to denoise, normalize node-dimensional features, and extract spatiotemporal information from key human skeletal node sequences and bone vector sequences, respectively. The data processing steps of each dynamic frequency denoising module include: The power spectrum of the original frequency domain features is calculated by using the input of each dynamic frequency denoising module as the original frequency domain feature. Establish a binary mask matrix. If the power spectrum value of the original frequency domain feature is greater than a trainable threshold, the corresponding element of the binary mask matrix is ​​1; otherwise, the corresponding element of the binary mask matrix is ​​0. The original frequency domain features are processed using a binary mask matrix to obtain the denoised original frequency domain features; The original frequency domain features and the original frequency domain features after denoising are weighted and fused using two trainable filter matrices to obtain the output of the dynamic frequency denoising module.

4. The human motion prediction method based on spectral-geometric joint representation according to claim 3, characterized in that, The data processing process performed by the dynamic frequency denoising module is represented as follows: in, This is the output of the dynamic frequency denoising module; Represents the original frequency domain characteristics; Represents the original frequency domain features of the denoised data; These are learnable global filtering parameters; For learnable local filtering parameters; This represents element-wise multiplication. Represents a binary mask matrix The element in the i-th row and j-th column; This represents the power of the j-th joint coordinate at the i-th frequency component; This indicates the trainable threshold.

5. The human motion prediction method based on spectral-geometric joint representation according to claim 1, characterized in that, The hyperbolic spatiotemporal awareness enhancement module processes data in the following ways: Based on observation data of T consecutive frames, the adjacency matrix of the key node sequence of the human skeleton is expanded into a three-dimensional adjacency matrix. Each element in this matrix represents the adjacency relationship between the observation data of the i-th frame and the observation data of the j-th frame. This element is a J×J matrix. ,matrix The element in the u-th row and v-th column This represents the adjacency relationship between joint u in frame i and joint v in frame j in the spatiotemporal graph. Set an observation window τ and introduce a K-order adjacency matrix decomposition mechanism in the observation window. That is, the local matrix within the observation window is decomposed into K-order. In the k-order matrix, if the minimum reachable hops between two nodes is k, then the element at the corresponding position in the k-order matrix is ​​1, otherwise it is 0. The node features within the window are first projected onto the hyperbolic space through an exponential mapping, and then hyperbolic graph convolution operations are performed on the neighboring nodes of each order within the window to obtain the output of the hyperbolic spatiotemporal perception enhancement module.

6. The human motion prediction method based on spectral-geometric joint representation according to claim 5, characterized in that, When performing hyperbolic graph convolution operations on neighborhood nodes of each order within the window, the hyperbolic graph convolution operation includes cascaded multi-layer operations, and the output of the (l+1)th layer is represented as: in, Represents the exponential mapping operation in hyperbolic space; This represents the window feature of the output of the l-th layer in Euclidean space. The coordinates of each key point within the window are used as the window feature of the first layer of hyperbolic graph convolution. Subsequently, the output of the previous layer of hyperbolic graph convolution is used as the window feature of the current layer of hyperbolic graph convolution. The window feature representing the output of the (l+1)th layer in hyperbolic space; Represents the activation function in hyperbolic space; Represents the aggregation transformation term of the l-th layer; Represents the addition operation in hyperbolic space; Represents the multiplication operation in hyperbolic space; This represents the maximum number of hops considered when constructing multi-scale spatial adjacency relationships; Let represent the normalized local spatiotemporal neighbor matrix with hop count k; The logarithmic mapping operation for hyperbolic space is represented; Let L be the weight matrix of the l-th layer in the hyperbolic graph convolution; is the bias vector of the l-th layer in the hyperbolic graph convolution.

7. A human motion prediction method based on spectral-geometric joint representation according to claim 5 or 6, characterized in that, After expanding the adjacency matrix into a three-dimensional adjacency matrix, it is represented as: Based on the three-dimensional adjacency matrix, a K-order adjacency matrix decomposition mechanism is introduced, which yields a K-order local spatiotemporal adjacency matrix. ; The element in the i-th row and j-th column of the K-order local spatiotemporal adjacency matrix is ​​represented as: ,in This indicates that the shortest path length in the skeletal diagram is... The connection relationships between node pairs, k∈{1,2,…,K}; initial adjacency matrix. It is an identity matrix.

8. The human motion prediction method based on spectral-geometric joint representation according to claim 1, characterized in that, The fused features are weighted and fused with the bone vector sequence processed by the dynamic frequency denoising module to obtain the fused position of each joint point, represented as: in, This represents the sequence of fused joint positions in the (T+1)th frame, predicted based on T observation frames. This is the keypoint prediction sequence for frame T+1; The bone vector prediction sequence for frame T+1; These are learnable weights.

9. A human motion prediction method based on spectral-geometric joint representation according to claim 1 or 8, characterized in that, For joints other than those at the starting position, the position of the current joint is corrected based on the position of its parent joint, including: in, This indicates the corrected position of the k-th joint. This represents the predicted value of the k-th bone vector in the bone vector prediction sequence of frame T+1; This represents the predicted value of the parent node of the k-th joint in the joint prediction sequence of frame T+1. Let represent the fusion position of the parent node of the k-th joint in the joint fusion position sequence of the (T+1)-th frame predicted based on T observation frames; This represents the predicted value of the k-th joint in the joint prediction sequence of frame T+1. , and its dual weights , These are learnable parameters.

Images