Human motion prediction method based on noise prediction residual diffusion

Abstract

The invention is suitable for the technical field of motion prediction, and provides a human motion prediction method based on noise prediction residual diffusion, which comprises the following steps of: performing discrete cosine transform processing on an observed human motion sequence to obtain embedded representation; a residual error between the human body action observed in the hidden space and the complete human body action is constructed, distribution modeling is carried out through a residual error diffusion module, and parameters of the residual error diffusion module are optimized in an original action space; a deterministic sampling process is adopted to establish deterministic mapping between prediction results of the residual diffusion module; accelerating a sampling process by using a model distillation technology, and proposing a student residual diffusion model to learn a mapping relation; the randomness of human body motion prediction is adjusted through a noise predictor module so as to generate diversified prediction actions. The residual diffusion module, the model distillation technology and the noise predictor module are adopted, the reasoning speed and the prediction precision of the model are remarkably improved, and the method has high practical application value in a real-time scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of motion prediction, and particularly relates to a human motion prediction method based on noise prediction residual diffusion. Background Art

[0002] The core of the random human motion prediction task lies in predicting multiple future unobserved motion sequences based on the observed motion sequences. In recent years, with the booming development of deep learning technology, this task has been widely applied in many fields such as autonomous driving, animation production, and human-computer interaction, and significant progress has been made. However, due to the highly conscious-driven and diverse nature of human motion, achieving accurate and diverse motion prediction still faces huge challenges. Specifically, human motion is not only restricted by physical laws but also affected by multiple factors such as individual intentions, which makes the motion prediction task highly complex and uncertain. The field of human motion prediction has undergone a paradigm shift from deterministic models to stochastic models. To effectively simulate the uncertainty and diversity of human motion, deep generative models have been introduced into the research of the random human motion prediction (HMP) problem.

[0003] In existing work, models such as generative adversarial networks (GANs) and conditional variational autoencoders (CVAEs) have been widely used to model the predictive distribution of human actions and can generate multiple possible human action prediction sequences. Although these methods have made some progress in generating diversity, there are still problems such as mode collapse and unstable training. With the continuous evolution of generative model technology, diffusion models have achieved great success in fields such as image generation, image denoising, and video generation and have been introduced into the human action prediction task, demonstrating good prediction performance. Among them, the conditional diffusion model constructs the association between the Gaussian noise domain and the predicted human action domain by adding noise to the predicted human action. The observed human predicted action only serves as conditional information and implicitly guides the generation process during the reverse inference process. However, this modeling method has inherent limitations: on the one hand, the inference efficiency is poor because it requires a multi-step denoising process, resulting in high computational costs and long time consumption. This is because the distribution differences between the Gaussian noise domain and the predicted human action domain are significant. The former follows a simple normal distribution, while the latter exhibits complex spatio-temporal correlations and multi-modal characteristics. The model needs nearly 1000 diffusion steps to fit the complex mapping relationship between the two. During reverse inference, 10 - 100 sampling processes need to be iteratively executed, and the inference time reaches hundreds of milliseconds, far less efficient than competing models such as CVAEs and GANs, greatly limiting its use in real-time applications; on the other hand, in terms of diversity modeling, existing conditional diffusion models rely on a randomly initialized Gaussian distribution as the generation source and achieve diversity prediction by sampling a finite number of independent and identically distributed samples during the inference stage. There are essential defects, that is, finite samples are difficult to comprehensively cover the exact modes of the action space, and there is a lack of an explicit adaptation mechanism with the observed action sequence. Once a key sampling deviation occurs in the initial Gaussian distribution, the prediction performance will drop significantly.

[0004] In view of the many deficiencies in the existing technology for human action prediction, the present invention proposes a human action prediction method based on noise prediction residual diffusion. Summary of the Invention

[0005] The purpose of the present invention is to provide a human action prediction method based on noise prediction residual diffusion, aiming to solve the problems raised in the above background technology.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A human action prediction method based on noise prediction residual diffusion includes the following steps:

[0008] Step 1: Perform discrete cosine transform processing on the observed human action sequence to obtain an embedded representation;

[0009] Step 2: Construct the residual between the observed human motion and the complete human motion in the latent space, perform distribution modeling through the residual diffusion module, and optimize the parameters of the residual diffusion module in the original motion space;

[0010] Step 3: Establish a deterministic mapping between the prediction results of the residual diffusion module using a deterministic sampling process; Use model distillation technology to accelerate the sampling process and propose a student residual diffusion model to learn the mapping relationship;

[0011] Step 4: Adjust the randomness of human motion prediction through the noise predictor module to generate diverse predicted actions.

[0012] Furthermore, the specific process of Step 1 is as follows:

[0013] For the human observed motion sequence X, perform transformation using DCT, and only use the first 20 rows of the predefined DCT basis for modeling to obtain the embedded representation Z = DCT(X).

[0014] Furthermore, the specific process of Step 2 is as follows:

[0015] Define the complete human motion sequence as X0 = {x1, …, x N , x N+1 , …, x N+F}, where the first N frames are observed data and the last F frames are frames to be predicted; regard the human observed sequence {x1, …, x N} as a condition, and repeatedly fill the last frame x N to the prediction length, denoted as Y = {x1, …, x N , x N , …, x N}; map X0 and Y to the frequency domain through DCT transformation to obtain the corresponding DCT coefficients Z0 and Z y ;

[0016] Establish the residual E0 = Z y - Z0 between the observed human motion and the complete human motion in the latent space. The distribution modeling of the residual diffusion at the t-th step is q(Z t |Z0, Z y ) = N(Z t ; Z0 + η t E0, κ 2 η t I), t = 1, …, T, where T represents the number of diffusion steps, and q, N represent probability distribution symbols; Z t is the t-th step in the residual diffusion; E0 is the residual; I is the identity matrix; η tThe shift sequence for the t-th step, which is used to control the scheduling of residual diffusion; κ is a hyperparameter; it is processed through a residual diffusion module composed of a diffusion network and a refinement network. After the diffusion network obtains the predicted human action, it is concatenated with the observed human action and then input into the refinement network to capture the human joint dependency. The model directly fits Z0; the parameters θ of the residual diffusion module are optimized in the original action space, and the loss function is expressed as:

[0017]

[0018] Among them, L θ is the loss function of the residual diffusion module; is the joint expectation of sampling the complete human action sequence X0 and the number of noise-adding steps t; X N+1:N+F represents the true predicted part of the human action sequence; G θ (Z t , t, Z y ) represents the output prediction Z0 of the residual diffusion module; iDCT(G θ (Z t , t, Z y )) N+1:N+F represents the human action sequence predicted by the residual diffusion module;

[0019] Calculate the loss between the true predicted part of the human action sequence and the DCT embedding representation predicted by the residual diffusion module in the original action space between.

[0020] Furthermore, the specific process of step 3 is as follows:

[0021] Adopt a deterministic sampling process to establish a deterministic mapping between the embedding representation predicted by the residual diffusion module and Z T , and the formula is as follows:

[0022]

[0023] Among them, Z t-1 is the speculated embedding at the (t - 1)-th step; k t , m t , j t are coefficients; is the DCDT embedding representation predicted by the residual diffusion module; Z T is the diffusion embedding representation at the T-th step; η t is the shift sequence at the t-th step; η t-1 is the shift sequence at the (t - 1)-th step;

[0024] Propose a student residual diffusion model F θ to learn Z0 and Z TA deterministic mapping between them and is optimized through a loss function; the loss function of model distillation is defined as:

[0025]

[0026] where L distill is the loss function of model distillation; is the DCT embedding representation predicted by the residual diffusion module; T is the total number of diffusion steps; Z y is the DCT embedding representation of the observation sequence.

[0027] Furthermore, the specific process of step 4 is as follows:

[0028] Input the embedded DCT coefficients Z of the human motion sequence y into the noise predictor module, use a graph convolutional network to model the spatial dependencies between human joints, and then adjust the output through a fully connected layer to match the target distribution, denoted as Noise 1:K = αtanh(GCN(Z y ))), where Noise 1:K are the K noises predicted by the noise module; α is a hyperparameter; apply the 3σ principle to process Gaussian noise, set the hyperparameter α = 3, and use the tanh activation function to limit the output within the range of [-3, 3], and finally output a set of noise samples [noise 1 , noise 2 , …, noise K ;

[0029] The loss of the noise predictor is defined as follows:

[0030]

[0031] where L noise_predictor is the loss of the noise predictor; X0 is the complete human action sequence; represents the o-th generated motion prediction;

[0032] Fuse the noise samples with the embedding to obtain Z T , and then input Z T into the student residual diffusion model F θ to perform a single-step prediction and obtain the human prediction sequence.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] The present invention adopts a residual diffusion module, model distillation technology and a noise predictor module, which significantly improves the reasoning speed and prediction accuracy of the model, and can execute a single-step efficient reasoning mechanism in the reasoning process. Compared with the existing diffusion model, its reasoning speed advantage is obvious, which makes the present invention extremely valuable in real-time scenarios. Specifically, in terms of prediction accuracy and performance improvement, the adaptive sampling technology implemented by the noise predictor module plays a key role. The entire prediction process only requires one step of reasoning to adaptively sample the observed action. The residual diffusion process is used to model the conversion from observed human actions to predicted actions. In this process, the observed action is introduced as a deterministic feature of the diffusion, and the noise mechanism is combined to regulate the diversity of the predicted action, thereby greatly reducing the number of diffusion steps. In terms of sampling and prediction accuracy optimization, the model distillation technology is used to achieve single-step fast sampling, and the noise predictor module transforms the original sampling process from the Gaussian noise distribution into a learnable process, further improving the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Flow chart of the method of the present invention.

[0036] Figure 2 This is the reasoning process of the entire framework of the present invention.

[0037] Figure 3 This is a comparison chart of the model inference time of Example 1. DETAILED DESCRIPTION

[0038] In order to have a clearer understanding of the technical features, objectives and beneficial effects of the present invention, the technical solution of the present invention is now described in detail below, but it should not be understood as limiting the scope of implementation of the present invention.

[0039] One embodiment of the present invention provides a method for predicting human motion based on noise prediction residual diffusion, aiming to solve key problems in human motion prediction tasks, such as time series modeling, inference efficiency, and non-probabilistic sampling. Through the synergistic effect of three core modules, the residual diffusion module, the model distillation module, and the noise predictor module, the method can efficiently and diversely generate motion sequences and accurately predict multiple possible future human motions. The flowchart of the method is shown in the figure. Figure 1 The specific steps are as follows:

[0040] Step 1: Perform discrete cosine transform on the observed human motion sequence to obtain an embedded representation;

[0041] The discrete cosine transform (DCT) is used to transform human actions in the time domain and frequency domain to reduce the dimension and obtain a more compact action representation. Given that the important features of human motion are mainly concentrated in the low-frequency signals, the present invention only uses the first 20 rows of the predefined DCT basis for modeling. For the human observed action sequence X, the embedded representation Z = DCT(X) is obtained after DCT transformation. Since DCT is an orthogonal transformation, the original action sequence X = iDCT(Z) can be recovered from the frequency domain through the inverse DCT transformation (IDCT).

[0042] Step 2: Construct the residual between the observed human action and the complete human action in the latent space, perform distribution modeling through the residual diffusion module, and optimize the parameters of the residual diffusion module in the original action space;

[0043] Define the complete human action sequence as X0 = {x1,…,x N ,x N+1 ,…,x N+F}, where the first N frames are the observed data and the last F frames are the frames to be predicted. Regarding the human observed sequence {x1,…,x N} as a condition, the last frame x N of the observed sequence is repeatedly filled to the prediction length, denoted as Y = {x1,…,x N ,x N ,…,x N}. Map X0 and Y to the frequency domain through DCT transformation to obtain the corresponding DCT coefficients Z0 and Z y .

[0044] Subsequently, establish the residual E0 = Z y - Z0 between the observed human action and the complete human action in the latent space. At the t-th step (t = 1,…,T, where T represents the number of diffusion steps), the distribution modeling of the residual diffusion is q(Z t |Z0,Z y ) = N(Z t ; Z0 + η t E0, κ 2 η t I), where q and N represent the probability distribution symbols; Z t is the t-th step in the residual diffusion; E0 is the residual; I is the identity matrix; η t is the shift sequence at the t-th step, which is used to control the scheduling of the residual diffusion; κ is a hyperparameter responsible for adjusting the variance.

[0045] The residual diffusion module G θIt is a simple and efficient architecture, which consists of two main components: a diffusion network and a refinement network. After the diffusion network obtains the predicted human actions, they are concatenated with the observed human actions and then input into the refinement network to capture the dependencies of human joints. The model directly fits Z0. To avoid introducing additional losses during the domain transformation process, the parameters θ of the residual diffusion module are optimized in the original action space, and the loss function is expressed as:

[0046]

[0047] where L θ is the loss function of the residual diffusion module; the joint expectation of sampling the complete human action sequence X0 and the number of noise-adding steps t; X N+1:N+F represents the true predicted part of the human action sequence; G θ (Z t ,t,Z y ) represents the output prediction Z0 of the residual diffusion module; iDCT(G θ (Z t ,t,Z y )) N+1:N+F represents the human action sequence predicted by the residual diffusion module;

[0048] Calculate the loss between the true predicted part of the human action sequence and the DCT embedding representation predicted by the residual diffusion module in the original action space between.

[0049] Step 3: Establish a deterministic mapping between the prediction results of the residual diffusion module using a deterministic sampling process; use the model distillation technique to accelerate the sampling process, and propose a student residual diffusion model to learn the mapping relationship, where the trained residual diffusion module is the teacher model, and the distilled model is the student residual diffusion model;

[0050] This method does not adopt the way of introducing random noise during inference in the model inference process. This is because the deterministic Z T will make the predicted Z0 random when establishing the mapping, which is not conducive to model distillation. This method adopts a deterministic sampling process similar to the DDIM sampling idea, and the formula is as follows:

[0051]

[0052]

[0053] where Z t-1 is the (t - 1)-step speculated embedding; k t ,m t ,j t are coefficients; The DCT embedding representation predicted by the residual diffusion module; Z T The diffusion embedding representation at the T-th step; η t The shift sequence at the t-th step; η t-1 The shift sequence at the (t - 1)-th step.

[0054] This deterministic sampling establishes a deterministic mapping between the prediction of the residual diffusion module and Z T To further accelerate the sampling process, the present invention proposes a student residual diffusion model F θ to learn and Z T The deterministic mapping between them. The student residual diffusion model is consistent with the residual diffusion module in architecture. And it is optimized through the loss function. The loss function for model distillation is defined as:

[0055]

[0056] where L distill is the loss function for model distillation; is the DCT embedding representation predicted by the residual diffusion module; T is the total number of diffusion steps; Z y is the DCT embedding representation of the observed sequence. Since this process only requires 5-step inference, the inference calculation overhead of the residual diffusion module during training is acceptable. Although distillation may cause a slight decrease in the prediction performance of the student residual diffusion model due to the limitations of the teacher model, this trade-off is meaningful because the significant improvement in inference speed and the efficient training of the noise predictor module make up for this.

[0057] Step 4: Adjust the randomness of human motion prediction through the noise predictor module to generate diverse predicted actions;

[0058] The present invention parameterizes the Gaussian noise sampling process as a learnable process to adjust the randomness of the student residual diffusion model F θ in predicting human motion. Review the sampling process of the model: project the observed human motion sequence Y into the DCT domain to obtain Z y ; then randomly add Gaussian noise to Z y to generate the latent variable Z T . The student residual diffusion model F θ performs deterministic single-step prediction. Different Z T values will correspond to different human motion predictions. The added Gaussian noise introduces diversity and randomness to the prediction.

[0059] Based on the above principle, the present invention proposes a noise predictor module to model this process. For each observed human action, the noise predictor module generates K noise samples, and then generates K predicted actions through the student residual diffusion model. Specifically, the noise predictor module takes the embedded human motion sequence DCT coefficients Z y as input and processes it using a simple network architecture: first, a graph convolutional network (GCN) is used to model the spatial dependencies between human joints, and then the output is adjusted through a fully connected layer to match the target distribution. Formally, it can be expressed as: Noise 1:K = αtanh(GCN(Z y ))), where Noise 1:K are the K noises predicted by the noise module; α is a hyperparameter. Since the noise predictor module simulates sampling from a standard normal distribution, the 3σ principle is applied to process Gaussian noise, the hyperparameter α = 3 is set, and the tanh activation function is used to limit the output within the range of [-3, 3], and finally a set of noise samples [noise 1 , noise 2 , …, noise K is output.

[0060] The loss of the noise predictor is defined as follows:

[0061]

[0062] where L noise_predictor is the loss of the noise predictor; X0 is the complete human action sequence; represents the i-th generated motion prediction. This loss can ensure that at least one motion sequence is generated close enough to the real human motion.

[0063] In the whole model workflow, the observed action undergoes DCT transformation to obtain the embedded human motion sequence Z y , Z y is input into the noise predictor module to obtain a corresponding set of noises, these noises are fused with the embedded human motion sequence Z y to obtain Z T , and then Z T is input into the student residual diffusion model F θ to perform a single-step deterministic prediction, and finally multiple human prediction sequences are obtained. The overall inference process of the framework is as Figure 2 shown.

[0064] Through the above technical solutions, the present invention can achieve one-step inference, adaptively observe human action sampling, and achieve accurate prediction and ensure diversity.

[0065] The following describes the specific implementation of the present invention in detail with reference to specific embodiments.

[0066] Example 1: In this example, the method of the present invention is experimentally verified on different datasets, evaluated through a series of metrics, and compared with other methods to test its performance.

[0067] The Human3.6M and AMASS datasets are selected for experimental verification. The Human3.6M dataset uses a human pose representation containing 17 joints. The model is trained on data from 5 subjects (S1, S5, S6, S7, and S8) and evaluated on data from 2 subjects (S9 and S11). In the experiment, a 25-frame (0.5-second) observation window is used to predict the next 100 frames (2 seconds). The AMASS dataset is a large-scale dataset that integrates 24 different datasets and adopts a unified joint configuration. After downsampling to 60Hz, it contains 9 million frames. In the experiment, a 30-frame (0.5-second) observation window is used to predict the next 120 frames (2 seconds).

[0068] A set of comprehensive metrics are used to quantitatively evaluate the performance of the method of the present invention: The Average Pairwise Distance (APD) calculates the average l2 distance between all generated sample pairs and is used to measure the diversity of the prediction results; the Average Displacement Error (ADE) calculates the average l2 distance between the ground truth and the closest generated action and evaluates the accuracy of the entire sequence; the Final Displacement Error (FDE) calculates the l2 distance between the prediction result and the ground truth at the last predicted frame and is used to evaluate the accuracy of the last step; the Multimodal Average Displacement Error (MMADE) is a multimodal version of ADE, where the ground truth actions are grouped according to similar observations, and it evaluates the model's ability to generate multimodal predictions; the Multimodal Final Displacement Error (MMFDE) is a multimodal version of FDE and is used to evaluate the accuracy of the last frame.

[0069] Table 1 Experimental Results on Human3.6M

[0070] APD ADE FDE MMADE MMFDE TPK 6.723 0.461 0.560 0.522 0.569 DLOW 11.741 0.425 0.518 0.495 0.531 GSPS 14.757 0.389 0.496 0.476 0.525 DivSamp 15.310 0.370 0.485 0.475 0.516 HumanMAC 6.301 0.369 0.480 0.509 0.545 BeLFusion 7.602 0.372 0.474 0.473 0.507 CoMusion 7.632 0.350 0.458 0.494 0.506 The method of the present invention 5.571 0.318 0.428 0.466 0.475

[0071] Table 2 Experimental Results on AMASS

[0072]

[0073]

[0074] On the Human3.6M and AMASS datasets, the method of the present invention was compared with methods such as TPK, DLOW, GSPS, DivSamp, HumanMAC, BeLFusion, and CoMusion. From the data in Table 1 and Table 2, the method of the present invention achieved state-of-the-art performance in all accuracy metrics. For example, based on CoMusion, the ADE of the method of the present invention increased by 9.14% on the Human3.6M dataset and 7.89% on the AMASS dataset, which benefited from the effectiveness of its residual diffusion module and noise prediction module.

[0075] The present invention was evaluated by generating 50 predictions for a single sample and calculating the average inference time of each sample on the validation set under the condition of a single RTX 4090 GPU with a batch size of 16. As Figure 3 shown, HumanMAC had the longest inference time because it required 100 inference steps. Although both CoMusion and BeLFusion only required 10 inference steps, BeLFusion had a shorter inference time than CoMusion operating in the original action space because it performed diffusion in the latent space. In contrast, as a single-step prediction method, SARD (the method of the present invention) was significantly faster in inference speed than other diffusion model-based methods, with a 15.97-fold improvement in inference speed compared to the fastest BeLFusion on the Human3.6M dataset and an 18.21-fold improvement on the AMASS dataset.

[0076] In summary, the present invention significantly improved the inference speed and various accuracy metrics of the diffusion model through the combination of the residual diffusion module, the distilled residual model, and the noise predictor module. In the experiments on the Human3.6M and AMASS datasets, compared with advanced baseline models such as CoMusion and BeLFusion, the present invention achieved significant improvements in accuracy metrics (ADE, FDE, etc.) and diversity quality metrics (MMADE, MMFDE, etc.). This achievement gives the model a significant advantage in time-critical scenarios. In addition, the present invention only needs to train the noise predictor module to adapt to the performance requirements under different sampling numbers, showing higher flexibility and scalability in practical applications.

[0077] The above is only the preferred embodiment of the present invention. It should be noted that for those skilled in the art, without departing from the concept of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent.

Claims

1. A human motion prediction method based on noise prediction residual diffusion, characterized in that, It includes the following steps: Step 1: Perform discrete cosine transform processing on the observed human motion sequence to obtain an embedded representation; Step 2: Construct the residual between the observed human motion and the complete human motion in the latent space, perform distribution modeling through the residual diffusion module, and optimize the parameters of the residual diffusion module in the original motion space; Step 3: Establish a deterministic mapping between the prediction results of the residual diffusion module using a deterministic sampling process; use model distillation technology to accelerate the sampling process and propose a student residual diffusion model to learn the mapping relationship; Step 4: Adjust the randomness of human motion prediction through the noise predictor module to generate diverse predicted actions.

2. The human motion prediction method based on noise prediction residual diffusion according to claim 1, wherein The specific process of Step 1 is as follows: For the human observed motion sequence X, perform transformation using DCT, and only use the first 20 rows of the predefined DCT basis for modeling to obtain the embedded representation Z = DCT(X).

3. The human motion prediction method based on noise prediction residual diffusion according to claim 2, wherein The specific process of Step 2 is as follows: Define the complete human motion sequence as \(X_0=\{x_1,\ldots,x\) N ,x\) N+1 ,\ldots,x\) N+F \}, where the first \(N\) frames are observed data and the last \(F\) frames are frames to be predicted; regard the human observation sequence \(\{x_1,\ldots,x\) N \} as a condition, and repeatedly fill the last frame \(x\) N of the observation sequence to the prediction length, denoted as \(Y = \{x_1,\ldots,x\) N ,x\) N ,\ldots,x\) N \}; map \(X_0\) and \(Y\) to the frequency domain through DCT transform to obtain the corresponding DCT coefficients \(Z_0\) and \(Z\) y ; Establish the residual E0 = Z between the observed human motion and the complete human motion in the latent space y - Z0. Model the distribution of residual diffusion at the t-th step as q(Z t |Z0, Z y ) = N(Z t ; Z0 + η t E0, κ 2 η t I), t = 1, …, T, where q and N represent probability distribution symbols; T represents the total number of diffusion steps; Z t is the t-th step in residual diffusion; E0 is the residual; I is the identity matrix η t is the shift sequence at the t-th step, which is used to control the scheduling of residual diffusion; κ is a hyperparameter; it is processed through a residual diffusion module composed of a diffusion network and a refinement network. After the diffusion network obtains the predicted human action, it is concatenated with the observed human action and then input into the refinement network to capture the human joint dependence. The model directly fits Z0; the parameters θ of the residual diffusion module are optimized in the original action space, and the loss function is expressed as: Among them, L θ is the loss function of the residual diffusion module; is the joint expectation of sampling the complete human motion sequence X0 and the number of noise addition steps t; X N+1:N+F represents the human motion sequence of the true prediction part; G θ (Z t , t, Z y ) represents the output prediction Z0 of the residual diffusion module; iDCT(G θ (Z t , t, Z y )) N+1:N+F represents the human motion sequence predicted by the residual diffusion module; Calculate the loss between the true predicted part of the human action sequence in the original action space and the DCT embedding representation predicted by the residual diffusion module. between them.

4. The human motion prediction method based on noise prediction residual diffusion according to claim 3, wherein The specific process of Step 3 is as follows: Build a deterministic mapping between the prediction of the residual diffusion module using a deterministic sampling process and Z T where Z T is the diffusion representation at the T-th step, and the formula is as follows: Among them, Z t-1 is the speculation embedding at the (t - 1)th step; k t , m t , j t are coefficients; is the DCT embedding representation predicted by the residual diffusion module; Z T is the diffusion embedding representation at the Tth step; η t is the shift sequence at the tth step; η t-1 is the shift sequence at the (t - 1)th step; Propose the student residual diffusion model F θ to learn and Z T the deterministic mapping between, and optimize through the loss function; the loss function of model distillation is defined as: Among them, L distill is the loss function for model distillation; is the DCT embedding representation predicted by the residual diffusion module; T is the total number of diffusion steps; Z y is the DCT embedding representation of the observed sequence.

5. The method for predicting human body movements based on noise prediction residual diffusion according to claim 3, wherein, The specific process of Step 4 is as follows: Embed the DCT coefficients Z of the human motion sequence y into the input noise predictor module, which uses a graph convolutional network to model the spatial dependencies between human joints and then adjusts the output through a fully connected layer to match the target distribution, denoted as Noise 1:K = αtanh(GCN(Z y ))), where Noise 1:K is the K noises predicted by the noise module; α is a hyperparameter; the 3σ principle is applied to handle Gaussian noise, the hyperparameter α = 3 is set, and the tanh activation function is used to limit the output within the range [-3, 3], and finally a set of noise samples [noise 1 , noise 2 , …, noise K is output; The loss of the noise predictor is defined as follows: Among them, L noise_predictor is the loss of the noise predictor; X0 is the complete human motion sequence; represents the i-th generated motion prediction; the noise sample is fused with the embedding to obtain Z T , and then Z T is input into the student residual diffusion model F θ to perform a single-step prediction to obtain the human prediction sequence.

Images