World coordinate system human motion trail prediction method and related equipment

Abstract

The invention discloses a world coordinate system human body motion trail prediction method and related equipment, and the method comprises the steps: firstly obtaining human body posture parameters and direction information under a camera coordinate system through a human body posture estimation network, and converting the human body posture parameters and direction information into self-center representation; then, an innovative conditional diffusion model is adopted to carry out probability modeling on the human body track, and global displacement and rotation parameters are predicted; and finally, carrying out physical optimization correction based on the contact state of the tail end joint point to ensure the naturalness of the motion. The invention further provides an efficient long sequence processing strategy, and the input sequence of any length is processed through overlapping window segmentation and progressive fusion. By introducing the probability diffusion model and the physical constraint optimization, the problem of uncertainty of human motion reconstruction in a dynamic camera scene is effectively solved, and meanwhile, the trajectory reconstruction precision and the physical rationality are remarkably improved. The method can be widely applied to the fields of computer animation, virtual reality and the like, and has important practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and human posture estimation, and in particular to a method for predicting human motion trajectory in a world coordinate system and related equipment. Background Art

[0002] Globally consistent motion reconstruction is crucial for tasks such as automated planning and human interaction, understanding human motion in field videos, and understanding and recognizing human interaction information. It can be applied to sports events, video games, and augmented reality. Reconstructing 3D human motion from monocular video under dynamic camera conditions is a more complex and challenging task. For example, in a football match broadcast, a human observer can easily perceive the camera following the athletes running on the field. However, when this dynamic 3D scene is reprojected onto a 2D image, the athletes appear to be constantly in the center of the video frame because the camera is following the movement. The image only captures the underlying human motion and camera trajectory. Therefore, many video-based human motion reconstruction methods rely solely on 2D motion detection results and are unable to recover the original trajectory in world coordinates.

[0003] Reconstructing global human trajectory and motion in a world coordinate system remains a challenging task. Scale ambiguity introduced by camera motion and focal length variations, along with the inherent scale uncertainty of monocular visual odometry, makes it difficult to accurately estimate human trajectory in a world coordinate system. Existing methods often employ deterministic prediction models that are unable to effectively handle these inherent uncertainties, resulting in unstable and inaccurate predictions. Furthermore, existing methods often suffer from defects such as foot-to-ground penetration, sliding, and foot-to-floating, resulting in suboptimal reconstruction results.

[0004] Solving such problems has enormous application prospects. It can be used in complex scenarios where motion capture equipment cannot be used directly, such as the highly flexible and complex motion reconstruction of various sports events. At the same time, as a low-cost way to obtain motion data, it can be used to directly obtain high-quality data outside the data distribution of existing motion capture datasets from a large number of Internet videos, and applied to downstream motion generation tasks. Summary of the Invention

[0005] In order to at least partially solve one of the technical problems existing in the prior art, the purpose of the present invention is to provide a method and related equipment for predicting human motion trajectories in a world coordinate system based on a diffusion model, which can effectively deal with the uncertainty problem in motion prediction and achieve accurate world coordinate system trajectory reconstruction.

[0006] The first technical solution adopted by the present invention is:

[0007] A method for predicting human motion trajectory in a world coordinate system comprises the following steps:

[0008] Get T frames of monocular input video sequence I;

[0009] Using the visual odometry algorithm, the corresponding camera angular velocity ω in the video is obtained according to the video sequence I, and the rotation matrix R from the camera to the world coordinate system is calculated according to the camera angular velocity ω; the initial global direction O in the world coordinate system is calculated according to the rotation matrix R w and the initial global position T w ;

[0010] Through the video-based monocular video human pose estimation network, the human pose θ, human shape parameters β and the orientation O of the human root joint in the camera coordinate system are obtained according to the video sequence I. cam ;

[0011] The human body posture θ, the initial global direction O in the world coordinate system w and the initial global position T w , converted to the egocentric representation Ψ;

[0012] According to the egocentric representation Ψ, human posture θ, and human shape parameter β, a diffusion model is constructed as the trajectory prior to perform probabilistic modeling of the human trajectory and predict the global displacement τ, rotation parameter γ, and the contact state CT of the end joint, where CT∈{0,1};

[0013] Based on the contact state CT of the end joints, the predicted trajectory is physically optimized and corrected using the L-BFGS algorithm to ensure the physical rationality of the motion.

[0014] Furthermore, the conversion method of the egocentric representation Ψ is as follows:

[0015] The global trajectory is converted to egocentric coordinates using the GlobalToEgo function:

[0016] Ψ 1:T =GlobalToEao(τ 1:T ,γ 1:T )

[0017] Calculate the relative xy translation (δx, δy), absolute height z, and local rotation δψ; generate an alignment trajectory representation based on the heading vector η; the relative translation and rotation are calculated as follows:

[0018]

[0019] η t =ToHeading(γ t)

[0020] Where, subscript t represents the tth frame, and ToHeading represents a translation and rotation conversion to the heading The function of the orientation coordinate system is defined as, is the translation τ t The xy components of is the translation τ t-1 The xy components of is τ t The z component of is the rotation γ t The orientation angle.

[0021] Furthermore, a Transformer-based trajectory prediction module is used to perform probabilistic modeling of the human body trajectory to predict global displacement and rotation parameters; the trajectory prediction module includes:

[0022] The conditional embedding unit concatenates the orientation vector and local body joint positions along the embedding dimension and then maps them to the embedding space via linear projection;

[0023] Feature fusion unit, which combines the projected conditional embedding with the noisy latent representation via element-wise addition to ensure alignment across time steps;

[0024] The Transformer encoder is responsible for processing the spliced time sequence and position embedding information; the final trajectory prediction is obtained by extracting the relevant dimensions corresponding to the noise trajectory from the encoder output and passing it through the linear transformation layer.

[0025] Furthermore, the specific working mode of the trajectory prediction module is as follows:

[0026] Given the input body motion θ 1:T , body shape parameter β, the trajectory predictor PDTP outputs the corresponding global trajectory (τ 1:T ,γ 1:T ), including the root node translation and rotation; to handle potential ambiguity in the global trajectory, Diffusion is used to build a global trajectory predictor:

[0027] Ψ 1:T =PDTP(θ 1:T ,β,η)

[0028] Where η represents an optional orientation vector, which is used to integrate the camera angular velocity information to achieve better trajectory prediction. The direct output of the global trajectory predictor PDTP is the egocentric trajectory Ψ 1:T , the trajectory is converted to a global trajectory (τ) by the following transformation function EgoToGlobl (the inverse function of GlobalToEgo) 1:T ,γ1:T ):

[0029] (τ 1:T ,γ 1:T )=EgoToGlobal(Ψ 1:T )

[0030] The motion trajectory of the root joint of the human body 1:T And the contact status CT of each joint and scene of the person 1:T To model, the target variable to be modeled is: x0=(Ψ 1:T ,CT 1:T );

[0031] Its forward process is defined as:

[0032] The basic simple objective function, that is, the reconstruction loss, is:

[0033]

[0034] The body shape processing module inputs the posture parameters and shape coefficient β into the model to generate standardized 3D joint coordinates

[0035] Define the forward kinematics mapping function F, and transform the human body posture parameter θ 1:T and shape coefficient β are mapped to the normalized 3D joint position space:

[0036]

[0037] stO smpl =0,T smpl =0

[0038] Where FK represents the forward kinematics function that converts joint rotation into joint position; smpl =0,T smpl = 0 ensures normalized representation of joint positions

[0039] The joint coordinates are mapped to a 512-dimensional feature space through a multi-layer perceptron (MLP) and feature fused with the noise code;

[0040] The final control signal is:

[0041]

[0042] The trained predictor network PDTP generates denoised sequences in N steps:

[0043]

[0044] Where G represents the predictor network PDTP, x n is the noise sample of the nth step in the diffusion model, and n is the number of diffusion steps.

[0045] Furthermore, a geometric loss function is introduced as a regularization term to ensure that the generated motion complies with basic physical constraints: the loss function includes the following:

[0046] 1) The position loss function measures the deviation between the predicted pose and the target pose in the global coordinate system:

[0047]

[0048] 2) The foot contact loss function ensures the velocity constraint at the contact point to prevent the foot from sliding:

[0049]

[0050] 3) Speed loss function ensures the continuity and smoothness of motion speed:

[0051]

[0052] Where F(·) is the transformation function, f i is the binary foot contact mask for each frame i, is the i-th frame of the target variable in the training data, is the i-th frame of the variable predicted by PDTP at the 0th step of the diffusion process, and N is the total length of the human motion trajectory / total number of frames; specifically, the superscript i represents the i-th frame, the subscript 0 represents the 0th step of the diffusion process, and the colon indicates that it is predicted by the predictor network PDTP;

[0053] The final total training loss function is:

[0054] L=L simple +λ pos L pos +λ vel L vel +λ foot L foot

[0055] Where λ pos ,λ vel ,λ foot is the weight, L simple is a simple objective function.

[0056] Furthermore, the physical optimization correction of the predicted trajectory based on the contact state CT of the end joint includes:

[0057] A global motion parameter correction algorithm based on static joint constraints, combined with inverse kinematics optimization, achieves better physical rationality optimization:

[0058] First, you need to construct the joint position mapping in the world coordinate system:

[0059] (Ψ 1:T ,CT 1:T )=PDTP(·)

[0060]

[0061] Where, J world Represents the complete joint position sequence in the world coordinate system;

[0062] A specific set of key control points K where contact may occur is selected, including the left and right ankle joints, toes, and wrists. The position sequence of these key points is obtained by the following mapping:

[0063] J static =J world [:,:,K]

[0064] Calculate the time series displacement difference of these key points:

[0065] ΔJ static =J static [:,1:]-J static [:,:-1]

[0066] Calculate the global displacement correction:

[0067]

[0068] Where, CT j is the static confidence; 1 static An indicator function indicating the presence of at least one static joint;

[0069] Update the global translation parameters based on the correction amount:

[0070]

[0071] renew Then finally perform Gaussian smoothing and ground alignment:

[0072]

[0073] Where G represents Gaussian smoothing operation, e y is the y-axis unit vector;

[0074] Inverse Kinematics Optimization:

[0075] For a specific set of motion control key points K (including key motion nodes such as left and right ankle joints, toes, and wrists), a time series smoothing strategy based on confidence weights is adopted. The core algorithm is expressed as follows:

[0076]

[0077] Where, represents the target joint position after smoothing; J t-1 represents the joint position at time step t-1; J t represents the joint position at the current time step t; C t-1 is the static confidence weight of time step t-1; t is the time step index; T is the total length of the human motion trajectory / total number of frames;

[0078] For human joint chains, an optimization-based pose optimization method is used:

[0079] Define the objective function:

[0080]

[0081] in:

[0082]

[0083] Where θ 1:T represents the joint angle parameter; FK(·) is the forward kinematics function; J is the target joint set.

[0084] Furthermore, the method for predicting human motion trajectory in a world coordinate system further includes the following long sequence processing steps:

[0085] For input processing of very long sequences, an efficient processing solution based on progressive fusion is designed: an overlapping window segmentation strategy is adopted to achieve parallel processing of input posture data, and a linear interpolation algorithm of the intersection area is used to ensure the continuity of the results;

[0086] Window fusion is performed based on the following formula:

[0087]

[0088] Where C is the number of overlapping frames, N is the window length, is the trajectory data of frame j in segment i.

[0089] The second technical solution adopted by the present invention is:

[0090] An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a world coordinate system human motion trajectory prediction method as described above.

[0091] The third technical solution adopted by the present invention is:

[0092] A computer-readable storage medium stores at least one instruction, at least one program, a code set, or an instruction set, which is loaded and executed by a processor to implement a method for predicting human motion trajectories in a world coordinate system as described above.

[0093] The fourth technical solution adopted by the present invention is:

[0094] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the above-mentioned method for predicting a human motion trajectory in a world coordinate system.

[0095] The beneficial effects of the present invention are as follows: the present invention proposes a novel world coordinate system human body trajectory prediction scheme based on a probability diffusion model. In terms of trajectory prediction accuracy, the human body motion trajectory is modeled by innovatively adopting a probability diffusion model, which effectively solves the prediction uncertainty problem in dynamic camera scenes. Through the progressive fusion algorithm, the present invention can efficiently process input sequences of arbitrary length. In terms of physical rationality, the present invention significantly improves the naturalness of the reconstructed motion by introducing a physical optimization correction module based on the contact state of the end joints. This physical constraint-based optimization method ensures that the reconstructed motion conforms to the kinematic characteristics of the human body. The technical solution provided by the present invention has good versatility and robustness. The present invention can not only handle various types of dynamic camera scenes, but also adapt to different human body shapes and motion patterns. This wide applicability enables the present invention to play an important role in multiple fields such as computer animation and virtual reality. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0097] Figure 1 This is a flowchart of the steps of a method for predicting human motion trajectory in a world coordinate system according to an embodiment of the present invention;

[0098] Figure 2 Schematic diagram of the network structure of the trajectory prediction module based on the probability diffusion model in an embodiment of the present invention;

[0099] Figure 3 It is a flow chart of the steps of global motion parameter correction and inverse kinematics physical optimization in an embodiment of the present invention. DETAILED DESCRIPTION

[0100] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. For the step numbers in the following embodiments, they are provided only for the convenience of explanation and are not intended to limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0101] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise clearly defined, words such as setting, installing, and connecting should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0102] In the description of this application, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0103] In the description of this application, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The terms "first" and "second" are used solely to distinguish technical features and are not to be construed as indicating or implying relative importance, or as implicitly specifying the number or order of the technical features indicated.

[0104] In the description of this application, "and / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0105] Explanation of terms:

[0106] Visual odometry (VO): A technique that uses image sequences to determine the position and movement of a camera.

[0107] Diffusion Model (DM): A probabilistic generative model that models complex data distributions by gradually adding and removing Gaussian noise from the data. The model consists of two key stages: a forward diffusion process and a backward denoising process.

[0108] Human pose and shape estimation (HPS) is a computer vision technique that estimates the 3D pose and shape of a human body from images or videos. This technique is typically based on a parameterized human body model called SMPL (Skinned Multi-Person Linear Model), which models the human body as a deformable mesh with shape and pose parameters.

[0109] Inverse Kinematics (IK): A mathematical process that calculates the joint angles required to position a robot's end effector in a specific position and orientation. It is primarily used in robotics and computer animation to create more realistic and dynamic motion sequences.

[0110] Transformer: A deep learning model structure for processing sequence data. It captures contextual information in the sequence through self-attention and positional encoding, and has good modeling capabilities and parallel computing performance.

[0111] PDTP: refers to the method proposed in this invention, Probabilistic Diffusion-based Trajectory Prior (PDTP) based on the human motion trajectory prior of the diffusion model.

[0112] DDIM, or Denoising Diffusion Implicit Models, is a generative model developed from the Denoising Diffusion Probabilistic Models (DDPM). The core concept of DDIM is to construct a non-Markovian diffusion process. While maintaining the same training objective as DDPM, it makes the reverse sampling process faster, generating high-quality samples while improving generation efficiency.

[0113] In response to the existing technical problems, the present invention provides a world coordinate system human trajectory prediction scheme based on probability diffusion. The scheme receives the input video sequence through a video acquisition module, obtains the camera angular velocity parameters using a camera motion estimation module, and obtains the human body posture and direction information in the camera coordinate system through a human posture estimation network.

[0114] The core of the invention consists of three main modules: 1) a trajectory representation module that converts human posture data into egocentric representation; 2) a Transformer-based trajectory prediction module that uses a conditional diffusion model to probabilistically model human trajectories to predict global displacement and rotation parameters; and 3) a physical optimization and correction module that optimizes the predicted trajectory using the L-BFGS algorithm based on the contact state of the end joints to ensure the physical rationality of the movement.

[0115] Furthermore, the proposed method includes a long sequence processing strategy, which, through a progressive fusion algorithm, enables efficient processing of input sequences of arbitrary length. This method not only effectively addresses the uncertainty issue of human motion reconstruction in dynamic camera scenarios, but also significantly improves the accuracy and physical rationality of trajectory reconstruction, thus possessing significant practical value.

[0116] Example 1

[0117] like Figure 1 As shown, this embodiment provides a method for predicting human motion trajectory in a world coordinate system, comprising the following steps:

[0118] S1. Obtain T frames of monocular input video sequence I;

[0119] S2. Use the visual odometry algorithm to obtain the corresponding camera angular velocity ω in the video according to the video sequence I, and calculate the rotation matrix R from the camera to the world coordinate system according to the camera angular velocity ω; calculate the initial global direction O in the world coordinate system according to the rotation matrix R w and the initial global position T w ;

[0120] S3, through the video-based monocular video human pose estimation network, obtain the human pose θ, human shape parameter β and the direction O of the human root joint point in the camera coordinate system according to the video sequence I cam ;

[0121] S4, the human body posture θ, the initial global direction O in the world coordinate system w and the initial global position T w , converted to the egocentric representation Ψ;

[0122] S5. Based on the egocentric representation Ψ, human posture θ, and human shape parameter β, a diffusion model is constructed as the trajectory prior to perform probabilistic modeling of the human trajectory and predict the global displacement τ, rotation parameter γ, and the contact state CT of the end joint, where CT∈{0,1};

[0123] S6. Based on the contact state CT of the end joint, the predicted trajectory is physically optimized and corrected through the L-BFGS algorithm to ensure the physical rationality of the motion.

[0124] The method of this embodiment is explained in detail below with reference to the accompanying drawings and specific implementation methods.

[0125] (1) Transformation of Egocentric Trajectory Representation (ETR)

[0126] In order to convert the global trajectory into relative local differences to make it suitable for predicting long trajectories, and at the same time perform a similar normalization process on the trajectory (making the values all in a unified range and space), this embodiment converts the global trajectory into ego-centered coordinates:

[0127] Ψ 1:T ={(δx t ,δy t ,z t ,δψ t ,η t ) 1:T}

[0128] And use the following conversion:

[0129] Ψ 1:T =GlobalToEgo(τ 1:T ,γ 1:T )

[0130] Where (δx t ,δy t ) represents the relative xy translation, z t Maintain absolute height, because the height of the human body relative to the ground does not change much and is highly correlated with human movement, δψt represents the local rotation, we use 6D rotation representation to avoid discontinuity, η t Represents the orientation vector of the ego-center. This orientation-aligned representation ensures invariance to global xy translation and orientation, and is unaffected by absolute xy translation and orientation. This representation is more suitable for predicting long trajectories because the network only needs to output the local trajectory change in each frame, rather than the potentially large global trajectory offset.

[0131] The egocentric trajectory representation is defined in the heading coordinate system. The heading vector points in the direction of the human face and is parallel to the ground. We align the z-axis of the human root coordinate with the z-axis of the world coordinate, and use the aligned y-axis of the root coordinate as the heading vector. This method of obtaining the heading is more reliable than using the Euler angle yaw angle, which is prone to singularities and instability. The definition process of the heading coordinate system is: first, the world coordinate origin is placed at the position of the human root node, and then the world coordinate system is rotated around the z-axis (vertical direction) to align the y-axis with the heading vector. By definition, representing and predicting the human trajectory in the heading coordinate system can make the prediction result unaffected by the absolute xy translation and orientation of the human body.

[0132] Specifically, the egocentric trajectory at time t Ψ=(δx t ,δy t ,z t ,δψ t ,η t The specific calculation of ) is as follows:

[0133]

[0134] η t =ToHeading(γ t )

[0135] in is the translation τ t The xy components of is τ t The z component (height), is the rotation γ t ToHeading is a method that converts translation and rotation to the heading angle. The function of the orientation coordinate system, η t is the local rotation. (δx0,δy0) and δφ0 are used to store the initial xy translation and orientation These initial values are fixed to 0 during training and can be arbitrary during inference (since trajectories can start from any position and orientation).

[0136] (2) Dataset processing

[0137] As an optional implementation, the training and validation datasets used were the AMASS dataset, a large publicly available motion capture (mocap) database containing over 11k motion sequences from 344 different people, which was fitted to the SMPL model. This database consolidates and standardizes many motion capture datasets into a unified dataset.

[0138] (2.1) Data preprocessing, specifically including: 1) cropping the middle 80% of each action sequence and reducing the sampling rate to 30Hz. 2) using the GLobalToEgo function described above to convert the trajectory of the left and right motion data (including the displacement and rotation of the root joint) to the EgoCentric representation Ψ. 3) using the finite difference method to estimate the velocity, and using an automatic heuristic method based on foot contact to remove action sequences that contain uneven terrain interactions (such as stairs, ramps or platforms). The ground contact (CT) of the 8 body joints (left and right toes, heels, knees and hands) is automatically annotated based on velocity and height. 1:T Specifically, a joint is considered to be in contact if its movement in the previous time step is less than 0.5 cm and its z-axis component is within 8 cm of the ground. A stricter height threshold of 4 cm is used for the toe joints. 4) The Ray data processing framework is used to accelerate this data preprocessing process.

[0139] (2.2) Training set partitioning, specifically including: CMU, MPI Limits, TotalCapture, Eyes Japan, KIT, BMLrub, BMLmovi, EKUT, and ACCAD datasets. During the validation phase of the training process, this example uses the MPIHDM05, SFU, and MPI MoSh datasets. Finally, during the evaluation phase, the HumanEva and Transitions datasets are used.

[0140] (3) Network architecture and training details

[0141] See also Figure 2, the network architecture consists of three main components: first, the conditional embedding unit, which connects the orientation vector and the local body joint position along the embedding dimension, and then maps it to the embedding space through linear projection. Second, there is the feature fusion mechanism, which combines the projected conditional embedding with the noisy latent representation through element-by-element addition to ensure alignment across time steps. Finally, there is the Transformer encoder, which is responsible for processing the concatenated temporal and positional embedding information. The final trajectory prediction is obtained by extracting the relevant dimensions corresponding to the noisy trajectory from the encoder output and passing it through a linear transformation layer. This design achieves effective integration of conditional information while maintaining the temporal coherence required for trajectory prediction.

[0142] The trajectory prediction module is implemented based on the Transformer architecture, using an 8-layer encoder structure with 4 attention heads per layer and integrating a feedforward network (FFN) with a hidden dimension of 1,024. The system uses a 200-frame sliding window for sequence processing.

[0143] Given the input body motion θ 1:T , body shape parameter β, the trajectory predictor PDTP outputs the corresponding global trajectory (τ 1:T ,γ 1:T ), including the root node translation and rotation. To handle potential ambiguity in the global trajectory, this embodiment uses Diffusion to build a global trajectory predictor.

[0144] Ψ 1:T =PDTP(θ 1:T ,β,η)

[0145] Where η represents an optional orientation vector, which is used to integrate the camera angular velocity information predicted by VO to achieve better trajectory prediction results. It can also be set to empty during inference according to different application purposes. In the above formula, the direct output of the global trajectory predictor PDTP is the egocentric trajectory Ψ 1:T , the trajectory can be converted to a global trajectory (τ 1:T ,γ 1:T ):

[0146] (τ 1:T ,γ 1:T )=EgoToGlobal(Ψ 1:T )

[0147] In this embodiment, the conditional motion synthesis is performed by N (N represents Diffusion Steps, here N = 1000) denoising back diffusion process to model the distribution p(x0|c). We do not predict ∈t , but directly predict the signal itself, that is, x0=G(x t ,t,c). We calculate the motion trajectory of the root joint of the person Ψ 1:T And the contact status CT of each joint and scene of the person 1:T To model, the target variable to be modeled is: x0=(Ψ 1:T ,CT 1:T ).

[0148] Its forward process is defined as:

[0149] The basic simple objective function, that is, the reconstruction loss, is:

[0150]

[0151] The body shape processing module inputs the posture parameters and shape coefficient β into the SMPL model to generate standardized 3D joint coordinates By converting the body posture parameters into the 3D position coordinates of the joints in the standardized 3D Euclidean space, we can better incorporate the human body shape. We define the forward kinematics mapping function F, which converts the human posture parameters θ 1:T and shape coefficient β are mapped to the normalized 3D joint position space:

[0152]

[0153] Among them, FK represents the forward kinematics function that converts joint rotations into joint positions, which is equivalent to the nonlinear mapping of the SMPL forward kinematics layer. Its parameters are: body posture parameters, body shape parameters, root key point rotation and displacement. T is the time series length, 24 represents the number of human key points, and 3 represents the three-dimensional space coordinates. By the constraint condition O smpl =0,T smpl = 0 ensures normalized representation of joint positions This mapping provides a normalized feature expression for subsequent human pose estimation and motion synthesis.

[0154] The joint coordinates are mapped to a 512-dimensional feature space using a multi-layer perceptron (MLP) and fused with noise encoding. This conditional embedding mechanism remains stable throughout the training process, effectively simplifying the computational complexity of the inference phase.

[0155] So finally our control signal is:

[0156]

[0157] The trained predictor network PDTP generates denoised sequences in N steps:

[0158]

[0159] (4) Loss function

[0160] A geometric loss function is introduced as a regularization term to ensure that the generated motion adheres to basic physical constraints. In some embodiments, three key geometric loss functions are designed: position loss, foot contact loss, and velocity loss. These loss functions correspond to different physical properties of the motion and together ensure the naturalness and coherence of the generated motion.

[0161] 1) The position loss function measures the deviation between the predicted pose and the target pose in the global coordinate system:

[0162]

[0163] 2) The foot contact loss function ensures the velocity constraint at the contact point to prevent the foot from sliding:

[0164]

[0165] 3) Speed loss function ensures the continuity and smoothness of motion speed:

[0166]

[0167] in:

[0168] The transformation function F(·) performs the following mapping process: first, the motion sequence Ψ is extracted from the input variables 1:T , which is then mapped to the global displacement τ by coordinate system transformation 1:T and rotation γ 1:T Finally, the joint position in the world coordinate system is calculated by the forward kinematics function FK(·):

[0169]

[0170] (τ 1:T ,γ 1:T )=EgoToGlobal(Ψ 1:T )

[0171] J global =FK(θ 1:T ,β,τ 1:T ,γ 1:T )

[0172] θ 1:T ,β uses GT values during training and is obtained using the existing monocular video human pose estimation algorithm during inference.

[0173] f i∈{0,1} is the binary foot contact mask for each frame i. This mask is only related to the foot and is used to indicate whether the foot is in contact with the ground. It reduces the foot sliding effect by returning the velocity to zero when it touches the ground. The foot contact label here is the same as the variable CT to be modeled previously. 1:T The difference is that here the 3D spatial position is obtained directly through forward kinematics and the label is determined in a self-inspired way.

[0174] To summarize, the total training loss function is:

[0175] L=L simple +λ pos L pos +λ vel L vel +λ foot L foot

[0176] (5) Implementation of trajectory prediction technology integrating camera perception

[0177] Using the PDTP model as a priori for human motion scale estimation, through deep integration with the visual odometry (VO) system and fusing camera-aware trajectory prediction (CA-TP), we effectively address the inherent limitations of traditional VO systems in absolute scale estimation and achieve more accurate human motion trajectory prediction and reconstruction.

[0178] In actual engineering implementation, the system first obtains the camera angular velocity ω cam , calculate the rotation matrix R from the camera to the world coordinate system by time domain integration c2w :

[0179]

[0180] Based on the obtained coordinate system transformation matrix, the system performs accurate spatial transformation operations. The global direction in the world coordinate system is obtained by the following transformation:

[0181]

[0182] Accordingly, the global position is calculated using a compensated coordinate transformation:

[0183]

[0184] Further get the egocentric trajectory representation:

[0185]

[0186] On this basis, the PDTP model receives Ψ globalProvided global direction η global and the joint position J in the normalized coordinate system cano As conditional input, output motion scale prediction result:

[0187] (δx,δy,z,δψ,CT)=PDTP(J cano ,η global )

[0188] Among them, (δx, δy) represents the relative root displacement, z defines the absolute root height, δψ encodes the local direction information, and CT is the contact information of the end joint point.

[0189] (6) Long sequence data processing technology solution

[0190] To process extremely long sequence inputs, this system has designed an efficient processing solution based on Progressive Trajectory Fusion during Inference (PTI). This solution uses an overlapping window segmentation strategy to parallelize the input pose data and ensures the continuity of the results through linear interpolation of the intersection region. This solution effectively ensures the coherence of the time series data while maintaining computational efficiency.

[0191] The specific fusion algorithm is implemented as follows:

[0192]

[0193] in, Defined as the trajectory data for frame j in segment i, N = 128 is the segment length parameter, and C = 16 is the number of frames in the overlapping region. This algorithm achieves smooth transitions in sequence data through adaptive weight allocation. The above fusion operation exploits the linear additivity properties of our trajectory representation and 6D rotation representation.

[0194] (7) Global motion parameter correction and inverse kinematics physical optimization (IK-PostProcess)

[0195] See also Figure 3 , a global motion parameter correction algorithm based on static joint constraints is proposed, and inverse kinematics optimization is combined to achieve better physical rationality optimization.

[0196] (7.1) Construct the joint position mapping in the world coordinate system. This process can be achieved through the following transformation sequence:

[0197] (Ψ 1:T ,CT 1:T )=PDTP(·)

[0198]

[0199] in, Represents the complete joint position sequence in the world coordinate system, where T is the time series length and N is the number of joints.

[0200] A specific set of key control points K where contact may occur is selected, including the left and right ankle joints (indices: 7, 10), toes (indices: 8, 11) and wrists (indices: 20, 21). The position sequence of these key points can be obtained by the following mapping:

[0201] J static =J world [:,:,K],K={7,10,8,11,20,21}

[0202] Calculate the time series displacement difference of these key points:

[0203] ΔJ static =J static [:,1:]-J static [:,:-1]

[0204] Calculate the global displacement correction:

[0205]

[0206] The static confidence CT j It is predicted by PDTP and indicates whether the eight joints in the jth frame are stationary (the CT predicted in PDTP is a probability value between 0 and 1. During training, MSE is used directly for training without using SoftMax to convert to probability). static An indicator function indicating the presence of at least one static joint.

[0207] Update the global translation parameters based on the correction amount:

[0208]

[0209] renew Then finally perform Gaussian smoothing and ground alignment:

[0210]

[0211] Among them, G represents Gaussian smoothing operation, e y is the y-axis unit vector.

[0212] (7.2) Inverse kinematics optimization

[0213] Before inverse kinematics optimization is performed, trajectory smoothing is performed to ensure temporal consistency of the generated motion. This process begins by recalculating the joint positions in the world coordinate system through forward kinematics.

[0214] For a specific set of motion control key points K (including key motion nodes such as the left and right ankle joints, toes, and wrists), we use a time series smoothing strategy based on confidence weights. The core algorithm can be expressed as follows:

[0215]

[0216] in, represents the target joint position after smoothing; J t-1 represents the joint position at time step t-1; J t represents the joint position at the current time step t; C t-1 is the static confidence weight at time step t-1; t is the time step index, and the domain is [2, T].

[0217] This smoothing strategy effectively reduces the jitter of the joint trajectory and provides more stable target position constraints for subsequent inverse kinematics optimization.

[0218] (7.3) For the human joint chain, use the optimization-based posture optimization method:

[0219] Define the objective function:

[0220]

[0221] in

[0222]

[0223] Where θ 1:T represents the joint angle parameter; FK(·) is the forward kinematics function; J is the target joint set.

[0224] Here we only optimize θ 1:T parameter. The parameters are fixed during the optimization process.

[0225] In this project, we use an iterative optimization method based on the objective function to solve the optimal human posture parameters. This optimization problem can be formally expressed as finding the optimal parameters:

[0226]

[0227] During the optimization process, we consider four main kinematic chains of the human body, namely:

[0228] 1) Left leg kinetic chain: Starting from the hip joint, through the knee joint to the ankle joint, and finally to the end of the toes.

[0229] 2) Right leg kinetic chain: Also follows the hierarchical structure of hip joint → knee joint → ankle joint → toe.

[0230] 3) Left arm kinetic chain: Starting from the shoulder joint, extending through the elbow joint to the end of the wrist.

[0231] 4) Right arm kinematic chain: Maintain the same joint chain structure as the left arm.

[0232] This kinematic chain-based optimization strategy can better maintain the kinematic constraint relationship between joints.

[0233] In the specific engineering implementation, we chose the Limited-memory BFGS (L-BFGS) algorithm as the core optimizer. As a memory-efficient variant of the quasi-Newton method, the L-BFGS algorithm approximates the Hessian matrix by maintaining a compact gradient history, which is particularly suitable for handling large-scale optimization problems. In actual deployment, the optimizer is configured with an upper limit of 10 iterations, an initial learning rate of 1e-2, a gradient history of 10, and a convergence threshold of 1e-5. Finally, the optimizer outputs the time series human posture parameters The sequence length T and the number of joints J = 24.

[0234] This module complements the global trajectory prediction in the aforementioned PDTP framework through the organic combination of static constraints and kinematic optimization, jointly ensuring the physical rationality and temporal coherence of motion reconstruction.

[0235] (8) Effect verification

[0236] To verify the effectiveness of the proposed method, Tables 1 and 2 show a comprehensive quantitative comparative analysis of our method and the existing state-of-the-art models on the EMDB dataset. The evaluation method combines both kinematics-based and physics-based metrics to achieve a comprehensive performance evaluation. Specifically, we use the following evaluation metrics: (1) W-MPJPE, which divides the motion sequence into 100-frame segments and aligns them with the true value. 100 and WA-MPJPE 100Indicators, where the former uses the first two frames for alignment and the latter uses the entire segment for alignment, both in millimeters; (2) Root translation error (RTE), used to evaluate the overall trajectory error after alignment with the initial camera pose, expressed as a percentage, in meters; (3) Foot lift (FL) and foot slide (FS) indicators used to evaluate ground contact inaccuracy, where FS focuses on the displacement between consecutive frames and FL measures the vertical distance from the ground to the nearest mesh vertex, both in millimeters; (4) G-Accel indicator used to evaluate acceleration in the global coordinate system and Jitter indicator used to measure the degree of motion jitter (in meters per second 3); (5) Center of mass error (E-COM) indicator used to quantify the difference between the center of mass of the predicted motion and the actual motion, used to reflect the static physical rationality of the motion. In terms of kinematic indicators, we focus on the accuracy of trajectory estimation in the world coordinate system, which is crucial for the trajectory prediction task; in terms of physical indicators, we focus on evaluating the quality of the motion in terms of physical properties.

[0237] Acceleration (Accel) is measured in mm / s²; all other metrics are measured in mm. Bold numbers indicate optimal performance for that metric. Bold numbers indicate optimal performance for each metric. For all metrics, lower is better.

[0238] Table 1: Comparison of global motion estimation accuracy with existing state-of-the-art models on the EMDB dataset

[0239]

[0240]

[0241] Table 2: Comparison of physical plausibility evaluation indicators with existing state-of-the-art models on the EMDB dataset

[0242] method Jitter↓ COM-Err↓ Accel↓ FS↓ FL↓ WHAM 28.7 20.1 8.9 4.3 1134.8 PDTP 27.1 18.2 8.5 5.4 54.9 PDTP w / IK-PostProcess 26.8 17.9 8.3 3.5 51.2

[0243] In summary, the method of the present invention has the following advantages and beneficial effects compared to the prior art:

[0244] (1) This paper innovatively uses a diffusion model to probabilistically model human trajectories, effectively solving the problem of prediction uncertainty in dynamic camera scenarios. By modeling trajectory prediction as a conditional probability distribution, the system can better handle scale ambiguity in monocular vision and significantly improve the accuracy of trajectory reconstruction. Experiments show that this method reduces the W-MPJPE index to 320.7mm, an improvement of more than 40% compared to existing technologies.

[0245] (2) The egocentric trajectory representation method proposed in this paper provides a more effective feature expression for global motion modeling. By converting global trajectories into relative local differences, this representation method not only simplifies the complexity of the prediction task but also provides a more stable optimization objective, enabling the system to better handle long sequence inputs. Test results show that this representation method effectively reduces the root node translation error (RTE) to 3.5%.

[0246] (3) The physical optimization and correction module designed by the present invention based on the contact state of the end joints significantly improves the naturalness of the reconstructed motion. Through the L-BFGS-based optimization framework, the module can effectively eliminate unnatural phenomena in the motion, improving the foot sliding index from 5.4mm to 3.5mm while maintaining a low foot lift error (51.2mm). This physical constraint-based optimization method ensures that the reconstructed motion conforms to the kinematic characteristics of the human body.

[0247] (4) This invention significantly improves computational efficiency while ensuring prediction quality. By adopting the DDIM sampling strategy, the number of diffusion steps is reduced from 1000 to 100; a progressive fusion algorithm is used to achieve smooth transitions for long sequences. This efficient computational strategy enables the system to meet the performance requirements of practical applications.

[0248] (5) The present invention has excellent versatility and scalability. It can not only adapt to different camera motion modes and human motion types, but can also be easily integrated with other deep learning frameworks. By decoupling the body posture parameters and camera angle parameters, the human motion estimation prior can be used in more downstream tasks such as three-dimensional character animation generation, further enhancing the generation effect of character animation generation, including naturalness and physical rationality. This flexible architectural design provides a solid foundation for subsequent technical improvements and application expansion.

[0249] Example 2

[0250] This embodiment provides a world coordinate system human motion trajectory prediction system, including:

[0251] The video acquisition module is used to acquire T frames of monocular input video sequence I;

[0252] The posture estimation module is used to obtain the corresponding camera angular velocity ω in the video according to the video sequence I, calculate the rotation matrix R from the camera to the world coordinate system according to the camera angular velocity ω; calculate the initial global direction O in the world coordinate system according to the rotation matrix R w and the initial global position T w ; According to the video sequence I, the human body posture θ, human body shape parameter β and the direction of the human root joint point O in the camera coordinate system are obtained cam ;

[0253] Trajectory representation module, used to transform human posture θ, initial global direction O w and the initial global position T w , converted to the egocentric representation Ψ;

[0254] The trajectory prediction module is used to construct a diffusion model as a trajectory prior based on the egocentric representation Ψ, human posture θ, and human shape parameter β, perform probabilistic modeling of the human trajectory, and predict the global displacement τ, rotation parameter γ, and contact state CT of the end joints;

[0255] The physical optimization module is used to perform physical optimization and correction on the predicted trajectory based on the contact state CT of the end joint points.

[0256] As a preferred embodiment, the trajectory prediction module adopts a diffusion model architecture based on Transformer, which includes a conditional embedding layer, a feature fusion layer and an encoder layer, to achieve high-precision prediction of human motion trajectories. The physical optimization module corrects the predicted trajectory by establishing an optimization objective function based on contact constraints and combining the optimization algorithm to effectively improve the physical rationality of the motion. The present invention effectively solves the uncertainty problem of human motion reconstruction in dynamic camera scenes by introducing a probability diffusion model and physical constraint optimization, while significantly improving the accuracy and physical rationality of trajectory reconstruction. Compared with the existing technology, this system achieves the best results compared with existing methods on a public verification dataset, while maintaining a low computational overhead. The present invention can be widely used in computer animation, virtual reality and other fields, and has important practical value.

[0257] Specifically, since the system is a world coordinate system human motion trajectory prediction system of an embodiment of the present invention, and the principle of solving the problem by the system is similar to that of the method, the implementation of the device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0258] Example 3

[0259] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the following Figure 1 A method for predicting human motion trajectory in a world coordinate system is shown.

[0260] It is understood that the memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the server, etc.

[0261] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the server. It executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory. Optionally, the processor may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs, while the modem handles wireless communications. It is understood that the modem may not be integrated into the processor and may be implemented separately via a single chip.

[0262] Since the electronic device is an electronic device corresponding to a world coordinate system human motion trajectory prediction method of an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0263] Example 4

[0264] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 1 A method for predicting human motion trajectory in a world coordinate system is shown.

[0265] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0266] Since the storage medium is a storage medium corresponding to a world coordinate system human motion trajectory prediction method of an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0267] Example 5

[0268] In some possible implementations, various aspects of the method of the embodiments of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps of a method for predicting human motion trajectories in a world coordinate system according to various exemplary embodiments of the present application as described above in this specification. The executable computer program code or "code" for executing each embodiment may be written in a high-level programming language such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0269] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0270] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0271] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A method for predicting human motion trajectory in a world coordinate system, characterized in that: The following steps are involved: Get T frames of monocular input video sequence I; According to the video sequence I, the corresponding camera angular velocity ω in the video is obtained, and the rotation matrix R from the camera to the world coordinate system is calculated according to the camera angular velocity ω; the initial global direction O in the world coordinate system is calculated according to the rotation matrix R w and the initial global position T w ; According to the video sequence I, the human body posture θ, human body shape parameter β and the direction of the human root joint point O in the camera coordinate system are obtained cam ; The human body posture θ, the initial global direction O w and the initial global position T w , converted to the egocentric representation Ψ; Based on the egocentric representation Ψ, human posture θ, and human shape parameter β, a diffusion model is constructed as the trajectory prior to perform probabilistic modeling of the human trajectory and predict the global displacement τ, rotation parameter γ, and contact state CT of the end joints. Based on the contact state CT of the end joint, the predicted trajectory is physically optimized and corrected.

2. The method for predicting human motion trajectory in a world coordinate system according to claim 1, characterized in that: The egocentric representation Ψ is transformed as follows: The global trajectory is converted to egocentric coordinates using the GlobalToEgo function: P 1:T =GlobalToEgo(τ 1:T ,c 1:T ) Calculate the relative xy translation (δx, δy), absolute height z, and local rotation δψ; generate an alignment trajectory representation based on the heading vector η; the relative translation and rotation are calculated as follows: v t =ToHeading(γ t ) Where, subscript t represents the tth frame, and ToHeading represents a translation and rotation conversion to the heading The function of the orientation coordinate system is defined as, is the translation τ t The xy components of is the translation τ t-1 The xy components of is τ t The z component of is the rotation γ t The orientation angle.

3. The method for predicting human motion trajectory in a world coordinate system according to claim 1, wherein: A Transformer-based trajectory prediction module is used to perform probabilistic modeling of human body trajectories to predict global displacement and rotation parameters. The trajectory prediction module includes: The conditional embedding unit concatenates the orientation vector and local body joint positions along the embedding dimension and then maps them to the embedding space via linear projection; Feature fusion unit, which combines the projected conditional embedding with the noisy latent representation via element-wise addition to ensure alignment across time steps; The Transformer encoder is responsible for processing the spliced time sequence and position embedding information; the final trajectory prediction is obtained by extracting the relevant dimensions corresponding to the noise trajectory from the encoder output and passing it through the linear transformation layer.

4. The method for predicting human motion trajectory in a world coordinate system according to claim 3, wherein: The specific working mode of the trajectory prediction module is as follows: Given the input body motion θ 1:T , body shape parameter β, the trajectory predictor PDTP outputs the corresponding global trajectory (τ 1:T ,γ 1:T ), including the root node translation and rotation; to handle potential ambiguity in the global trajectory, Diffusion is used to build a global trajectory predictor: P 1:T =PDTP(θ 1:T ,b,h) Where η represents an optional orientation vector; the direct output of the global trajectory predictor PDTP is the egocentric trajectory Ψ 1:T , the trajectory is converted to a global trajectory (τ 1:T ,γ 1:T ): (t 1:T ,c 1:T )=EgoToGlobal(Ψ 1:T ) The motion trajectory of the root joint of the human body 1:T And the contact status CT of each joint and scene of the person 1:T To model, the target variable to be modeled is: x0=(Ψ 1:T ,CT 1:T ); Its forward process is defined as: The basic simple objective function, that is, the reconstruction loss, is: The pose parameters and shape coefficient β are input into the model to generate the standardized 3D joint coordinates Define the forward kinematics mapping function F, and transform the human body posture parameter θ 1:T and shape coefficient β are mapped to the normalized 3D joint position space: s.t.O smpl =0,T smpl =0 Where FK represents the forward kinematics function that converts joint rotation into joint position; smpl =0,T smpl = 0 ensures normalized representation of joint positions The joint coordinates are mapped to the feature space through a multi-layer perceptron and then fused with the noise code; The final control signal is: The trained predictor network PDTP generates denoised sequences in N steps: Where G represents the predictor network PDTP, x n is the noise sample at step n in the diffusion model.

5. The method for predicting human motion trajectory in a world coordinate system according to claim 1, wherein: A geometric loss function is introduced as a regularization term to ensure that the generated motion complies with basic physical constraints: the loss function includes the following: 1) The position loss function measures the deviation between the predicted pose and the target pose in the global coordinate system: 2) The foot contact loss function ensures the velocity constraint at the contact point to prevent the foot from sliding: 3) Speed loss function ensures the continuity and smoothness of motion speed: Where F(·) is the transformation function, f i is the binary foot contact mask for each frame i, is the i-th frame of the target variable in the training data, is the predicted variable of the i-th frame in the 0th step of the diffusion process, and N is the total length of the human motion trajectory / total number of frames; The final total training loss function is: L=L simple +λ pos L pos +λ vel L vel +λ foot L foot Where λ pos ,λ vel ,λ foot is the weight, L simple is a simple objective function.

6. The method for predicting human motion trajectory in a world coordinate system according to claim 1, characterized in that: The physical optimization correction of the predicted trajectory based on the end joint contact state CT includes: A global motion parameter correction algorithm based on static joint constraints, combined with inverse kinematics optimization, achieves better physical rationality optimization: First, you need to construct the joint position mapping in the world coordinate system: (P 1:T ,CT 1:T )=PDTP(·) Where, J world Represents the complete joint position sequence in the world coordinate system; A specific set of key control points K where contact may occur is selected, including the left and right ankle joints, toes, and wrists. The position sequence of these key points is obtained by the following mapping: J static =J world [:,:,K] Calculate the time series displacement difference of these key points: ΔJ static =J static [:,1:]-J static [:,:-1] Calculate the global displacement correction: Where, CT j is the static confidence; 1 static An indicator function indicating the presence of at least one static joint; Update the global translation parameters based on the correction amount: renew Then finally perform Gaussian smoothing and ground alignment: Where G represents Gaussian smoothing operation, e y is the y-axis unit vector; Inverse Kinematics Optimization: For a specific set of motion control key points K, a time series smoothing strategy based on confidence weight is adopted, and its core algorithm is expressed as follows: Where, represents the target joint position after smoothing; J t-1 represents the joint position at time step t-1; J t represents the joint position at the current time step t; C t-1 is the static confidence weight of time step t-1; t is the time step index; T is the total length of the human motion trajectory / total number of frames; For human joint chains, an optimization-based pose optimization method is used: Define the objective function: in: Where θ 1:T represents the joint angle parameter; FK(·) is the forward kinematics function; J is the target joint set.

7. The method for predicting human motion trajectory in a world coordinate system according to claim 1, wherein: The method for predicting human motion trajectory in a world coordinate system further includes the following steps of processing a long sequence: For input processing of very long sequences, an efficient processing solution based on progressive fusion is designed: an overlapping window segmentation strategy is adopted to achieve parallel processing of input posture data, and a linear interpolation algorithm of the intersection area is used to ensure the continuity of the results; Window fusion is performed based on the following formula: Where C is the number of overlapping frames, N is the window length, is the trajectory data of frame j in segment i.

8. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises computer instructions, which are used to perform the method according to any one of claims 1 to 7 when executed by a processor.

Images