Method for fast synthesis of human-object interaction actions based on priori and physical constraints

Abstract

The application discloses a human-object interaction action rapid synthesis method based on priori and physical constraints, and relates to the technical field of computer graphics and artificial intelligence. The application comprises a training stage and an inference stage, and realizes efficient generation through the fine-tuning strategy of the training stage and the physical constraint optimization of the inference stage. In the training stage, the pre-training diffusion model parameters are loaded, a multi-step denoising process is compressed by at least a step, and end-to-end training is performed in combination with multi-dimensional supervision signals such as basic reconstruction, key joints and contact constraints; in the inference stage, random noise is used as initial input, and an iterative denoising process with physical constraints is performed to generate a motion sequence that conforms to semantics and is physically reasonable. The application solves the problems of numerous denoising steps and low efficiency when the existing diffusion model generates interactive motion, and in the case of greatly reducing the denoising steps, the generation quality close to the original multi-step model is maintained, and the application is suitable for real-time scenes such as robot interaction and virtual animation.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics and artificial intelligence, and in particular to a method for rapid synthesis of human-object interaction actions based on prior distillation and physical constraints. Background Technology

[0002] In the field of human-object interaction, generating human motion with high spatiotemporal synchronization and physical realism has long been a key challenge in the intersection of computer vision, graphics, and robotics. Such technologies have key value in applications such as virtual reality interaction, service robot behavior planning, and film and animation production. With the rise of the metaverse and digital twin concepts, the demand for high-fidelity, real-time human motion synthesis is showing an exponential growth trend.

[0003] We analyze how to automatically generate realistic human motion sequences based on given object information and natural language descriptions. Specifically, when the system receives a target object and its corresponding action command, it needs to generate a series of continuous and natural human movements. These movements must accurately express the intention of the action described in the language, maintain reasonable physical interaction with the target object, and conform to the laws of human motion and the dynamic constraints of the physical world. The core challenge lies in how to coordinate the relationship between language semantics, object attributes, and physical laws to generate realistic and believable human-object interaction actions that meet the command requirements. Existing diffusion models have achieved good performance in motion generation quality, but require a large number of diffusion steps, typically 1000 steps, to obtain high-quality results, resulting in low generation efficiency. Furthermore, they still have shortcomings in physical plausibility, such as contact constraints, joint stability, and consistency of object manipulation, failing to achieve the desired effect. Summary of the Invention

[0004] The purpose of this invention is to address the aforementioned shortcomings of existing technologies by providing a rapid synthesis method for human-object interaction actions based on prior knowledge and physical constraints that can generate high-quality and physically plausible results even with a low number of diffusion steps.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints includes two phases: a training phase and an inference phase, and is carried out in the following steps:

[0007] The training phase includes:

[0008] Step 1: Obtain input information during the training phase, including:

[0009] a) Obtain the 3D geometric information of the interactive object, represented using a basic point set method; then input the point set into the MLP to obtain the geometric features of the interactive object. ;

[0010] The geometric information of the object is represented by a base point set consisting of multiple three-dimensional sampling points to characterize the shape and pose of the object. The base point set is defined in the local coordinate system of the object. When the pose of the object changes, all base point sets are transformed synchronously with the rotation matrix and translation vector of the object, providing a geometric reference for subsequent motion prediction.

[0011] Specifically: Points are uniformly sampled from the surface of the object's 3D mesh model, covering the object's vertices, midpoints of edges, and locations of abrupt changes in surface curvature (such as the junction of a chair leg and seat, or the corners of a box). The Poisson disk sampling algorithm can be used for sampling. Uniform random sampling is used to ensure that the point set is evenly distributed in space and does not overlap, so as to avoid local point clouds being too dense or too sparse.

[0012] b) Encode the text instructions describing the interactive actions into text feature vectors. ;

[0013] The text instructions are extracted into fixed-dimensional text feature vectors using a pre-trained text encoding model CLIP. These vectors have the same dimension as the geometric feature vectors.

[0014] c) Implicit representation of noisy human-object interaction motion sequences, including multi-frame human meshes and object poses. That is, a vector is sampled from the standard Gaussian distribution as the implicit representation of the first denoising. In each subsequent denoising round, the denoising result of the previous round is used as the input for the next round.

[0015] d) Broadcast the current denoising level to each frame, with the denoising level vector having the same dimension as the text feature vector. Similarly, the vector is used to extract features through MLP, and the resulting feature vector is used as the embedding feature of the noise level.

[0016] Furthermore, the denoising process involves: first, merging and adjusting the geometric features and implicit representations through a Linear layer; then, adding the noise level features and text features to obtain... Then The output of the Linear layer is fed into the Transformer structure, and the output of the Transformer is passed through another Linear layer to obtain a clean implicit representation of the human body and objects; then, according to the diffusion model... Parameters, by noise level Add noise to obtain the intermediate motion sequence for the next input.

[0017] Step 2: Map the object geometric features, text features, noise level embedding features, and noisy motion sequences described in Step 1 to a unified feature space.

[0018] Step 3: Use a Transformer-based diffusion model backbone network to model the multimodal features. During the modeling process, load and fine-tune the pre-trained model to generate a single denoised original action, and combine it with the noise input to obtain the intermediate motion sequence for the next step.

[0019] Furthermore, the loading process for the pre-trained model is as follows:

[0020] a) Use a general HOI generation network with multi-step diffusion that has been trained on a human-object interaction dataset as initial parameters;

[0021] b) Modify the parameters related to the number of denoising iterations, changing the original parameters in the diffusion model. Parameter vector deletion is performed, and based on the requirements of the classic diffusion model, new denoising iterations and cosine curve calculations are used. The parameter vector is adapted to fit a low-step diffusion model.

[0022] c) Perform additional training based on b) to maintain the quality of the generated data.

[0023] Step 4: After the complete diffusion process, the obtained iteratively denoised original action is compared with the ground truth. The mean squared error (MSE) of the generated human sequence is calculated with the ground truth and used as the loss function. Then, the entire network is trained by backpropagation.

[0024] In the loss calculation, the monitoring signals specifically include:

[0025] a) Basic reconstruction supervision: Reconstruct human joints and objects from implicit representations and calculate the error MSE for each;

[0026] b) Monitoring of critical joints: Monitoring the position of critical joints such as the feet and hands;

[0027] c) Object key point supervision: Compare the positions of key points in the object's true base point set with those in the predicted pose;

[0028] d) Contact constraint supervision: Constrain the contact probability between the foot and the ground and the support height. Calculate the MSE by combining the true contact probability with the output contact probability, and then add the support height as the contact loss.

[0029] e) Foot sliding constraint supervision: When the foot contacts the ground, restrict its sliding in the horizontal direction. Extract the height of the toe and ankle from the implicit representation. If the toe height is lower than the ground height, the foot will be restricted from sliding in the horizontal direction. And the ankle is lower than If a foot is on the ground, then it is considered that the foot is touching the ground; if a foot is touching the ground, then the position difference in the xy direction between two adjacent frames is used as the sliding loss of this foot in this frame.

[0030] Reasoning stage

[0031] Step 5: In the reasoning stage, the input information obtained in Step 1 and the feature space obtained in Step 2 are used as inputs;

[0032] Step 6: Use the trained diffusion model to model the multimodal features, generate a single denoised original action, and combine it with the noise input to obtain the intermediate motion sequence for the next step, and repeat the denoising process.

[0033] Step 7: Input the geometric information of the interactive object, the text instructions for the interactive action, and randomly sampled Gaussian noise. After each step of the diffusion model is denoised, physical constraints are introduced for optimization.

[0034] Specifically, in physical constraint optimization, the clean implicit representation of the denoised output in this round is denoted as... ,Open The gradient is calculated, the loss is calculated according to the physical constraints, and then a backpropagation is performed based on the loss. The input vector after backpropagation correction is the vector obtained under the guidance of physical rationality. ;Will and Interpolation, then according to the diffusion model The parameters add noise to the interpolation result, which is then used as input to the next round of the denoising network.

[0035] The physical constraints of the inference phase include:

[0036] a) Contact constraint: guide the hand to make reasonable contact with the object, extract the contact mark of the hand in the implicit representation, and if the hand is marked as contact, then the distance between the hand and the nearest point in the object point cloud is taken as the contact loss of the hand.

[0037] b) Non-penetrating constraint: Prevents geometric penetration between the human body mesh and the object model. Specifically, the calculation of the non-penetrating constraint only applies to the mesh vertices that enter the object. The depth of the human body mesh vertices is sampled from the object's SDF, and a penalty is applied based on the penetration depth of the human body vertices to prevent geometric penetration between the human body mesh and the object model. The specific implementation method is as follows:

[0038] SDF construction: The symbolic distance field (SDF) is pre-calculated for the 3D model of each interactive object, and the object space is divided into voxel meshes;

[0039] Vertex sampling: During the inference phase, the human body mesh is sampled, and the vertex coordinates are transformed to the local coordinate system through geometric transformation. Then, the corresponding values ​​of the human body vertex coordinates in the object's SDF are queried.

[0040] Depth calculation: Vertices with SDF < 0 are considered as vertices that penetrate the interior. Depth as a vertex.

[0041] Penalty function: Apply a linear penalty to the vertex depth, or use a squared penalty to strongly suppress severe penetration.

[0042] Step 8: After the final round of denoising, output a 3D mesh sequence of human-object interaction in three-dimensional motion that conforms to semantic description and has physical rationality, including continuous changes in human posture and object pose.

[0043] This invention provides a method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints. It achieves the ability to maintain semantic consistency and physical rationality of the generated results while significantly reducing the number of diffusion steps, ensuring efficient and high-quality generation of human-object interaction motions. The final output of the human-object interaction 3D motion sequence is close to the multi-step diffusion model in terms of generation quality, while significantly improving the generation speed. Attached Figure Description

[0044] Figure 1 This is a flowchart of the synthesis method of the present invention;

[0045] Figure 2 This is a flowchart of the model training process of the present invention;

[0046] Figure 3 This is a flowchart of the model reasoning process of the present invention. Detailed Implementation

[0047] The following is in conjunction with the appendix Figure 1 , 2 The present invention will be further described in detail in sections 3 and 4 and the embodiments.

[0048] This invention is a method for rapidly synthesizing human-object interaction actions based on prior knowledge and physical constraints, including a training phase and an inference phase, as shown in Figure 1. The processing flow for obtaining input information in step (1) of the training phase of this invention is as follows:

[0049] S100: Collects the three-dimensional geometric information of interactive objects, and uses a basic point set method to represent it. It uniformly samples 100 three-dimensional points from the surface of the object as a reference point set to characterize the shape features of the object.

[0050] S110: Broadcast the basic point set along the time dimension to each frame of the motion sequence, and then obtain the low-dimensional geometric feature vector G through an MLP network;

[0051] S120: Obtain the text instruction describing the interactive action, and extract the text feature vector through the pre-trained CLIP model; here, the text feature vector has the same dimension as the geometric feature vector;

[0052] S130: Uses a noisy sequence of human-object interaction motions, implicitly including the vertex coordinates of the human body mesh and the pose parameters of the object. In each round of denoising, the noise level decreases progressively. The first round of denoising uses level 20, and the last round uses level 1. The current noise level is broadcast to each frame, with the same dimension as the text feature vector. This vector is then used to extract features through an MLP, and the resulting feature vector is used as the embedded feature of the noise level.

[0053] As shown in Figure 1, in steps (2) and (3) of the training phase of this invention, the processing flow of the denoising model for the input data is as follows:

[0054] S200: Embedded object geometric features, text features, and noise levels are concatenated with noisy motion sequence features and mapped to a unified feature space;

[0055] S210: The multimodal features are modeled using a Transformer-based diffusion model backbone network to generate a single denoised original action, which is then combined with the noise input to obtain the intermediate motion sequence for the next step.

[0056] Specifically, the geometric features and implicit representations are first merged and adjusted through a linear layer, and then the noise level features and text features are added together to obtain the joint features. Then The output of the Linear layer is fed into the Transformer structure, and the output of the Transformer is passed through another Linear layer to obtain the final implicit representation of the human body and objects.

[0057] S220: In each denoising step, the network predicts the current single denoising original action and combines it with the noise input to generate the intermediate motion sequence for the next step.

[0058] Specifically, if this is the final round of denoising, the clean implicit representation is directly used as the final generated result. Otherwise, the diffusion model is used. Add noise to obtain an implicit representation with noise, which is then used as input for the next round of denoising.

[0059] This invention sets the number of denoising steps in the training phase to 20, compressing the original model's 1000-step denoising process, and achieving efficient generation with fewer steps through parameter adjustment.

[0060] As shown in Figure 2, the loading process of the pre-trained model in step (3) of the training phase of this invention is as follows:

[0061] S300: Load the parameters of a multi-step diffusion model that has been trained on an existing task as initial weights. This model has been trained with 1000 steps of denoising.

[0062] S310: Modify the parameters related to the number of denoising steps in the initial weights to adapt to the 20-step denoising process, and perform additional training on this basis.

[0063] Specifically, the original parameters in the diffusion model will be... Parameter vector deletion is performed, and based on the requirements of the classic diffusion model, new denoising iterations and cosine curve calculations are used. Parameter vector.

[0064] As shown in Figure 2, the processing flow of steps (4) and (5) in the training phase of this invention is as follows:

[0065] S400: The network output contains the original actions of iterative denoising of the human body and objects, which is a clean implicit representation;

[0066] S500: Enable gradient calculation for all network parameters, use a 20-step diffusion process to infer the input information, and obtain the original iterative denoising action;

[0067] S510: After complete diffusion, the obtained iteratively denoised original action is compared with the ground truth. The mean squared error (MSE) of the generated human sequence and the ground truth is calculated and used as the loss function (Loss). Then, the entire network is trained by backpropagation.

[0068] As shown in Figure 3, the processing flow of steps (7) and (8) in the reasoning stage of this invention is as follows:

[0069] S600: Preprocessing network input. The input includes the 3D geometric information of the interactive object, processed in the same way as training phases S100-S110; text instructions describing the interaction intent are input, encoded using CLIP; random Gaussian noise is generated as the initial raw action, conforming to a standard normal distribution.

[0070] Similar to S210, a denoising network is used to denoise the input data, and a clean implicit representation is obtained after each round of denoising.

[0071] S700: After the final round of denoising, it outputs a 3D mesh sequence of human-object interaction in three-dimensional motion that conforms to semantic description and has physical rationality, including continuous changes in human posture and object pose.

[0072] Specifically, in physical constraint optimization, gradient calculation of the input vector is enabled, and the loss is calculated based on contact and penetration constraints. Then, backpropagation is performed based on the loss. The input vector corrected by backpropagation is the vector obtained under the guidance of physical rationality.

[0073] The implicit representation obtained after optimization is interpolated with the clean implicit representation output from this round of denoising, and then processed according to the diffusion model. The parameters add noise to the interpolation result, which is then used as input to the next round of the denoising network.

[0074] After the final round of denoising, the output is a 3D mesh sequence of human-object interaction in three-dimensional motion that conforms to semantic description and has physical rationality, including continuous changes in human posture and object pose.

[0075] Example 1

[0076] This invention is a method for rapidly synthesizing human-object interaction actions based on prior knowledge and physical constraints, comprising a training phase and an inference phase:

[0077] The publicly available Fullbody and 3D-FUTURE datasets are used. The datasets already include: static meshes of interactive objects; moving human meshes: i.e., T-frame human meshes and joint positions; contact information: whether the person's hands are in contact with objects and whether their feet are in contact with the ground in each frame. Vectors; the language that drives human-object interaction.

[0078] The first phase is the training stage:

[0079] 1. Load the pre-trained HOI action diffusion generation network weights. Obtain the trained weights (1000 epochs denoised) from an existing HOI project. Next, select 20 epochs for denoising and remove the original weights from the pre-trained weights.

[0080] Parameters, then use cosine to generate 20 rounds. Parameters; then load the modified weights.

[0081] 2. Obtain the geometric feature vector. Use the Poisson disk sampling algorithm: One hundred three-dimensional points are uniformly sampled from the surface of the object as a reference point set, i.e., one A vector. This vector is broadcast along the time dimension to T frames, i.e., a... The vector is then processed by an MLP network to extract a 1024-dimensional feature vector.

[0082] 3. Obtain text feature vectors. Download pre-trained CLIP weights from the publicly available CLIP project, and extract 1024-dimensional feature vectors from the text instructions describing the interaction actions using CLIP.

[0083] 4. Sample a value from a Gaussian distribution. A 2D vector is used as the initial implicit human body representation vector. The number of denoising rounds is set to 20, with the noise level in the first round starting at 20 and decreasing progressively.

[0084] 5. In the nth round of denoising, with noise level n, the implicit human body representation vector is first... The geometric features are fed into a linear layer, and the output vector, along with the denoising level, is fed into a Transformer network to output a clean human representation, denoted as . Then to Add noise of level n-1 to obtain the noise input for the next round of denoising. After 20 rounds of noise reduction, This is the final clean human body sequence.

[0085] 6. Next, regarding Supervised analysis is performed to obtain the merged vectors of corresponding moving human joints, contacts, and object translations and rotations from the dataset. For each frame:

[0086] a) Basic Reconstruction Supervision: Calculation Output and MSE, as the basic reconstruction loss .

[0087] b) Monitoring of key joints: i.e., the hand joint index is... The foot joint index is Therefore, the losses to the hands and feet are calculated as follows: .

[0088] c) Object keypoint supervision: Transform the object to the world coordinate system, uniformly sample 100 points from both the generated object and the dataset, and calculate the KL divergence between these two point sets as the object keypoint loss. .

[0089] d) Contact constraint supervision: The smaller value of the height of the generated bipedal joints is used as the support height. The contact information in the generated vector is then compared with the contact information in the dataset to calculate the MSE. The sum of the two values ​​is the contact constraint supervision result. .

[0090] e) Foot sliding constraint supervision: For the generated foot joints, calculate the sliding loss of the left and right feet respectively.

[0091] First, determine the grounding status of the foot: if the toe height of the foot is lower than... And the ankle height is lower than Therefore, it is assumed that this foot is on the ground. .

[0092] 7. For grounded feet, calculate the horizontal displacement between adjacent frames. So, slip loss .

[0093] The final total loss function for training is... .

[0094] 8. The training process is performed for 12,000 rounds with a batch size of 32. The final parameter weights are then used as inference weights.

[0095] Next, we proceed to the reasoning phase:

[0096] 9. During the inference phase, the processing and settings of the input data are the same as in steps 2-4, and 20 rounds of denoising are performed.

[0097] 10. For each object in the dataset, calculate in the object's local coordinate system. The SDF file is stored.

[0098] 11. In the nth round of denoising, with noise level n, first convert the implicit human body representation vector... The geometric features are fed into a linear layer, and the output vector, along with the denoising level, is fed into a Transformer network to output a clean human representation, denoted as . .

[0099] If the current noise level 0, then then for Adding noise of level n-1 yields This serves as the input for the next round of noise reduction.

[0100] If the current noise level ,Will Output after optimization guidance, according to weights and Interpolation obtained Then to Adding noise of level n-1 yields As the input for the next round of noise reduction. .

[0101] 12. Specifically, in the optimization guide: first open... The gradient is calculated, and for a single frame, the loss function is as follows:

[0102] Contact constraint: 100 point clouds are uniformly sampled from the object mesh, and the contact markers of the hand in the implicit representation are extracted. If the hand is marked as a contact, the distance between the hand joint and the nearest point in the object point cloud is used as the contact loss for this hand. .

[0103] Penetration Loss: The human joints are skinned using the publicly available SMPL model to obtain a complete human mesh. For each vertex on the human mesh, the negative value of the vertex in the object's SDF is used as the penetration loss, i.e. .

[0104] Ultimately As a guide loss, perform a backpropagation to correct it. .

[0105] 13. After performing 20 rounds of noise reduction, This is a clean implicit representation of the human body. Each frame... The joints were extracted, and SMPL skinning was used to obtain a complete and clean human motion mesh.

Claims

1. A method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints, characterized in that, It includes a training phase and an inference phase, with the specific steps as follows: Training phase Step 1: Obtain input information during the training phase, including: a) Obtain the 3D geometric information of the interactive object, represented using a basic point set method; then input the point set into the MLP to obtain the geometric features of the interactive object. ; b) Encode the text instructions describing the interactive actions into text feature vectors. ; c) Implicit representation of noisy human-object interaction motion sequences, including multi-frame human meshes and object poses. That is, a vector is sampled from the standard Gaussian distribution as the implicit representation of the first denoising. In each subsequent denoising round, the denoising result of the previous round is used as the input for the next round. d) Broadcast the current denoising level to each frame, with the denoising level vector having the same dimension as the text feature vector. Similarly, the denoised level vector is used to extract features through MLP, and the resulting feature vector is used as the embedding feature of the noise level. Step 2: Map the geometric features, text features, noise level embedding features, and noisy motion sequences of the interactive objects described in Step 1 to a unified feature space to obtain multimodal features; Step 3: Use a Transformer-based diffusion model backbone network to model the multimodal features. During the modeling process, load and fine-tune the pre-trained model to generate a single denoised original action, and combine it with the noise input to obtain the intermediate motion sequence for the next step. Step 4: After the complete diffusion, the obtained iteratively denoised original action is compared with the ground truth. The generated human sequence is compared with the ground truth to calculate the MSE, which is used as the loss. Then, the entire network is trained by backpropagation. Reasoning stage Step 5: In the reasoning stage, the input information obtained in Step 1 and the feature space obtained in Step 2 are used as inputs; Step 6: Use the trained diffusion model to model the multimodal features, generate a single denoised original action, and combine it with the noise input to obtain the intermediate motion sequence for the next step, and repeat the denoising process. Step 7: Input the 3D geometric information of the interactive object, the text command of the interactive action, and randomly sampled Gaussian noise. After each step of the diffusion model is denoised, physical constraints are introduced for optimization. Step 8: After the final round of denoising, output a 3D mesh sequence of human and object interaction three-dimensional motion that conforms to semantic description and has physical rationality, including continuous human posture changes and object pose changes.

2. The method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints according to claim 1, characterized in that, The three-dimensional geometric information of the interactive object is represented by a base point set consisting of multiple three-dimensional sampling points to characterize the shape and pose of the object. The base point set is defined in the object's local coordinate system. When the object's pose changes, all base point sets are transformed synchronously with the object's rotation matrix and translation vector, providing a geometric reference for subsequent motion prediction.

3. The method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints according to claim 1, characterized in that, The text instructions are extracted into fixed-dimensional feature vectors using a pre-trained text encoding model CLIP. These vectors have the same geometric feature dimensions as the interactive object.

4. The method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints according to claim 1, characterized in that, The denoising process is as follows: first, the geometric features and implicit representations are merged and adjusted through a Linear layer; then, the noise level features and text features are added together to obtain the desired result. Then The output of the Linear layer is fed into the Transformer structure, and the output of the Transformer is passed through another Linear layer to obtain a clean implicit representation of the human body and objects; then, according to the diffusion model... Parameters, by noise level Add noise to obtain the intermediate motion sequence for the next input.

5. The method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints according to claim 1, characterized in that, The loading process for the pre-trained model in step 3 is as follows: a) Use a general HOI generation network with multi-step diffusion that has been trained on a human-object interaction dataset as initial parameters; b) Modify the parameters related to the number of denoising iterations, changing the original parameters in the diffusion model. Parameter vector deletion is performed, and based on the requirements of the classic diffusion model, new denoising iterations and cosine curve calculations are used. The parameter vector is adapted to fit a low-step diffusion model. c) Perform additional training based on b) to maintain the quality of the generated data.

6. The method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints according to claim 1, characterized in that, In the loss calculation during the training phase, the supervision signal specifically includes: a) Basic Reconstruction Supervision: Reconstruct human joints and objects from implicit representations and calculate MSEs separately; b) Monitoring of critical joints: Monitoring the position of critical joints such as the feet and hands; c) Object key point supervision: Compare the positions of key points in the object's true base point set with those in the predicted pose; d) Contact constraint supervision: Constrain the contact probability between the foot and the ground and the support height. Calculate the MSE by combining the true contact probability with the output contact probability, and then add the support height as the contact loss. e) Foot sliding constraint supervision: When the foot contacts the ground, restrict its sliding in the horizontal direction. Extract the height of the toe and ankle from the implicit representation. If the toe height is lower than the ground height, the foot will be restricted from sliding in the horizontal direction. And the ankle is lower than If a foot is on the ground, then it is considered that the foot is touching the ground; if a foot is touching the ground, then the position difference in the xy direction between two adjacent frames is used as the sliding loss of this foot in this frame.

7. The method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints according to claim 1, characterized in that, The physical constraints of the inference phase include: a) Contact constraint: guide the hand to make reasonable contact with the object, extract the contact mark of the hand in the implicit representation, and if the hand is marked as contact, then the distance between the hand and the nearest point in the object point cloud is taken as the contact loss of the hand. b) Non-penetration constraint: Only for mesh vertices that enter the interior of the object, the depth of the human body mesh vertices is sampled from the object's SDF, and a penalty is applied based on the penetration depth of the human body vertices to prevent geometric penetration between the human body mesh and the object model.

8. The method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints according to claim 7, characterized in that, The non-penetrating constraint is implemented using the following method: SDF construction: The symbolic distance field (SDF) is pre-calculated for the 3D model of each interactive object, and the object space is divided into voxel meshes; Vertex sampling and depth calculation: During the inference phase, the vertices of the human body mesh are sampled. The vertex coordinates are transformed to the local coordinate system of the object through geometric transformations of rotation and translation. Then, the corresponding values ​​of the human body vertices in SDF are queried. Vertices with SDF<0 are retained as penalty objects to reduce the computational overhead of vertices in non-interactive areas. Penalty function: Apply a linear penalty to vertices with SDF < 0, or apply a squared penalty to strongly suppress severe penetration.

9. The method for rapid synthesis of human-object interaction actions based on prior knowledge and physical constraints according to claim 1, characterized in that, The physical constraint optimization is to denote the clean implicit representation of the current round of denoising output as... ,Open The gradient is calculated, the loss is calculated according to the physical constraints, and then a backpropagation is performed based on the loss. The input vector after backpropagation correction is the vector obtained under the guidance of physical rationality. ;Will and Interpolation, then according to the diffusion model The parameters add noise to the interpolation result, which is then used as input to the next round of the denoising network.

Images