Method for interacting objects by humanoid robot guided by diffusion model based on key frame

Abstract

The invention discloses a humanoid robot object interaction method guided by a diffusion model based on key frames. Comprising the following steps: S1, generating interaction between a human body and an object based on a diffusion model of a key frame; s2, guiding the humanoid robot to interact with the object based on the contact instruction; and S3, interaction between the humanoid robot and the object based on the action generation guidance. According to the method, in an action generation stage, key frames of a robot and an object within a period of time are predicted according to a current state, a motion sequence is recovered by using interpolation, and then a universal robot and object interaction strategy is finely adjusted based on a generated and simulated closed-loop structure and a fixed diffusion model weight. The interaction strategy is rewarded due to accurate simulation of aligned movement and contact information and successful object control, the gap between the output of the diffusion model and the training data set of the simulation environment is made up, and the reward does not need to be designed for a specific task.

Description

Technical Field

[0001] The present invention relates to the technical field of humanoid robot control, and in particular to a method for humanoid robot interaction with objects guided by a keyframe diffusion model. Background Art

[0002] Synthesizing realistic human actions in 3D environments is a core challenge in computer graphics, embodied artificial intelligence, and robotics. This task requires intelligent agents to precisely interact with objects in complex scenes, performing everyday activities with explicit intent, such as adjusting furniture, manipulating lighting, or organizing items. Unlike traditional motion generation methods, this task requires not only simulating human motion trajectories but also emphasizing the coordinated interaction between humans, objects, and the environment, guided by linguistic intent. Linguistic descriptions, as high-level semantic carriers, can clearly express interaction intent, providing critical guidance for building intelligent systems that can generate continuous action sequences. While some recent generative research has explored human-object interactions, they all generate simple actions based on static scenes (e.g., lying in bed), limited by the interactive properties of real-world objects. Even though some recent works consider egocentric 3D occupancy information and multiple action labels as input, object motion is still optimized through hand movements, which does not satisfy the laws of physics. Another approach proposes integrating the same object information as human-like motion at a noisy level. They use sparse waypoints as a condition for generating controllable human-object interactions without detailed hand motions. It is difficult for learning methods to be consistent with the real world; problems with foot sliding and penetration still exist. Therefore, subsequent work has adopted a coarse-to-fine model to reduce the unrealistic nature of hand motion prediction and used the output as the target of humanoid robots in physical simulation, with significant results. However, as the model size increases, the inference speed slows down, resulting in a large gap between generation and simulation. Meanwhile, a simple combination of a lightweight motion diffusion model and an imitation strategy can provide a more robust and physically consistent framework for humanoid robot motion generation. However, because the humanoid robot model used does not have hand joints such as fingers, it is still limited to simple interactions such as hitting objects.

[0003] On the basis of achieving behavior synthesis in virtual environments, how to transfer the learned interaction strategies to real robot systems becomes a key issue. To this end, a simulation training platform was built. Its core task is defined as: given an object mesh model and the key point positions of the target posture, control a bi-dexterous humanoid robot (153 degrees of freedom) to complete the complete operation process of object interaction-displacement-placement while maintaining the tracking accuracy of the reference human body movements (such as Figure 1(as shown). This technology can be widely used in AV / VR content generation and has the potential to be transferred to the field of physical robotics. However, this system faces three technical bottlenecks: 1) the contradiction between bipedal balance and high-degree-of-freedom precision manipulation; 2) stable interaction generation of diverse objects; and 3) the coupling of full-body kinematic chain dynamics and end-effector control. Existing solutions often adopt simplistic strategies: either simplifying the system to a floating hand model, sacrificing physical realism (directly controlling the hand root pose through virtual forces); or superimposing a hand controller on a body motion simulator. However, due to the 30mm-level hand tracking error (the latest approach), precise object manipulation is difficult to achieve. For example, the paper "Closd: Closing The Loop Between Simulation And Diffusion For Multi-task Character Control" proposes using the updated human state results from the simulation environment as input to a diffusion model for action generation to achieve closed-loop control of the robot for a simple task objective. The tasks and humanoid robot models used in Closd's work lack dexterous hands, so only simple interactive actions such as striking can be performed. The paper "Hoifi: Human-Object Interaction from Human-Level Instructions" proposes dividing motion generation into three stages: first, generating a rough general motion, then optimizing the hand motion based on existing physically credible interactive motion generation work, then using an optimization network to obtain a more detailed optimized motion sequence, and finally inputting the motion into the lower-level simulator to track the joints of the humanoid robot. Hoifi's work uses a large number of diffusion networks to optimize the entire motion generation process, resulting in a long time to generate a single motion. This inevitably results in a large time interval between downstream tracking. If the object's trajectory changes during this time, or if the robot loses its grip on the object and drops it, the motion sequence must be regenerated, which is time-consuming and cannot respond to dynamic objects in real time. Summary of the Invention

[0004] The purpose of the present invention is to overcome the problems of the above-mentioned prior art and provide a method for humanoid robot interaction with objects guided by a keyframe-based diffusion model. The upper-level generation is combined with an adaptive keyframe extraction method, which changes the input and output of the model from a constant 30-frame action to a prediction of keyframes over a sliding window length to reduce the prediction redundancy of the generated model. Based on a simulation environment, this application designs a strategic observation and reward method to achieve tracking of input robot actions, hand contact instructions, and the future trajectory of the object. Finally, the two are combined to achieve physically credible robot-object interaction in a simulation environment, requiring only the input of the machine's position in the first frame and the object's starting and ending positions.

[0005] To achieve the above object, the present invention provides a method for humanoid robot interaction with objects guided by a keyframe diffusion model, comprising the following steps:

[0006] The following steps are involved:

[0007] S1, keyframe-based diffusion model for human-object interaction generation;

[0008] S2, guiding the humanoid robot to interact with objects based on contact instructions;

[0009] S3. Humanoid robot interaction with objects based on action generation guidance.

[0010] Preferably, step S1 specifically includes the following steps:

[0011] S11. Implicitly encode the conditional input, extract implicit feature representations of the input text information, object geometry information and other task conditions through a predefined encoder, and use them as input conditions for the diffusion model;

[0012] S12. Extract key frames from the action sequences in the training data set. Use dynamic programming combined with joint weighted reconstruction error to extract action key frames from the complete human action sequence in the training data, emphasizing the motion precision control of the hands and feet.

[0013] S13. Integrate the model data used for training and construct training samples for key frame interpolation and reconstruction in a sliding window form. The first frame of each window is used as the current state, and the remaining key frames are filled in chronological order to construct a complete training sample.

[0014] S14, supervised training, based on the above interpolated and reconstructed data, adopts the diffusion model to perform denoising prediction training, and fits the real key frame distribution through multi-step reverse optimization;

[0015] S15. Adaptively reconstruct the model output, generate a series of key frames according to the initial conditions, and then restore the complete action through interpolation method according to the target frame rate.

[0016] Preferably, the encoding of the object geometry is obtained by constructing a direction vector through a base point set (BPS). First, a set of base points is sampled from a spherical volume with a radius of 1 meter. Then, for each sampled point, the minimum Euclidean distance to the nearest point on the object grid is calculated, and the direction vector from the base point to its nearest neighbor is recorded. The final BPS representation is recorded as Represents 1024 sampling points, each of which contains a vector indicating its spatial relationship with the object surface, and then projects the object BPS representation G into a low-dimensional vector, which is then broadcast to each frame, recorded as Adopting a masked motion data representation To represent the initial state and waypoint conditions, the initial state contains the human body posture and object posture of the first frame, the waypoint conditions consist of a series of 2D object positions every 30 frames and the 3D object position of the last frame, the rest of S is filled with zeros, and finally the encoded object geometry vector and the mask motion condition vector are connected.

[0017] Preferably, a loss function is introduced in the supervised training process to supervise the object motion prediction. The basic point set (BPS) representation is used to first calculate the nearest neighbor points of the object grid in the static posture for each fixed point set, and then 100 points are sampled from the 1024 nearest neighbor points to capture the rough outline of the object shape. These selected points are defined as Represents the selected object vertices in the static pose. The predicted object rotation and position are used to calculate the corresponding positions of these selected vertices at each time step of the model. The geometric loss of the object is calculated as follows:

[0018]

[0019] in and Denote the predicted object rotation and translation, K t represents the real vertex at time step t.

[0020] Preferably, the diffusion model uses a denoising network based on a transformer structure, and the diffusion model consists of a forward diffusion process and a backward diffusion process. The forward diffusion process gradually adds noise to the clean data, while the backward diffusion process is trained to reverse this process.

[0021] Preferably, step S2 specifically includes the following steps:

[0022] S21. Extract contact information from the dataset, combine the spatial contact data of the action and the object, extract contact information such as the human body contact position and temporal relationship, and construct a high-quality contact interaction supervision signal;

[0023] S22. Construct observations and rewards for reinforcement learning strategies, defining observation variables and multi-objective reward functions for strategy learning in a simulation environment;

[0024] S23. Use the proximal policy (PPO) to train the general robot-object interaction strategy.

[0025] Preferably, the definition process of the observed variables is as follows:

[0026] Based on proprioception and target observation Define Reward For training strategies, the proprioceptive Including the robot's own state, the target observation Includes simulated observation Object Observation and contact information guide c order , which simulates the observation It is defined as the difference between the current state and the reference state of the next frame, and is defined as follows:

[0027]

[0028] The contact information of the dataset is calculated. If the shortest distance between the point on the surface of the object and the human hand is less than a certain threshold, it is judged that contact has occurred. Finally, the object observation is defined as:

[0029]

[0030] It contains the reference object pose and the difference between the reference object trajectory and the current object state in the next φ frames, is the object shape latent code computed using the canonical object pose and basis point set (BPS), is the difference between the current object position and the position of each hand joint, c t is the contact force on all fingers, Indicates the position of an object, Indicates the direction of the object, The linear velocity of the object, Represents the angular velocity of the object.

[0031] Preferably, the multi-objective reward function is defined as follows:

[0032] Define action imitation reward:

[0033]

[0034] Where w represents the reward weight of each physical tracking quantity;

[0035] Define the physical trajectory tracking reward:

[0036]

[0037] The final reward function is defined based on the action imitation reward and the physical trajectory tracking reward:

[0038]

[0039] α represents the weight used to balance the action imitation reward and the physical trajectory tracking reward.

[0040] Preferably, step S3 specifically includes the following steps:

[0041] S31. Use the human action sequence output by the diffusion model as a reference action to fine-tune the general robot-object interaction strategy;

[0042] S32, using the generated human motion and object trajectory as inputs to the general robot-object interaction strategy, and inputting the generated target motion sequence and object motion trajectory into the general robot-object interaction strategy module in simulation tracking to guide the robot to imitate the target posture and achieve precise interaction;

[0043] S33. After updating for a period of time, the robot's status is fed back to the diffusion model to form a closed loop.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] This invention integrates humanoid robot motion tracking and contact perception to guide the robot to interact with objects in a specified manner. This not only guides the agent to follow the object's trajectory, but also guides the agent to align with the posture of a reference humanoid robot and make contact with the object at the appropriate time.

[0046] 2. This invention uses the generated data within a closed-loop architecture to fine-tune a generalized robot-object interaction policy. During training, the diffusion model weights are fixed, and tasks are randomly drawn in each episode, assigned different object trajectories, text prompts, and goal configurations. The generalized robot-object policy is rewarded for accurately imitating the diffusion-generated motion, successfully manipulating the object, and achieving physical realism, all without task-specific rewards or manual resetting during massively parallel reinforcement learning training. This bridges the gap between the diffusion model output and the training dataset of the simulated environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of an intelligent agent tracking a given action and completing object interaction in the background technology of the present invention;

[0048] Figure 2 The overall closed-loop block diagram provided by the present invention;

[0049] Figure 3 This is the structure diagram of the human motion generation + object trajectory prediction model of the present invention;

[0050] Figure 4 This is a structural diagram of the diffusion model based on the transformer structure of the present invention;

[0051] Figure 5 Schematic diagram of the reconstruction error of the key frame of the human motion sequence of the present invention;

[0052] Figure 6 This is a structural diagram of the robot-object interaction strategy of the present invention. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0054] Example 1

[0055] This embodiment provides a method for humanoid robot interaction with objects guided by a diffusion model based on key frames. For a given indoor scene, the user inputs a command, which is passed to a large language model to refine an abstract command to each specific step. A semantic segmentation method is used to locate the object position based on the target object in the text of the given command. The large language model gives the target object position through a text description of the scene. The initial state of the robot and the initial state of the object in the scene are used as inputs to the diffusion model for action generation. The model outputs the predicted key frame human body movements and the corresponding trajectory of the object within a period of time. The prediction is interpolated according to the expected output frequency and used as the target of the humanoid robot in the simulation environment. The humanoid robot tracks it. After tracking for a few frames, the state of the humanoid robot in the simulation environment and the state of the object after simulation are converted as inputs to the generation model, thereby forming a closed loop, including the following steps:

[0056] S1. Human-object interaction generation based on keyframe diffusion model.

[0057] The specific steps include:

[0058] 1. Implicitly encode the conditional input. For task conditions such as input text information, target description, or object geometry information, implicit feature representation is extracted through a predefined encoder and used as the input condition of the generative model.

[0059] 2. Extract key frames from the action sequences in the training dataset. Use dynamic programming combined with joint weighted reconstruction error to extract action key frames from the complete human action sequences in the training data, highlighting the motion precision control of the hands and feet.

[0060] 3. Integrate the model data used for training and construct training samples for keyframe interpolation and reconstruction in the form of sliding windows. The first frame of each window is used as the current state, and the remaining key frames are filled in chronological order to construct a complete training sample.

[0061] 4. Supervised training: Based on the above interpolated and reconstructed data, a diffusion model is used for denoising prediction training, and the real key frame distribution is fitted through multi-step reverse optimization.

[0062] 5. Adaptively reconstruct the model output during inference. Generate a series of keyframes based on the initial conditions, and then restore the complete motion through interpolation based on the target frame rate.

[0063] S2. Guide the humanoid robot to interact with objects based on contact instructions.

[0064] The specific steps include:

[0065] 1. Extract contact information from the dataset, combine the spatial contact data of the action and the object, extract contact information such as the human body contact parts and temporal relationships, and construct a high-quality contact interaction supervision signal.

[0066] 2. Construct observations and rewards for reinforcement learning strategies. Define observation variables (such as joint states, object poses, and contact information) and multi-objective reward functions for strategy learning in a simulation environment, including target error, imitation error, and contact success rate.

[0067] 3. Use the proximal policy (PPO) optimization method to train it. Use PPO and other reinforcement learning methods to train the humanoid robot control strategy based on the contact target to achieve learning and generalization of dynamic contact tasks

[0068] S3. Humanoid robot interaction with objects based on action generation guidance.

[0069] The specific steps include:

[0070] 1. Fine-tune the strategy based on the generated motion data, use the human motion sequence output by the diffusion model as the reference motion, and further improve the execution accuracy of the strategy through the PPO fine-tuning method.

[0071] 2. The generated human motion and object trajectory are used as strategy inputs. In simulation tracking, the generated target motion sequence and object motion trajectory are input into the control strategy module to guide the robot to imitate the target posture and achieve precise interaction.

[0072] 3. After a period of updating, the robot's state is fed back to the generation model to form a closed loop. During execution, the robot's current real state is periodically fed back to the generation model, forming a generation-control closed loop mechanism, improving the robustness and interactive stability of the overall system.

[0073] Example 2

[0074] The embodiment provides a method for generating human motion based on a diffusion model of key frames, such as Figure 3 As shown, it mainly includes the following steps:

[0075] Step 1: Object encoding.

[0076] In an embodiment of the present invention, the object point cloud is geometrically encoded and then transformed in dimension through a multi-layer perceptron, with the goal of encoding the three-dimensional xyz information of the point cloud into implicit features that are easier for the model to understand.

[0077] Step 2: Text encoding.

[0078] In the embodiment of the present invention, the input text instructions are language-encoded into implicit features, with the purpose of converting the text information into a one-hot vector and encoding it into implicit features that are easier for the model to understand.

[0079] Step 3: Extract key frames of human body and object action sequences.

[0080] In the embodiment of the present invention, an extraction evaluation error is constructed for a human motion sequence, and a dynamic programming algorithm is used to obtain the key frame index to be extracted based on the evaluation error.

[0081] Step 4: Generate model training.

[0082] In an embodiment of the present invention, after completing the pre-steps 1-3 above, it is necessary to construct the conditional input for the diffusion model and perform supervised training. The data is constructed by using a fixed-length sliding window to collect data from the dataset. The first frame at the sliding window position is retained at the corresponding dataset position. Subsequent frames are sequentially filled into the sliding window using the key frames extracted in step 3 in order of distance, forming a training sample. Subsequently, the data to be generated is masked using a conditional mask, typically retaining only the starting and ending positions, as well as path points where objects are sparse, as the conditional input for the diffusion model. The object features and text features extracted in steps 1 and 2 are replicated to the same length as the prediction sequence. Finally, random Gaussian noise, the replicated object and text features, and the conditional input are integrated along the last dimension and fed into the diffusion model. The Euclidean distance between the model's prediction and the true value data before the conditional mask is calculated as the training error. The error is then back-propagated to update the model parameters, thereby supervising model generation.

[0083] In an embodiment of the present invention, the object geometry encoding in the aforementioned step 1 is obtained by constructing a direction vector through a basis point set (BPS), the text feature encoding in the aforementioned step 2 is obtained through the comparative language-image pre-training (CLIP) method, the data set in step 3 uses the open source human-object interaction data set (OMOMO) of Stanford University, and the diffusion model in step 4 uses a denoising network based on a transformer structure.

[0084] In order to more clearly demonstrate the technical solution provided by the present invention, the method provided in this embodiment is described in detail below.

[0085] 1. Methods for representing human and object motion data

[0086] The present invention represents human body motion as Where T and D represent the time step and the dimension of the human body posture respectively. t The human pose corresponding to the t-th frame consists of global joint positions and 6D continuous rotations. The present invention adopts the widely used parameterized human body model SMPL-X to reconstruct the human body mesh from the pose and shape parameters.

[0087] To represent the motion of an object, the present invention uses two variables: global 3D position and relative rotation. The global position is represented by the center of mass of the object, while the relative rotation (denoted as R in the tth frame) is represented by the center of mass of the object. rel ) is the rotation relative to the input object geometry V, satisfying V t =R rel V, where V t represents the vertex of the object at the tth frame. The object motion is represented by

[0088] The object geometry is represented according to the basis point set (BPS) method. First, a set of basis points is sampled from a spherical volume with a radius of 1 meter. Then, for each sampled point, the minimum Euclidean distance to the nearest point on the object grid is calculated. At the same time, the direction vector from the basis point to its nearest neighbor is recorded. The final BPS representation is recorded as Represents 1024 sampling points, each of which contains a vector indicating its spatial relationship with the object surface. Project the object BPS representation G to a low-dimensional vector and then broadcast it to each frame, denoted as The present invention adopts a mask motion data representation method To represent the initial state and waypoint conditions, we use . The initial state contains the human pose and object pose in the first frame. The waypoint conditions consist of a series of 2D object positions every 30 frames and a 3D object position in the last frame. The rest of S is padded with zeros. The encoded object geometry vector and the mask motion condition vector are then concatenated and used as part of the denoising network input.

[0089] 2. Action Sequence Key Frame Extraction

[0090] The present invention extracts a set of sparse keyframes from a dense human motion sequence and then reconstructs it using an interpolation method. In this process, the present invention pays special attention to accurately recovering the fine details of key joints (such as hands and feet). For a human motion sequence J, the origin position of the human joint is defined as in Let K denote the key frame sequence, where K = {k0,…k L}, where k iRepresents the position of the human joints in the keyframe. The approximate trajectory is reconstructed by interpolating between keyframes That is, for the interpolation function The reconstruction error of each joint is assigned a weight w according to its importance i , the loss function is defined as:

[0091]

[0092] Where J is the set of human joints, j and Represent the true trajectory and interpolated trajectory of the joint respectively, is the distance metric, w i Importance weights defined manually or heuristically.

[0093] The present invention uses a dynamic programming algorithm to extract keyframe sequences. For an action sequence J, the start and end points are selected as keyframes, a storage set f(S) is constructed, an error threshold η is set, and a memoization table M is created to store the keyframe sets of the calculated trajectory subsegments. First, a check is made to determine whether the reconstruction error between the start and end points is less than the threshold. If not, all intervening keyframes are enumerated, and the one with the smallest error is selected as the middle segment. The trajectory is then divided into two segments, the front and back. The above judgment and segmentation process is recursively repeated for these two segments, and the resulting set of minimum keyframes is recorded. Dynamic programming is used to cache the processed sub-trajectories to avoid repeated calculations, and the extracted keyframes are finally output.

[0094] 3. Action Generation Based on Diffusion Model

[0095] The present invention uses the conditional diffusion model to generate synchronized object and human motion. In order to improve the authenticity of hand-object interaction, the model of the present invention also predicts the contact labels of the left and right hands. These predicted contact labels play a key role in the sampling process, ensuring that the hand-object contact in the generated motion sequence is more accurate and realistic. The complete data representation in the model is denoted as τ = {X, O, H}, which includes both motion and contact data.

[0096] The model's conditional signals, denoted as c, include the initial state, sparse object pathpoints, object BPS representations, and language descriptions. The diffusion model consists of a forward diffusion process and a backward diffusion process. The forward diffusion process gradually adds noise to the clean data τ0, while the backward diffusion process is trained to reverse this process. The forward diffusion process introduces noise via a Markov chain and proceeds for N steps. Its formalization is:

[0097]

[0098] where β nrepresents a fixed variance schedule, and I is the identity matrix. The goal of this invention is to learn a model p0 to reverse the forward diffusion process,

[0099] p θ (τ n-1 |τ u ,c):=N(τ n-1 ;μ θ (τ n ,n,c),∑ n )

[0100] where μ θ represents the predicted mean, ∑ n is a fixed variance. Learning the mean can be re-parameterized as learning to predict a clean data representation τ0. The objective is defined as:

[0101]

[0102] The present invention adopts a transformer architecture as a denoising network, such as Figure 4 The input includes the object geometry representation Noise data representation τ under mask motion condition S and noise level n n The input is projected into a series of feature vectors through a linear layer. A multi-layer perceptron (MLP) is used to embed the noise level n, and then the noise level embedding is combined with the language embedding to form a single embedding vector e n The embedding vector e n The dimension of is the same as the feature vector and is input into the Transformer together with these feature vectors. The final prediction is the updated feature vector after the linear layer transformer is changed (excluding the feature vector with e n The corresponding time step) is obtained.

[0103] 4. Model Training

[0104] During the training phase, a loss function is introduced to supervise the object motion prediction. Using the basis point set (BPS) representation, the nearest neighbor points of the object mesh in the static pose are first calculated for each fixed point set. Then 100 points are sampled from the 1024 nearest neighbor points to capture the rough outline of the object shape. These selected points are defined as Represents the selected object vertices at rest. At each time step of the model, the predicted object rotation (converted to relative rotation with respect to the object geometry at rest) and position are used to calculate the corresponding positions of these selected vertices. This is expressed by the following formula, where and Denote the predicted object rotation and translation, K t Represents the true vertex at time step t. The object geometry loss is calculated as follows:

[0105]

[0106] This loss function plays a key role in guiding the model to accurately predict object transformations. By minimizing the error between predicted and true vertices, the model is able to better learn the object's trajectory and pose changes, thereby improving the realism and accuracy of the generated motion.

[0107] During the training phase of the interactive synthesis model, no explicit contact constraints are enforced in the loss. Including loss terms such as the hand-object contact loss and the object-floor penetration loss poses challenges for training. First, these types of loss terms are expensive to compute and can significantly slow down training. Second, introducing more loss terms requires careful balancing of the different losses, which often requires retraining the model with different settings. In contrast, enforcing these constraints during testing is more flexible and makes it easier to choose appropriate weights for the different terms.

[0108] In the diffusion model framework, classifier guidance is often applied during testing to control the generation process to meet specific goals or constraints. The typical way to apply classifier guidance is to perturb the noisy prediction mean at each denoising step. This can be done using the formula Denote where μ represents the predicted mean of the defined denoising step n. F represents a learning function or analytical function that determines how much penalty should be imposed on the predicted mean and α represents the strength of the perturbation. This guidance computes the gradient with respect to the noise mean and requires F to be trained on noisy data or to be a deterministic function designed for noisy data. Another approach is reconstruction guidance, which has been shown to be effective in controlling the generation process in previous work. Instead of perturbing the noise mean, it uses a representation τ with respect to the noisy input data n The gradient of perturbs the clean data representation of the prediction The process is formally expressed as:

[0109]

[0110] 5. Action Generation Indicator Verification

[0111] In order to verify the quality of the generated data, the following indicators are used to evaluate it from various aspects.

[0112] 1. Conditional Match: This metric calculates the Euclidean distance between the predicted and input object path points. It includes the start and end position errors (T s ,T e ) and path point error (T xy ).

[0113] 2. Human motion quality: This metric includes foot sliding score (FS), foot height (H feet ). FS is the weighted average of the cumulative translation in the xy plane and is expressed in centimeters (cm) according to previous studies. feet Assess the height of the foot, also in centimeters.

[0114] 3. Interaction quality: This metric evaluates the accuracy of hand-object interaction, including contact and penetration. For contact accuracy, the accuracy (C prec ), recall rate (C rec ) and F1 score (C F1 ) indicator. In addition, it also includes the contact percentage (C ％ ), which is determined by the proportion of frames in which contact is detected. To calculate the penetration score (P hand ), hand V i Each vertex of is used to query the pre-computed signed distance field (SDF) of the object. This process generates a corresponding distance value d for each vertex. i The penetration score is then calculated by averaging the negative distance values ​​(indicating penetration), formalized as The unit is centimeters (cm).

[0115] 4. Ground truth (GT) difference: This metric measures the deviation between the generated result and the ground truth motion. It includes the mean per joint position error (MPJPE), root joint translation error (T root ) and object position error (T obj ), are calculated using the Euclidean distance (in centimeters (cm)) between the predicted and actual ground truth positions. In addition, this metric also includes the root joint orientation error (O root ) and object orientation error (O obj ). These errors are calculated using the Frobenius norm of the rotational differences, as where R pred and R gt denote the predicted and true rotation matrices respectively.

[0116] As no previous work has addressed our task, we adapt related work, such as InterDiff, MDM, and OMOMO, to establish a baseline for comparison. InterDiff focuses on predicting human-object interactions using the previous 10 frames. MDM generates human motion based on language descriptions. OMOMO synthesizes human motion based on provided object motion trajectories. We adapt InterDiff to accept additional input conditions, including text and sparse waypoints. For MDM, we update the model to incorporate our object geometry representation and sparse waypoints. Furthermore, we enhance MDM to include object motion representation as an additional output. OMOMO requires a sequence of object states to generate full-body human poses; therefore, we implement a linear interpolation strategy for object positions based on provided start and end positions and predefined waypoints in the xy plane, while maintaining object rotation consistent with the initial frame throughout the sequence. Furthermore, as part of an ablation study, we introduce two variants, Pred-OMOMO (Pred-OM) and GT-OMOMO (GT-OM). Pred-OMOMO combines a text-conditioned object motion synthesis module with OMOMO. GT-OMOMO uses ground truth object motion as input to OMOMO. As well as the recent CHOIS method for controllable human motion generation. In addition, the present invention evaluates the importance of weighting the hands and feet in keyframe extraction based on the ablation method, where w / ow i The denoising step number of the diffusion model is shown in Table 1.

[0117] Table 1 Comparison of action generation model indicators

[0118]

[0119] Experimental results show that the proposed method (OURS) consistently outperforms the baseline in multiple evaluation aspects. It achieves higher condition matching, better hand-object interaction quality, and lower deviation from the true value motion. These results demonstrate the effectiveness of the keyframe-based generation and simulated perception design of the present invention in generating more accurate and physically accurate human-object interactions. The subscript on OURS represents the number of denoising steps in the diffusion model. As can be observed from the table, increasing the number of steps results in little performance improvement, which indicates that the model performance reaches saturation after exceeding a certain step size. w / ow iThe variant represents a model that does not consider the weights of the hands and feet during keyframe extraction. This leads to a significant drop in performance on multiple metrics. In this case, the hand penetration score improves, which can be attributed to the model's failure to capture detailed hand trajectories, especially at critical moments such as contact, resulting in output motion farther from the object and thus less penetration. Meanwhile, due to their inherent limitations, interpolation-based methods perform poorly on physics-related metrics, especially those related to foot motion. When static frames are filtered out based on velocity, interpolation often introduces subtle motion noise, resulting in higher errors and decreased physical plausibility. However, the main focus at this stage is on other metrics. Once motion is integrated into the simulation environment, the physics-related metrics improve significantly.

[0120] Example 3

[0121] This embodiment provides a strategy for a general humanoid robot to interact with objects in a simulated environment. This strategy utilizes established proximal optimization methods and the prior distribution of massive robot actions to accelerate training. This section describes the state and reward design of the strategy, which are the main contributions of this invention.

[0122] 1. Human body representation method and data definition in simulation environment

[0123] In the simulation environment, the human body posture is defined as in Represent the 3D joint rotations of all J joints (including the hands and body) using a 6-degree-of-freedom (6-DOF) rotation representation, Indicates the 3D position of the joint. In order to define the human body velocity information We have in represents the angular velocity, Represents linear velocity. The difference from the generative model is that the human joints are represented as global quantities here, while they are local quantities in the generative model.

[0124] For objects, use the object position direction Linear speed and angular velocity To define its 3D trajectory It should be noted that the quantities represented in the simulation are global physical quantities, which are different from those in the generative model. In the generative model, the relative rotation representation is performed based on the random frame and the object position is normalized based on the first frame. As a symbolic convention, we use represents kinematic quantities from a motion capture (MoCap) dataset, while plain notation (no additional sign) represents values ​​from a physics simulation. Represents a dataset containing diverse object meshes.

[0125] 2. Policy Observation and Reward Function

[0126] This invention integrates humanoid robot motion tracking and contact perception to guide the robot to interact with objects in a specified manner. Specifically, it not only guides the intelligent agent to follow the object's trajectory, but also guides the intelligent agent to align with the posture of a reference humanoid robot and make contact with the object at the appropriate time.

[0127] Based on proprioception and target observation We define rewards Used to train the policy. Use Proximal Policy Optimization (PPO) to maximize the discounted reward

[0128] Proprioception includes some of the robot's own states Where β represents the shape parameter of the human body and represents the length of each link of the robot. This includes simulated observation Object Observation and contact information guide c order . Among them, the simulated observation It is defined as the difference between the current state and the reference state of the next frame, and is defined as follows:

[0129] In order to facilitate the subsequent combination of generated outputs, the contact information of the data set is calculated. If the shortest distance between the point on the surface of the object and the human hand is less than a certain threshold, it is judged that contact has occurred. The observation of the strategy is represented by a 2D 01 code c order =(0 / 1,0 / 1), where 0 indicates no contact behavior and 1 indicates contact behavior.

[0130] To provide the mission strategy with information about the object and the desired trajectory, object observations are defined as:

[0131]

[0132] This contains the reference object pose and the following The difference between the reference object trajectory and the current object state for a frame. is the object shape latent code computed using the canonical object pose and the basis point set (BPS). is the difference between the current object position and the position of each hand joint. The last term c t is the contact force on all fingers.

[0133] The reward of the strategy is an important part of guiding the model in the direction we expect during training. In order to encourage the robot to follow the given reference action to approach the object before contact, an action imitation reward is given:

[0134]

[0135] This enables humanoid robots to accurately track the position, orientation, linear velocity, and angular velocity of reference joints over time. This can guide the character to always stay close to the generated motion plan and reduce artifacts such as jitter or unnatural transitions, where w represents the reward weight of each physical tracking volume.

[0136] In order to encourage the robot to move the object along the specified trajectory after contacting the object, an object trajectory tracking reward is given:

[0137]

[0138] The difference between the current and reference object poses is calculated and filtered by an indicator variable 1{C} which is set to true if the object is in contact with the humanoid robot's hand. Reward 1{C}·w c Encourage the humanoid robot's hands to make contact with objects.

[0139] The final reward uses a weight α to balance object tracking and motion imitation, ensuring that the humanoid robot can not only successfully interact with the object but also maintain humanoid-like motion throughout the interaction process.

[0140]

[0141] 3. Comparison of Action Physics Consistency Indicators in Physical Simulation

[0142] There are a lot of phenomena that do not conform to physical characteristics in the generated actions, such as the translation of footsteps in the air and the penetration of hands and objects. In order to compare the benefits of introducing a physical engine, the present invention saves the action data of the robot in the simulation environment and calculates the physical consistency indexes using the same method: footstep slip index FS, footstep height H feet And the hand's penetration index P hand , the results are shown in Table 2.

[0143] Table 2 Comparison of physical consistency indicators in simulation environment

[0144]

[0145] Experimental results show that combining with the physics engine can significantly improve the physical realism of human motion, especially reducing foot sliding and hand object penetration artifacts.

[0146] Example 4

[0147] This embodiment provides a structure for generating physically credible robot-object interactions under effective generation guidance, such as Figure 1 As shown, the pipeline begins with a conditional diffusion model that generates sparse keyframes of people and objects. These keyframes are linearly interpolated based on motion amplitude to produce a smooth, dense sequence. A short segment is then passed to a general robot-object interaction policy for execution. At each step, the reinforcement learning policy tracks the interpolated pose and interacts with the object. After executing a segment, the final, latest frame is converted back into the diffusion input and used as a prefix for the next generation, thus forming a closed loop between generation and physics-based execution.

[0148] To bridge the gap between the output of the generative model and the training dataset from the simulation environment, the present invention uses the generated data within a closed-loop architecture to fine-tune a generalized robot-object interaction policy. During training, the diffusion model weights are fixed, and tasks are randomly drawn in each episode, assigned different object trajectories, text prompts, and goal configurations. The generalized robot-object policy is rewarded for accurately imitating the diffusion-generated motion, successfully manipulating objects, and achieving physical realism, all without the need for task-specific rewards or manual resetting during massively parallel reinforcement learning training.

[0149] 1. Evaluation Indicators of Robot-Object Interaction Strategies

[0150] In order to compare different combination methods of generation and simulation, the present invention adopts evaluation indicators, including position error (E pos ,mm), rotation error (E rot ,rad) and physics-based metrics such as velocity error (E vel , mm / frame) and acceleration error (E acc ,mm / frame 2 To evaluate the interaction performance, the contact success rate (Succ cont ), the interaction is considered successful if the object contacts the correct part of the body in the simulation.

[0151] The present invention also reports the trajectory target reaching rate (TTR), which measures the proportion of time steps in which the object is within 12cm of the target position. The success rate (Succ tgt ), which is used to evaluate whether the object has reached its final position.

[0152] The compared methods include CHOIS, which generates controlled human motions, coupled with the general SMPL-X model robot tracking method PHC-X, which tracks the joint points output by CHOIS, and OmniGrasp, a method for tracking diverse grasping strategies along trajectories. Because the human motions generated by CHOIS lack detailed hand grasping, the present invention simply uses GRIP, a method for generating interactive gestures using spatial cues and potential consistency, to generate detailed hand grasping. Finally, the object interaction capabilities (pickup and placement) of each method are compared in Table 3.

[0153] Table 3 Comparison of indicators of different methods combining generation and simulation environment

[0154]

[0155] Experimental results show that directly tracking motion generated by humanoid robots often fails to achieve reliable object interaction due to its high sensitivity to motion accuracy. OmniGrasp demonstrates robust grasping trajectory tracking capabilities; however, due to its pre-defined reward design, which encourages the agent to maintain constant hand contact with the object, this method loses the ability to perform natural pick-and-place behaviors, and interaction is limited to the hands, preventing interaction with other body parts. The method presented in this paper is capable of performing more general humanoid robot-object interactions, even enabling interaction with parts other than the hands, such as kicking.

[0156] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the invention. It should be understood that many modifications may be made to the illustrative embodiments, and that other arrangements may be devised, without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in ways other than those described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be employed in conjunction with other described embodiments.

Claims

1. A method for humanoid robot interaction with objects guided by a keyframe diffusion model, characterized in that: The following steps are involved: S1, keyframe-based diffusion model for human-object interaction generation; S2, guiding the humanoid robot to interact with objects based on contact instructions; S3. Humanoid robot interaction with objects based on action generation guidance.

2. The method for humanoid robot interaction with objects guided by a keyframe diffusion model according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11. Implicitly encode the conditional input, extract implicit feature representations of the input text information, object geometry information and other task conditions through a predefined encoder, and use them as input conditions for the diffusion model; S12. Extract key frames from the action sequences in the training data set. Use dynamic programming combined with joint weighted reconstruction error to extract action key frames from the complete human action sequence in the training data, emphasizing the motion precision control of the hands and feet. S13. Integrate the model data used for training and construct training samples for key frame interpolation and reconstruction in a sliding window form. The first frame of each window is used as the current state, and the remaining frames are filled in chronological order to construct a complete training sample. S14, supervised training, based on the above interpolated and reconstructed data, adopts the diffusion model to perform denoising prediction training, and fits the real key frame distribution through multi-step reverse optimization; S15. Adaptively reconstruct the model output, generate a series of key frames according to the initial conditions, and then restore the complete action through interpolation method according to the target frame rate.

3. The method for humanoid robot interaction with objects guided by a keyframe diffusion model according to claim 2, characterized in that: The encoding of the object geometry is obtained by constructing a direction vector through a base point set (BPS). First, a set of base points are sampled from a sphere with a radius of 1 meter. Then, for each sampled point, the minimum Euclidean distance to the nearest point on the object grid is calculated, and the direction vector from the base point to its nearest neighbor is recorded. The final BPS representation is recorded as Represents 1024 sampling points, each of which contains a vector indicating its spatial relationship with the object surface, and then projects the object BPS representation G into a low-dimensional vector, which is then broadcast to each frame, denoted as Adopting a masked motion data representation To represent the initial state and waypoint conditions, the initial state contains the human body posture and object posture of the first frame, the waypoint conditions consist of a series of 2D object positions every 30 frames and the 3D object position of the last frame, the rest of S is filled with zeros, and finally the encoded object geometry vector and the mask motion condition vector are connected.

4. The method for humanoid robot interaction with objects guided by a keyframe diffusion model according to claim 3, characterized in that: A loss function is introduced in the supervised training process to supervise the object motion prediction. The basic point set (BPS) representation is used to first calculate the nearest neighbor points of the object grid in the static posture for each fixed point set, and then 100 points are sampled from the 1024 nearest neighbor points to capture the rough outline of the object shape. These selected points are defined as Represents the selected object vertices in the static pose. The predicted object rotation and position are used to calculate the corresponding positions of these selected vertices at each time step of the model. The geometric loss of the object is calculated as follows: in and Denote the predicted object rotation and translation, K t represents the real vertex at time step t.

5. The method for humanoid robot interaction with objects guided by a keyframe diffusion model according to claim 4, characterized in that: The diffusion model uses a denoising network based on a transformer structure. The diffusion model consists of a forward diffusion process and a backward diffusion process; the forward diffusion process gradually adds noise to the clean data, while the backward diffusion process is trained to reverse this process.

6. The method for humanoid robot interaction with objects guided by a keyframe diffusion model according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21. Extract contact information from the dataset, combine the spatial contact data of the action and the object, extract contact information such as the human body contact position and temporal relationship, and construct a high-quality contact interaction supervision signal; S22. Construct observations and rewards for reinforcement learning strategies, defining observation variables and multi-objective reward functions for strategy learning in a simulation environment; S23. Use the proximal policy (PPO) to train the general robot-object interaction strategy.

7. The method for humanoid robot interaction with objects guided by a keyframe diffusion model according to claim 5, characterized in that: The definition process of the observed variables is as follows: Based on proprioception and target observation Define the reward function For training strategies, the proprioceptive Including the robot's own state, the target observation Includes simulated observation Object Observation and contact information guide c order , which simulates the observation It is defined as the difference between the current state and the reference state of the next frame, and is defined as follows: Where θ represents the current joint angle of the robot, p represents the current joint position of the robot, v represents the current joint velocity of the robot, and ω represents the current joint angular velocity of the robot. It is represented as a given target quantity, and the subscript represents the time step; The contact information of the dataset is calculated. If the shortest distance between the point on the surface of the object and the human hand is less than a certain threshold, it is judged that contact has occurred. Finally, the object observation is defined as: It contains the reference object pose and the difference between the reference object trajectory and the current object state in the next φ frames, is the object shape latent code computed using the canonical object pose and basis point set (BPS), is the difference between the current object position and the position of each hand joint, c t is the contact force on all fingers, Indicates the position of an object, Indicates the direction of the object, The linear velocity of the object, Represents the angular velocity of the object.

8. The method for humanoid robot interaction with objects guided by a keyframe diffusion model according to claim 6, characterized in that: The definition process of the multi-objective reward function is as follows: Define action imitation reward: Where w represents the reward weight of each physical tracking quantity, and the subscripts p, r, v, ω, and c represent position, rotation, linear velocity, angular velocity, and contact information, respectively. The variable ·' in the formula indicates that the key joints of the robot are taken out for calculation; Define the physical trajectory tracking reward, where the variable definition is consistent with the robot, adding superscript · obj The variable representing the object, where 1{C} represents the mask at the moment of contact. It is 1 at the time of contact, indicating that no object reward is added before contact, so that the robot focuses on imitating the joint points. c Represents the reward weight for contact information, which encourages the robot to make contact when necessary: The final reward function is defined based on the action imitation reward and the physical trajectory tracking reward: α represents the weight used to balance the action imitation reward and the physical trajectory tracking reward.

9. The method for humanoid robot interaction with objects guided by a keyframe diffusion model according to claim 1, characterized in that: Step S3 specifically includes the following steps: S31. Use the human action sequence output by the diffusion model as a reference action to fine-tune the general robot-object interaction strategy; S32, using the generated human motion and object trajectory as inputs to the general robot-object interaction strategy, and inputting the generated target motion sequence and object motion trajectory into the general robot-object interaction strategy module in simulation tracking to guide the robot to imitate the target posture and achieve precise interaction; S33. After updating for a period of time, the robot's status is fed back to the diffusion model to form a closed loop.

Images