Robot dynamic interaction skill learning method and system based on monocular video

Abstract

The invention provides a robot dynamic interaction skill learning method and system based on a monocular video, and relates to the technical field of robot control and computer vision crossing. According to the method, the end-to-end, high-fidelity and high-robustness imitation learning ability of the physical robot for executing the dynamic interaction task from monocular vision observation is realized. According to the method, the objective function is optimized, so that the reconstruction result is ensured not only to be consistent with the video visually, but also to be reasonable and achievable physically, and the reconstruction result which is highly consistent physically provides high-quality input for action redirection. In the action redirection process, space-time constraints for interactive contact events are introduced, and it is ensured that the generated robot reference joint trajectory meets physical executable conditions at key interactive moments. Finally, a gap from simulation to reality is reduced by learning a training control strategy, and end-to-end simulation learning from monocular vision observation to physical robot execution is realized.

Description

Technical Field

[0001] This invention relates to the field of robot control and computer vision interdisciplinary technology, and in particular to a robot dynamic interactive skill learning method and system based on monocular video. Background Technology

[0002] In the interdisciplinary field of robotics and computer vision, researchers explore using monocular or multi-view videos as input to extract motion trajectories for robots to imitate, and combining this with simulation training to achieve behavior transfer. Typical techniques include reference trajectory-based imitation learning (such as the DeepMimic framework) and the "real-to-simulation-to-real" paradigm (such as VideoMimic). The latter generates physically plausible simulation data by jointly reconstructing human motion and scene geometry from monocular videos, thereby training control policies that enable robots to perform complex actions dependent on environmental interaction (such as climbing stairs). Control policy optimization often employs reinforcement learning algorithms (such as PPO) combined with control policy distillation (such as DAgger) to reduce dependence on reference trajectories and improve generalization ability.

[0003] However, existing technologies have significant limitations in dynamic interactive tasks such as robot soccer: First, monocular video reconstruction suffers from scale ambiguity. Existing methods often rely on the prior average human height for scale normalization, an assumption that is highly unreliable under conditions of large individual differences, pose occlusion, or long-distance shooting, leading to distortion in subsequent dynamic simulations. Second, there is a lack of explicit modeling for dynamic interactive objects (such as spheres). Existing video-to-sim methods primarily target static environments and fail to recover the kinematic and dynamic states of dynamic objects like spheres, making it difficult to handle interactive tasks. Third, contact constraints are incomplete. The redirection process focuses primarily on foot-to-ground contact, lacking explicit constraints on the timing, position, and direction of human-ball / robot-ball contact, resulting in the inability to execute key interactive actions. Fourth, there is excessive reliance on perception and insufficient robustness. Control strategy deployment often requires high-precision motion capture systems (MoCap), depth maps, or fine scene meshes, whose performance degrades sharply under real-world sphere detection noise and latency.

[0004] Therefore, there is an urgent need for a method that can reliably learn dynamic interaction skills from monocular videos and can be robustly executed in real-world environments. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for learning dynamic interaction skills in robots based on monocular video. The technical problem to be solved is that robots in the prior art struggle to learn and robustly execute dynamic physical interaction skills from monocular video. This invention aims to jointly and accurately reconstruct physically plausible scaled models of the human body, dynamic objects, and scenes from monocular video lacking absolute scale information. Based on these models, physically executable robot control commands are generated, and ultimately, through training, the robot can robustly complete similar dynamic interaction tasks.

[0006] On one hand, this invention provides a method for learning dynamic interactive skills of robots based on monocular video, including the following steps: S100, Video parsing: Processing the input monocular video to extract key human body information and dynamic object information, and using a reference object with known physical size in the video to calculate the reconstruction scale of the monocular video to obtain scaled preliminary three-dimensional data; S200, Joint reconstruction: Using the preliminary three-dimensional data as input, by optimizing the objective function, jointly reconstructing the human body motion trajectory, dynamic object motion trajectory, and scene geometric model that are consistent with the standard scale of the dynamic object and are physically reasonable; S300, Action redirection: Mapping the reconstructed human body motion trajectory to the joint space of the target robot to generate robot reference joint trajectory, and introducing spatiotemporal constraints on the interaction contact events between the human body and the dynamic object during the mapping process; S400, Control strategy training: In a simulation environment, based on the robot reference joint trajectory and scene geometric model, training the control strategy through reinforcement learning; S500, Deployment and execution: Deploying the trained control strategy to the target robot, enabling the target robot to perform dynamic interactive tasks similar to the motion trajectory of the dynamic object.

[0007] Optionally, step S100 specifically includes the following steps: S101, extracting two-dimensional human keypoints from each frame of the monocular video using a human pose detection network; S102, estimating the human model parameters and three-dimensional joint positions relative to the camera for each frame using a monocular human reconstruction network based on the two-dimensional human keypoints; S103, obtaining the pixel trajectory of dynamic objects in the monocular video using object detection and tracking algorithms; S104, restoring the camera pose and generating a depth map based on structure of motion reconstruction and / or monocular depth estimation techniques; S105, standardizing object detection and feature extraction; S106, preliminary scale calculation and robustness enhancement.

[0008] Optionally, in step S200, the optimization variables of the objective function include global human body translation, global human body rotation, local joint pose of the human body, world coordinate trajectory and velocity vector of the dynamic object, and scene scale factor; the objective function includes a weighted sum of the following terms: reprojection error term between the human body 3D mesh calculated based on human body model parameters and the preliminary 3D data, and physical consistency term; the physical consistency term includes at least one of the following: the dynamic residual between the motion trajectory of the dynamic object and the motion trajectory of the dynamic object calculated based on the rigid body dynamics equation, the non-penetrating constraint between the human body 3D mesh and the scene geometric model, and the human body self-collision avoidance constraint.

[0009] Optionally, in the physical consistency term, the dynamic residual is calculated by minimizing the difference between the observed trajectory of the dynamic object and the trajectory generated by the physical simulation, and a momentum conservation constraint is introduced at the moment of the contact event.

[0010] Optionally, in step S300, the robot reference joint trajectory includes: the joint angles of the target robot and the root pose of the target robot; the spatiotemporal constraint is an optimization cost function, which includes a weighted sum of the following terms: the pose consistency term between the robot reference joint trajectory and the reconstructed human motion trajectory, the foot-ground contact point matching term, the position consistency term between the robot end effector and the dynamic object when an interactive contact event occurs, and the robot self-collision and world collision penalty term.

[0011] Optionally, in step S400, the training of the control strategy adopts a phased training approach, including: a motion capture pre-training phase: using a motion capture dataset to initially train the control strategy; a scene-conditional tracking phase: in a simulation environment, loading a scene geometry model and using the robot reference joint trajectory as a supervision signal to train the control strategy to track and reproduce the target behavior; a control strategy distillation phase: a teacher's control strategy guides the student's control strategy to learn to predict actions from local observation information, gradually reducing the dependence on the robot reference joint trajectory; and an underconditional reinforcement learning fine-tuning phase: under the condition of removing the robot reference joint trajectory as input, the control strategy is fine-tuned, and domain randomization is introduced.

[0012] Optionally, domain randomization includes randomly perturbing at least one of the following parameters: joint friction coefficient, mass of the dynamic object, sensor detection delay, sensor measurement noise, control command delay, and terrain height map resolution.

[0013] Optionally, the reward function of the control strategy includes a weighted sum of the following: a tracking reward for the robot's reference joint trajectory; and a task reward for interacting with a dynamic object, including: a contact reward when a collision between the foot and the dynamic object is detected; a directional reward when the direction of motion of the dynamic object after being knocked out is consistent with the target direction; and a hit reward when the dynamic object enters the target area.

[0014] Optionally, the deployment and execution steps also include running an independent safety monitoring module, which is configured to monitor the joint motor current and / or output torque of the robot in real time, and trigger a hardware protection mechanism if the safety threshold is exceeded. The robot's zero-torque point (ZMP) and / or capture point (CP) are calculated in real time to assess the robot's stability. If the robot is determined to be about to become unstable, a compliant tipping control strategy is triggered.

[0015] On the other hand, the present invention provides a robot dynamic interactive skill learning system based on monocular video, comprising: a processor and a memory; the memory stores computer program instructions, which, when executed by the processor, cause the system to perform the method described above.

[0016] Implementing this invention offers the following advantages: It achieves end-to-end, high-fidelity, and robust imitation learning capabilities from monocular vision observation of a physical robot performing dynamic interactive tasks. By optimizing the objective function, this invention ensures that the reconstruction results are not only visually consistent with the video but also physically reasonable and achievable. This highly consistent physical reconstruction provides high-quality input for action retargeting. During action retargeting, by introducing spatiotemporal constraints for interactive contact events, it ensures that the generated robot reference joint trajectories meet physically executable conditions at key interaction moments. Finally, by learning and training control strategies, the gap between simulation and reality is reduced, achieving end-to-end imitation learning from monocular vision observation of a physical robot performing tasks. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a robot dynamic interaction skill learning method based on monocular video in one embodiment. Figure 2 This is a schematic diagram of the process for obtaining preliminary 3D data based on monocular video in one embodiment. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0019] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0020] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0021] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0022] Example 1 In this embodiment, as Figure 1 The method for learning dynamic interactive skills in robots based on monocular video, as shown, includes the following steps: S100, Video Analysis: Process the input monocular video, extract human key point information and dynamic object information, and use reference objects with known physical dimensions in the video to calculate the reconstruction scale of the monocular video and obtain scaled preliminary three-dimensional data. S200, Joint Reconstruction: Using preliminary 3D data as input, the objective function is optimized to jointly reconstruct human motion trajectories, dynamic object motion trajectories, and scene geometric models that are consistent in scale and physically reasonable. S300, Motion Redirection: The reconstructed human motion trajectory is mapped to the joint space of the target robot to generate the robot reference joint trajectory. Spatiotemporal constraints on the interaction and contact events between the human body and dynamic objects are introduced during the mapping process. S400, Control Strategy Training: In a simulation environment, based on the robot's reference joint trajectory and scene geometry model, the control strategy is trained through reinforcement learning. S500, Deployment and Execution: Deploy the trained control strategy onto the target robot, enabling the target robot to perform dynamic interactive tasks similar to the motion trajectory of a dynamic object.

[0023] Specifically, step S100 includes the following steps: S101. Use a human pose detection network to extract two-dimensional human key points from each frame of a monocular video. S102. Using a monocular human reconstruction network, estimate the human model parameters and the three-dimensional joint positions relative to the camera for each frame based on two-dimensional human key points. S103. Use object detection and tracking algorithms to obtain the pixel trajectory of dynamic objects in monocular video; S104. Based on motion reconstruction structure technology and / or monocular depth estimation technology, recover the camera pose and generate a depth map; S105, Standardized object detection and feature extraction; Specifically, a deep learning-based instance segmentation model (such as Mask R-CNN or SAM) is used to extract the pixel-level mask of the reference object with known physical size; edge detection is performed on the mask, and the least squares method is used to fit an ellipse to extract the pixel lengths of the major and minor axes of the reference object with known physical size in the image plane, which are used as the core image features for calculating the scale factor.

[0024] S106. Preliminary Scale Calculation and Robustness Enhancement: Based on the camera model and the known physical dimensions of the reference object with known physical dimensions, establish the constraint relationship between depth, focal length, and pixel size.

[0025] In this embodiment, regarding motion blur and deformation that may exist in monocular video: If the mask confidence of a single frame detection is lower than a preset threshold... If the ratio of the major and minor axes of the fitted ellipse exceeds the distortion threshold, time series filtering is activated, and the pixel diameter of the current frame is smoothly corrected by combining the predicted trajectories of the previous and next frames.

[0026] In this embodiment, for partial occlusion that may exist in monocular video: the outline is restored by geometric center extrapolation of the visible edge part, or the scene is automatically switched to other static objects of known size (such as the width of the stadium markings, the distance between the goalposts) as backup scale references.

[0027] Final output: Solve for the initial estimate of the global scale factor and perform preliminary scaling on the 3D joint position and depth maps to obtain scaled preliminary 3D data.

[0028] Specifically, the human pose detection network uses a mature 2D pose detector to obtain 2D key points for each frame. The parameters of each frame of the SMPL (Skinned Multi-Person Linear Model) are estimated using a monocular human reconstruction network. 3D joints with relative camera Use instance segmentation / small object detection on dynamic objects to obtain the trajectory of dynamic objects in the pixel plane. If a monocular video provides a depth channel, SfM (Structure from Motion) can be used to further enhance / convert the 2D pixel trajectory of a dynamic object into a coarse 3D spatial trajectory. .

[0029] In this embodiment, the camera pose trajectory in the video sequence is recovered using structure-of-motion reconstruction (SOR) technology, and a depth map corresponding to each frame is generated using a monocular depth estimation method. .

[0030] To ensure that the monocular video reconstruction results have the absolute scale of the real physical world, a method of using geometric constraints of reference objects with known physical dimensions is used to solve the scale of dynamic objects.

[0031] For example, when the dynamic object is a soccer ball, the interactive sphere in the video is set to a size 5 soccer ball conforming to FIFA standards (physical diameter). The system performs the following steps:

[0032] Reference object feature extraction: Detect and extract the pixel contour of the sphere in the video frame, and calculate its major axis pixel diameter in the image plane. ; Geometric Scale Calculation: Depth Establishment Based on Camera Model ,focal length Constraints on physical dimensions; Global scale optimization: Incorporate the above geometric constraints into the joint optimization objective function and solve for the global scale factor. By leveraging the ubiquitous and standardized sizes of interactive objects in a scene, scale ambiguity in monocular vision reconstruction is effectively eliminated, ensuring the accuracy of reconstructed environmental parameters (such as gravitational acceleration). Mass of an object It maintains consistency with the real physical environment, providing an accurate physical benchmark for subsequent dynamic simulations.

[0033] Output (for downstream joint optimization): Key points per frame 3D joints (Camera frame), spherical world coordinate trajectory .

[0034] Specifically, , and The constraints are as follows: Formula (1).

[0035] In this embodiment, a mature 2D pose detector can be ViTPose (Vision Transformer Pose Estimation, pose estimation based on visual Transformer).

[0036] In this embodiment, instance segmentation / small object detection uses SAM (Segment Anything Model) or YOLO (You Only Look Once, a real-time object detection framework) for dynamic object detection, and KLT (Kanade-Lucas-Tomasi, a feature tracker) or DeAOT (DeformableAttention Object Tracking) for dynamic object tracking.

[0037] In this embodiment, the motion recovery structure uses Mega-SfM (a variant of large-scale SfM) or other algorithms with comparable performance.

[0038] In this embodiment, the optimization variables of the objective function in step S200 include global human body translation, global human body rotation, local joint pose of the human body, world coordinate trajectory and velocity vector of the dynamic object, and scene scale factor; the objective function includes a weighted sum of the reprojection error term between the human body 3D mesh calculated based on the human body model parameters and the preliminary 3D data, and a physical consistency term. The physical consistency term includes at least one of the following: the dynamic residual between the motion trajectory of the dynamic object and the trajectory calculated based on the rigid body dynamics equations, the non-penetrating constraint between the human body 3D mesh and the scene geometric model, and the human body self-collision avoidance constraint.

[0039] Specifically, in the physical consistency term, the dynamic residual is calculated by minimizing the difference between the observed trajectory of the dynamic object and the trajectory generated by the physical simulation, and a momentum conservation constraint is introduced at the moment of the contact event.

[0040] In one possible implementation, when the dynamic object is a soccer ball, Optimization variables include: global human body translation. Global rotation Local joint posture Scene scale Sphere World Coordinate Trajectory With speed

[0041] The objective function is as follows: Formula (2); in: It uses forward inference of SMPL to calculate the posture and shape parameters of the human body and decode them into a three-dimensional mesh representation; This is a physically consistent term.

[0042] Meanwhile, to ensure the physical feasibility of the sphere's trajectory, physical constraints based on rigid body dynamics are introduced as spatiotemporal constraints, and the trajectory is optimized through dynamic fitting.

[0043] In this embodiment, the sphere is assumed to be an ideal rigid body, whose motion is only affected by gravity and instantaneous collisions with the environment. Its trajectory should satisfy the discrete-time dynamic equation approximation. By aligning the observed trajectory with the trajectory generated by the physical simulation and applying kinematic and dynamic consistency constraints (such as velocity updates, energy conservation, collision response models, etc.), the reproducibility of the reconstructed trajectory in the simulation environment and the stability of the control strategy training are improved.

[0044] Specifically, a discrete-time motion model based on classical mechanics is introduced as a physical prior, and the motion update rules of the ball during the free flight and collision phases are described using a semi-implicit Euler integral form, as shown below: coordinate trajectory renew: Formula (3); speed renew: Formula (4); Specifically, the instantaneous impact term in formula (4) Used to describe sudden changes in velocity when a dynamic object collides with an intelligent agent (such as a foot) or the environment (such as the ground). This embodiment uses a collision model based on impulse:

[0045] Formula (5); in, The coefficient of restitution is used to characterize the energy loss during a collision. The relative velocity at the point of collision. The normal vector of the contact surface; This represents the dynamic modeling error term, used to characterize motion deviations caused by simplification of the air resistance model, energy loss from non-ideal rigid body collisions, and unobservable disturbances from the external environment (such as local wind force and nonlinear terms of ground friction). This is constrained in the optimization objective function of S200. The L2 norm ensures that the reconstructed trajectory satisfies physical laws while having strong anti-interference capabilities.

[0046] Preferably, to further improve reconstruction accuracy, the restoration coefficient... Instead of being preset as a fixed constant, it is used as a variable to be optimized in step S200. By minimizing the residual between the observed trajectory and the physical simulation trajectory, the material properties of the object that best fit the current video scene (such as the air pressure of the ball and the hardness of the ground) are automatically calculated in reverse.

[0047] In this embodiment, the contact time is allowed to fluctuate within a small range in the objective function to minimize the dynamic residual. A robust estimation method based on multi-source information fusion and temporal optimization is employed to optimize the contact time, as detailed below:

[0048] Multimodal fusion determination: Initial localization is performed by combining visual and kinematic features. Visual features include pixel-level ball-foot overlap region (IoU) and geometric distance between key points of the foot and the edge of the ball; kinematic features include anomalous changes in the velocity or acceleration vector of the human foot at the moment of contact (i.e., points of dynamic discontinuity).

[0049] Temporal window optimization: Considering the perceptual uncertainty caused by motion blur, the contact moment initially detected in the monocular video is regarded as a temporal variable to be optimized. In the joint reconstruction in step S200, the contact moment is allowed to be within a preset time window (e.g., ...). (Frame) Floating. By minimizing the dynamic residual in formula (4), we find the moment when the object's trajectory is most physically consistent, which is the final corrected precise contact moment, thereby overcoming the limit of perception capability under high-speed dynamic events.

[0050] Specifically, in this embodiment, the dynamic residual is minimized. To fit , .

[0051] In this embodiment, a sparse point cloud of the scene is obtained through a large-scale 3D reconstruction method, and the scale factor obtained through joint optimization is utilized. The point cloud is globally scaled to align with the real-world scale. Then, voxel grid downsampling is performed on the scaled point cloud to reduce redundancy, and outlier noise points are removed using statistical filtering and other methods. The denoised point cloud is then converted into a continuous and lightweight triangular mesh model to ensure its suitability for subsequent physical simulations. To improve the geometric plausibility of the simulation environment, the generated mesh is aligned with the gravity direction by analyzing the ground plane normal and rotating it to align with the vertical axis of the world coordinate system. Finally, based on the spatial distribution of the reconstructed human SMPL trajectory and sphere trajectory, a local scene mesh with a radius of ±2 m centered on the interaction area is trimmed and retained, focusing on key areas where human-object contact may occur.

[0052] The final output in step S200 includes: SMPL joint motion trajectories in the world coordinate system, sphere trajectories with velocity information, a simplified scene mesh model, and a set of annotated human-object / object-ground contact event times. This provides geometric and dynamic context support for subsequent control strategy learning and physical simulation.

[0053] Preferably, the large-scale 3D reconstruction method adopts Mega-SfM or MonST3R (a 3D geometry reconstruction method for dynamic scenes that extends the static scene model DUSt3R to handle moving and deformable scenes by directly estimating the point cloud at each time step).

[0054] Preferably, the NKSR (Neural Kernel Surface Reconstruction) or Poisson Surface Reconstruction method is used to transform the denoised point cloud into a continuous and lightweight triangular mesh model.

[0055] In step S300 of this embodiment, the robot reference joint trajectory includes: the joint angle of the target robot and the root pose of the target robot; the spatiotemporal constraint is an optimization cost function, which includes a weighted sum of the following terms: the posture consistency term between the robot reference joint trajectory and the reconstructed human motion trajectory, the foot-ground contact point matching term, the position consistency term between the robot end effector and the dynamic object when the interactive contact event occurs, and the robot self-collision and world collision penalty term.

[0056] In one possible implementation, when the dynamic object is a soccer ball, the reconstructed human trajectory is mapped to the joint trajectory of the target robot. At the same time, it satisfies the contact constraints (foot-ground and foot-ball).

[0057] The optimization variables for optimizing the cost function are: robot joint angles. Robot root pose Possible scaling factors .

[0058] The formula for the cost function is shown below: Formula (6); First item Used to maintain topology pose consistency; Second item Used to ensure that the contact point between the foot and the ground is matched (using TRO provides foot contact frames); Third item The robot is forced to align its "kick point" with the ball's position at the moment of contact (the kick point can be approximated by the robot's foot or shoe). Fourth item For self-collision and world collision penalties (collision detection between robot capsule model and scene mesh).

[0059] In the fifth item, The (weighting coefficient) represents the importance of collision penalty in the total cost function. By adjusting this coefficient, the relationship between the accuracy of motion mimicry and obstacle avoidance safety can be balanced. The collision loss function is used to quantify the degree of interference that occurs between the robot and other components. In calculations, each link of the robot is typically simplified to a geometrically bounding volume (such as a capsule or sphere), and the following are calculated: Self-collision: The penetration depth between different links within the robot itself (e.g., left leg and right leg). World-collision: The penetration depth between robot links and the scene geometry (such as the ground, walls, or obstacle mesh). When the robot's generated trajectories overlap in space (i.e., a collision occurs), This generates a huge positive penalty value, which forces the optimization algorithm to adjust the joint angles to ensure that the final generated robot reference trajectory does not intersect itself in physical space and does not penetrate the environment, thus ensuring the safety of the actions executed on real hardware.

[0060] In this embodiment, the Levenberg-Marquardt (LM) algorithm is used to optimize the robot's joint configuration frame by frame to improve the temporal continuity and solution stability of the motion trajectory. For cases where some human poses are unattainable due to robot physical dimensions, joint limits, or kinematic constraints, slack variables are introduced and a soft-constraint cost function is designed. By reducing the pose tracking weight at corresponding moments and increasing the priority of the contact consistency term, a certain degree of pose deviation is allowed. This mechanism preserves the original motion intent while generating physically realizable approximate solutions, avoiding optimization failures or severe jitter.

[0061] The final output includes robot reference joint trajectories for simulation. The system also includes time-series annotations of key contact events related to environmental interaction. The output is then used as a supervisory signal for trajectory tracking in subsequent reinforcement learning control policy training, and is used to construct a simulation reward function that includes multiple objectives such as pose matching and contact state alignment.

[0062] In step S400 of this embodiment, the training control strategy adopts a phased training approach, including: a motion capture pre-training phase: using a motion capture dataset to initially train the control strategy; a scene-conditional tracking phase: in a simulation environment, a scene geometry model is loaded, and the robot reference joint trajectory is used as a supervision signal to train the control strategy to track and reproduce the target behavior; a control strategy distillation phase: a teacher's control strategy guides the student control strategy to learn to predict actions from local observation information, gradually reducing the dependence on the robot reference joint trajectory; and an underconditional reinforcement learning fine-tuning phase: under the condition of removing the robot reference joint trajectory as input, the control strategy is fine-tuned, and domain randomization (DR) is introduced.

[0063] In one specific embodiment, a final deployable control policy is trained in a simulator. The simulation environment runs at a frequency of 200 Hz, and the controller employs a frequency reduction control strategy, performing an action decision every four simulation steps, i.e., a control cycle of 50 Hz (corresponding to a solution cycle of 0.02 s), consistent with the control architecture of the VideoMimic algorithm. Each simulation instance loads a reconstructed scene mesh model and incorporates the rigid body dynamics parameters of a sphere (including mass, radius, and contact friction coefficient). The mass and friction coefficient of the sphere are randomly perturbed during training using a domain randomization control strategy to improve the robustness of the control policy to parameter changes and its cross-environment generalization ability. To accelerate the training process, a massively parallelized scheme is adopted, with a single GPU supporting up to 4096 concurrent simulation environments. Combined with a multi-GPU distributed training framework, efficient large-batch reinforcement learning control policy optimization is achieved.

[0064] In this embodiment, the control strategy adopts the Actor-Critic reinforcement learning framework, in which the Actor network outputs joint target position (or torque) instructions based on the above observations, and the Critic network additionally uses privileged information for value estimation to improve training efficiency and stability.

[0065] In this embodiment, the observation space (actor part) for simulation training includes: Proprioception: Joint angle Joint angular velocity Base linear velocity / angular velocity, projected gravity vector; Object perception: The relative position of the ball in the robot's root coordinate system With relative velocity .

[0066] Observational augmentation: During the simulation training phase, before inputting the ground truth from the physics engine into the Actor network, the uncertainties of real sensors (such as LiDAR or cameras) are simulated, and the data is then processed accordingly. and Gaussian-distributed observation noise is injected into the system, and random time delays are introduced based on the typical latency of the camera's onboard sensing module (e.g., 20ms-50ms).

[0067] Sensor model simulation: Furthermore, considering the accuracy degradation of the airborne sensing module at long distances, the noise standard deviation is set to be positively correlated with the distance to the sphere, so that the control strategy learns the heuristic behavior of "the farther the distance, the less it trusts the observation data, and the closer the distance, the more accurate the action".

[0068] Topography: Local elevation map (11×11 or 21×21), grid spacing can be set to 0.05~0.1m; Command: Target root direction / target control strategy mode (shoot / pass / receive, which can be inferred from high-level commands or video).

[0069] In this embodiment, the motion space (Critic part) of the simulation training includes: the output is the joint target angle (or incremental angle) or low-order torque command (depending on the robot control stack), and the motion dimension corresponds to the robot's DoF (Degrees of Freedom).

[0070] In this embodiment, the rewards for simulation training consist of two main categories: trajectory tracking items and task objective items.

[0071] The trajectory tracking items specifically include: Joint tracking (data-driven): Formula (7); Link tracing (trunk / root direction): Formula (8); Consistent foot contact (to prevent slipping): Formula (9); in, Indicates the current simulation time. The actual angle vectors of each active joint of the robot; This represents the reference joint angle vector generated after action redirection in step S300; This is the gain coefficient (scale factor) for joint angle tracking reward, used to adjust the sensitivity of the reward function to attitude error. The aim is to encourage the robot's joints to match the reconstructed human movements as closely as possible in terms of timing; This represents the robot's root (usually the center of the torso or base) pose in the current simulation environment's world coordinate system, including position coordinates and orientation quaternions; This represents the pose of the corresponding reference root part in the reconstructed trajectory; The gain coefficient for root pose tracking reward is designed to ensure that while the robot mimics limb movements, its global movement path and body orientation remain consistent with the original video, preventing global drift. The contact status indication function is determined based on the contact label provided by the reference trajectory in step S300: the value is 1 when the reference trajectory indicates that the foot should be in contact with the ground; otherwise, it is 0. This represents the actual three-dimensional spatial coordinates of the robot's end-effector in the simulation environment. This indicates the preset coordinates of the foot's contact point with the ground in the reference trajectory;

[0072] This is the gain coefficient for foot contact reward.

[0073] Task objectives include rewards and penalties.

[0074] The specific rewards are for ball contact and hits (core mission objective), including: Contact moment reward: At the moment of contact If the simulation detects a football-ball collision, a successful contact reward is given. ; Ball speed / direction bonus: If the angle between the ball's speed and direction and the designated target direction is less than a threshold after the kick, a bonus is awarded. , The specific form is as follows: Formula (10); in, This represents the real-time velocity vector of the sphere in the simulation environment after the interaction occurs; The target direction is indicated by a pre-defined direction, which is obtained by means of: (1) extracting it from the dynamic object motion trajectory reconstructed in step S200, specifically extracting the direction after the ball is kicked in the video. (2) The average displacement direction of a frame (e.g., 10 frames); or a unit vector pointing to the center of the goal or the position of a teammate, as pre-defined by a higher-level task instruction based on the scene geometry model. The angle reward decay coefficient (or bandwidth coefficient) is used to control the rate at which the reward decreases as the angle deviation increases. The larger the value, the higher the requirement for directional accuracy. : is the speed threshold indication function, where This is a preset minimum speed (e.g., 0.5 m / s). The function's purpose is to activate the directional reward only when the ball is effectively hit and has a certain initial velocity, thus preventing the robot from receiving a reward due to meaningless slight collisions with the ball.

[0075] Goal / Shot Location Bonus (applicable to different scenarios): Additional bonus if the ball enters the target area or is received by a designated teammate.

[0076] The specific penalties include: Formula (11); Self-collision and excessive torque penalty; The penalty is severe for the ball passing through walls / scene penetration (physical inconsistencies must be avoided).

[0077] The total reward is a weighted sum: Formula (12); in, These are the weighting coefficients for joint tracking terms, used to adjust the strength of the robot's imitation of human motion trajectories (joint angles, root pose). Higher values ​​indicate a higher level of imitation. The chosen values ​​ensure that the robot-generated actions closely resemble human demonstrations in the video, both visually and topologically. This is the weighting coefficient for the foot consistency term, used to reinforce the physical constraints at the foot-ground contact point. By penalizing the "sliding" phenomenon, the stability of the robot's supporting legs is ensured, providing a foundation for subsequent interactive bursts. The interaction contact reward weighting coefficient is used to indicate the importance of a successful contact (such as kicking the ball). In the initial stages of training, it should be appropriately increased. It can guide robots to actively generate collisions with dynamic objects, overcoming the exploration challenges caused by sparse rewards; This is a weighting coefficient for the task objective reward, used to adjust the proportion of interaction quality (such as ball speed and shot direction) in the total reward. This determines whether the robot will evolve from "merely kicking the ball" to "precise shooting" or "directional passing".

[0078] In this embodiment, the entire control strategy training process is divided into four progressive stages.

[0079] First, motion capture pre-training (MPT) is performed, using human motion datasets to initially train the control strategy, enabling the robot to master basic balance control and gait generation capabilities. If real MoCap data is lacking, robot reference trajectories generated by motion redirection technology can be used as alternative supervision signals.

[0080] The robot then enters the Scene-Conditioned Tracking (MPT) phase. Using the control policy trained in the MPT phase as the initial model, it performs batched imitation learning on the reconstructed scene mesh, employing a DeepMimic-style reward mechanism to guide the control policy in replicating the target behavior. During this phase, the robot can still observe the target joint angle information to accelerate convergence and improve trajectory matching accuracy.

[0081] The third stage is policy distillation. Through the DAgger framework, teachers with complete observations generate action demonstrations based on their control policies, guiding students to learn control policies to predict reasonable actions from limited inputs such as proprioception, sphere state estimation, and local height maps, gradually eliminating reliance on global joint targets that students cannot obtain.

[0082] Finally, in the under-conditioned reinforcement learning (RL) fine-tuning stage, the PPO algorithm is used to fine-tune the control strategy end-to-end, further enhancing its task completion capability and robustness in low-information-density environments, while completely removing the target joint angle condition input. A stronger domain randomization mechanism is introduced in this stage, including dynamic parameter perturbations (such as friction coefficient and object mass) and simulation of sensor noise and time delay, to improve the generalization performance of the control strategy in real-world deployments.

[0083] In this embodiment, the human motion dataset uses existing publicly available high-quality datasets such as LAFAN to initially train the control strategy.

[0084] In this embodiment, domain randomization includes randomly perturbing at least one of the following parameters: joint friction coefficient, mass of the dynamic object, sensor detection delay, sensor measurement noise, control command delay, and terrain height map resolution.

[0085] In one possible embodiment, the parameters for domain randomization are specifically: Joint friction coefficient randomized by ±20%; ball mass randomized by ±30%; sensor (ball detection) delay randomized by 0-100 ms; position noise Gaussian σ. pos = 0.01-0.05 m; control delay / action delay 0-60 ms; terrain height map mask and resolution degradation simulate real LiDAR noise, and the introduction of the above domain randomization helps to improve the robustness of the actual machine.

[0086] In this embodiment, step S500 involves deploying and executing the deployment, which includes using an onboard LiDAR or depth camera to build a local height map (11×11 patch, 0.1 m grid) in real time and using a sphere detector to obtain the position and velocity estimates of the sphere pairs.

[0087] Preferably, the velocity estimate of the sphere is obtained by fusing LiDAR and camera data using Kalman filtering.

[0088] Preferably, the sphere detector uses YOLO or point cloud clustering to obtain the relative position of the sphere.

[0089] In this embodiment, the relative position of the sphere is estimated using tracklet filtering (uniform velocity / uniform acceleration model). With speed The estimation error is then reported to the control policy (which has been trained to tolerate the noise).

[0090] In this embodiment, the reward function of the control strategy includes a weighted sum of the following items: a tracking reward item for the robot's reference joint trajectory; and a task reward item for interacting with dynamic objects, which includes: a contact reward when a collision between the foot and the dynamic object is detected; a directional reward when the direction of motion of the dynamic object after being knocked out is consistent with the target direction; and a hit reward when the dynamic object enters the target area.

[0091] In this embodiment, the output of the control strategy is the target joint angle / increment, and the outer PD controller converts the target angle into a motor command.

[0092] Preferably, the control frequency is about 50Hz (or the highest safe frequency allowed by the robot hardware).

[0093] In this embodiment, the deployment and execution steps also include running an independent safety monitoring module. The safety monitoring module is configured to monitor the joint motor current and / or output torque of the robot in real time. If the current exceeds the safety threshold, a hardware protection mechanism is triggered. The module also calculates the robot's zero torque point (ZMP) and / or capture point (CP) in real time to assess the robot's stability. If the module determines that the robot is about to become unstable, a compliant tipping control strategy is triggered.

[0094] Specifically, to ensure the robot's safety and system stability when performing highly dynamic interactive tasks, this invention constructs a two-layer safety monitoring system independent of the core algorithm. This two-layer safety monitoring system has the highest control interrupt priority. The two-layer safety monitoring system includes a hardware-layer fuse protection mechanism and a software-layer real-time stability assessment and compliant control strategy.

[0095] Specifically, the hardware-level fuse protection mechanism includes: Driver status monitoring: Real-time monitoring of the current value and output torque of each joint motor. If the current is detected to continuously exceed the rated peak value (e.g., exceeding the safety threshold for 10ms), the underlying controller will directly trigger "zero torque mode" or perform emergency braking to physically prevent the risk of hardware damage caused by overcurrent.

[0096] Mechanical limit protection: Within the 5% buffer zone where the joint angle is close to the physical limit, a reverse damping torque is forcibly intervened to prevent hard collisions of the mechanical structure.

[0097] Specifically, the real-time stability assessment and compliant control strategies at the software layer include: Stability criteria based on model predictive control (MPC): The controller calculates the robot's zero-moment point (ZMP) and capture point (CP) states in real time. The system continuously monitors whether the capture point is within the safety margin of the support polygon, using this as a quantitative indicator of dynamic equilibrium.

[0098] Active compliant fall protection strategy: When the system determines that the instability is irreversible, it immediately switches from "task execution mode" to "compliant fall mode". In this mode, the control strategy actively reduces joint position gains (PD gains) and plans protective actions including covering the head and curling up the torso, using full-body contact with the ground to disperse impact energy and minimize impact damage to the robot's core components from the fall.

[0099] During the actual deployment phase, a small number of actual machine samples can be used for online fine-tuning (within a safe range) or retraining can be performed in the simulation using playback data (sim-to-real loop).

[0100] The real-time and deterministic guarantee mechanisms of the dual-layer security monitoring system are as follows: To ensure the effectiveness of security monitoring during high-speed dynamic interaction, this embodiment optimizes the execution architecture of the two-layer security monitoring system as follows: Independent real-time threads and high-frequency operation: Operating frequency: The dual-layer security monitoring system runs on the embedded controller's real-time operating system (RTOS) and is configured as an independent real-time task thread with the highest execution priority. Its operating frequency is set to 1000Hz (1ms sampling period), which is much higher than the 50Hz operating frequency of the main control strategy (Actor network).

[0101] Computational optimization: The stability criterion (ZMP / CP) and current monitoring algorithm have been optimized by fixed-point and instruction level to ensure that the computation time of a single monitoring logic is controlled within 0.5ms, thereby reserving sufficient time window for instruction issuance.

[0102] Highest priority preemption and hardware interrupts: Control interrupt priority: The safety monitoring system has the highest control interrupt priority. Once a hardware-level fuse threshold (such as a surge in motor current) or a software-level instability criterion is detected, the system will immediately preempt the computational tasks of the current main control strategy.

[0103] Bypass logic: When protection is triggered, the safety command is not forwarded through layers of high-level algorithms, but is directly written into the control word of the joint driver through bus-level (such as EtherCAT or CAN Open) emergency commands, realizing hardware-level "one-click response".

[0104] Response delay control: End-to-end latency: The end-to-end latency from sensor signal acquisition (current / gyroscope data), safety logic judgment to actuator (driver mode switching) is strictly controlled within 2ms.

[0105] Physical buffer design: In mechanical limit protection, the "braking distance" is pre-calculated based on the current joint angular velocity, leaving a sufficient buffer period before entering the physical limit to offset the physical response lag between communication and actuator.

[0106] Differentiated response grading: The system classifies security events into two levels: Level 1 warning (non-lethal): If ZMP deviates slightly from the safe zone, attempt to inject a small corrective torque (soft constraint) into the main control loop without interrupting the action flow.

[0107] Level 2 fuse (fatal): If motor overcurrent or severe CP deviation is detected, the "zero torque mode" or "compliant tipping control strategy" will be triggered directly to cut off power in milliseconds to protect hardware safety.

[0108] Example 2 This embodiment provides a robot dynamic interactive skill learning system based on monocular video, building upon Embodiment 1.

[0109] In this embodiment, the robot dynamic interaction skill learning system based on monocular video includes a processor and a memory; the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the robot dynamic interaction skill learning system based on monocular video performs the robot dynamic interaction skill learning method described above.

[0110] Example 3 This embodiment provides a dynamic interactive skill learning scheme for robotic arms based on industrial standard parts, building upon any of the above embodiments.

[0111] This embodiment demonstrates the application of a robot dynamic interaction skill learning method based on monocular video in an industrial automation scenario. The target intelligent agent is a six-degree-of-freedom industrial robotic arm, and the interaction task is to dynamically intercept and grasp standard logistics turnover boxes that are moving rapidly on a conveyor belt.

[0112] Step S100, the video parsing stage, includes the following steps: Standardized object definition: In this embodiment, the reference object with known physical dimensions is a plastic turnover box with international standard dimensions (such as 600mm×400mm×230mm).

[0113] Feature Extraction: A pixel-level mask of the turnover box in the video is extracted using an instance segmentation model based on a visual fundamental model (such as SAM) or a deep convolutional neural network. To address motion blur caused by the high-speed movement of the robotic arm, this embodiment extracts the oriented bounding box (OBB) of the mask and performs Kalman filtering on the multi-frame motion trajectories over time to fit the high-precision geometric center and boundary pixel features of the turnover box in the image plane.

[0114] Scale calculation: Using the known physical dimensions of the crate and the extracted pixel lengths from the image, the scale factor of the monocular video is calculated based on the camera model. .

[0115] In step S200, the joint reconstruction phase: This embodiment will initially calculate the scale factor. As variables to be optimized, a joint optimization objective function is constructed by combining the friction model between the turnover box and the conveyor belt, the gravity model, and the dynamic constraints of the human hand grasping the box. By minimizing the video reprojection error and the physical residual, the 6D pose trajectory of the turnover box is globally optimized and corrected. Even when the turnover box is partially occluded in the video, the system can still complete its three-dimensional motion trajectory using physical consistency constraints (such as the object cannot penetrate the surface of the conveyor belt).

[0116] Step S300, the action redirection phase, includes the following steps: Topology mapping: Mapping the grasping motion of a human hand in a video to the end effector of a robotic arm (such as a parallel pneumatic gripper).

[0117] Spatiotemporal constraint optimization: Identifying "critical interaction moments" where a human hand contacts the edge of a turnover box in a video. When generating the robotic arm's reference trajectory, contact constraints are introduced: requiring that at the interaction moment... The opening phase and spatial pose of the gripper must be precisely matched with the edge geometry of the turnover box, and the relative speed must be close to zero to ensure the stability of dynamic gripping.

[0118] In the S400 control policy training and S500 deployment and execution phase: Sim2Real Enhancement: During simulation training, Gaussian noise and a delay model (e.g., 20ms observation delay) that conform to the characteristics of an industrial camera are injected into the observations of the position and speed of the turnover box sensed by the robotic arm.

[0119] Safety Monitoring: In actual deployment, a dedicated safety monitoring thread runs at a frequency of 2kHz. This thread monitors the current changes of the motors at each joint of the robotic arm in real time. If the gripper experiences a collision during dynamic grasping, causing a sudden change in current, or if the turnover box slips and triggers a stability warning, the monitoring module directly triggers an emergency braking or compliant retraction protection mechanism via a hardware interrupt, with a command response latency of less than 1ms.

[0120] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for learning dynamic interactive skills in robots based on monocular video, characterized in that, Includes the following steps: S100, Video Analysis: Process the input monocular video, extract human key point information and dynamic object information, and use reference objects with known physical dimensions in the video to calculate the reconstruction scale of the monocular video to obtain scaled preliminary three-dimensional data. S200, Joint Reconstruction: Using the preliminary three-dimensional data as input, the human motion trajectory, the motion trajectory of the dynamic object, and the scene geometric model are jointly reconstructed by optimizing the objective function and having physical consistency with the standard scale of the dynamic object. S300, Action Redirection: The reconstructed human motion trajectory is mapped to the joint space of the target robot to generate a robot reference joint trajectory, and spatiotemporal constraints on the interaction and contact events between the human body and dynamic objects are introduced during the mapping process. S400, Control Strategy Training: In a simulation environment, based on the robot reference joint trajectory and the scene geometry model, the control strategy is trained through reinforcement learning. S500, Deployment and Execution: Deploy the trained control strategy onto the target robot, enabling the target robot to perform dynamic interactive tasks similar to the motion trajectory of the dynamic object.

2. The method according to claim 1, characterized in that, Step S100 specifically includes the following steps: S101. Use a human pose detection network to extract two-dimensional human key points from each frame of the monocular video. S102. Using a monocular human reconstruction network, estimate the human model parameters and the three-dimensional joint positions relative to the camera for each frame based on the two-dimensional human key points. S103. Using an object detection and tracking algorithm, obtain the pixel trajectory of the dynamic object in the monocular video; S104. Based on motion reconstruction structure technology and / or monocular depth estimation technology, recover the camera pose and generate a depth map; S105, Standardized object detection and feature extraction; S106. Preliminary Scale Solution and Robustness Enhancement.

3. The method according to claim 2, characterized in that, In step S200, The optimization variables of the objective function include global human translation, global human rotation, local joint pose of the human body, world coordinate trajectory and velocity vector of dynamic objects, and scene scale factor. The optimization objective function includes a weighted sum of the following terms: the reprojection error term between the human body 3D mesh calculated based on the human body model parameters and the preliminary 3D data, and the physical consistency term; The physical consistency term includes at least one of the following: the dynamic residual between the motion trajectory of the dynamic object and the motion trajectory of the dynamic object calculated based on the rigid body dynamics equation, the non-penetrating constraint between the human body 3D mesh and the scene geometric model, and the human body self-collision avoidance constraint.

4. The method according to claim 3, characterized in that, In the physical consistency term, the dynamic residual is calculated by minimizing the difference between the observed trajectory of the dynamic object and the trajectory generated by the physical simulation, and a momentum conservation constraint is introduced at the moment of the contact event.

5. The method according to claim 1, characterized in that, In step S300, The robot reference joint trajectory includes: the joint angles of the target robot and the root pose of the target robot; The spatiotemporal constraint is an optimization cost function, which includes a weighted sum of the following terms: The robot's reference joint trajectory and the reconstructed human motion trajectory are considered as follows: posture consistency, foot-to-ground contact point matching, robot end effector-to-dynamic object position consistency during the interactive contact event, and robot self-collision and world collision penalty.

6. The method according to claim 1, characterized in that, In step S400, the control strategy is trained using a phased training method, including: Motion capture pre-training phase: The control strategy is initially trained using the motion capture dataset; Scene-conditional tracking stage: In the simulation environment, the scene geometric model is loaded, and the control strategy is trained to track and reproduce the target behavior using the robot reference joint trajectory as a supervision signal; Control strategy distillation stage: A teacher guides students to learn control strategies to predict actions from local observation information, gradually reducing the reliance on the robot's reference joint trajectory; Underconditional reinforcement learning fine-tuning stage: The control strategy is fine-tuned by removing the robot reference joint trajectory as input and introducing domain randomization.

7. The method according to claim 6, characterized in that, The domain randomization includes randomly perturbing at least one of the following parameters: joint friction coefficient, mass of the dynamic object, sensor detection delay, sensor measurement noise, control command delay, and terrain height map resolution.

8. The method according to claim 1, characterized in that, The reward function of the control strategy includes a weighted sum of the following: a tracking reward term based on the robot's reference joint trajectory; The task reward items for interacting with the dynamic object include: a contact reward when a foot is detected to collide with the dynamic object; a directional reward when the direction of motion of the dynamic object after being knocked out is consistent with the target direction; and a hit reward when the dynamic object enters the target area.

9. The method according to claim 1, characterized in that, The deployment and execution steps also include running an independent safety monitoring module, which is configured to monitor the joint motor current and / or output torque of the robot in real time, and trigger a hardware protection mechanism if the safety threshold is exceeded. The robot's zero-torque point (ZMP) and / or capture point (CP) are calculated in real time to assess the robot's stability. If the robot is determined to be about to become unstable, a compliant tipping control strategy is triggered.

10. A robot dynamic interactive skill learning system based on monocular video, characterized in that, include: Processor and memory; The memory stores computer program instructions that, when executed by the processor, cause the system to perform the method as described in any one of claims 1 to 9.

Images