A method and system for cooperative control of a biped robot

Abstract

This application provides a cooperative control method and system for two humanoid robots, relating to the field of robot control technology. The method includes acquiring interaction data from two demonstrators; performing interactive perception motion redirection on the two humanoid robots based on the interaction data to generate a reference trajectory and topological prior that conform to robot kinematic constraints. Specifically, kinematic conflicts arising from heterogeneous redirection are resolved through dual-scale manifold optimization, accurately preserving the interaction intent; using the reference trajectory as the tracking target and guided by the topological prior, reinforcement learning is performed to obtain a cooperative control strategy for the two humanoid robots; the trained cooperative control strategy is deployed to the two humanoid robots, which exchange state information through real-time communication to perform physical cooperative tasks; this significantly improves the robustness of control and training efficiency; and ultimately achieves zero-sample transfer of the cooperative strategy from simulation to real-world operation.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a collaborative control method and system for two humanoid robots. Background Technology

[0002] With the development of humanoid robot technology, the mobility of individual robots has been significantly improved. However, in scenarios such as logistics handling, disaster relief, and home care, it is often necessary for two or more robots to perform physically coupled collaborative operations (such as jointly carrying heavy objects or assisting in walking). To endow robots with these capabilities, data-driven methods (such as imitation learning and reinforcement learning) have gradually become mainstream, which utilizes motion data from human-human interactions to train robots.

[0003] Despite breakthroughs in individual robot control, existing technologies still have the following significant drawbacks when applied to the collaborative control of two humanoid robots: (1) Existing motion retargeting methods usually treat the two robots independently. Due to the difference in human and machine body size, direct scaling or independent optimization can easily lead to the destruction of the interaction geometry, such as causing "handshake in the air" or limb clipping after the "holding hands" action is retargeted.

[0004] (2) Traditional imitation learning mainly tracks the trajectory of joints or ends, making it difficult for robots to maintain a stable relative posture in tasks that require precise contact.

[0005] (3) In the dual-robot collaborative scenario, the existing reinforcement learning methods often have sparse reward functions or only focus on individual stability, which makes it difficult to efficiently guide the policy to learn complex collaborative behaviors, resulting in low training efficiency and insufficient robustness of the learned policy.

[0006] This paper proposes a collaborative control method, system, computer equipment, computer-readable storage medium, and computer program product for two humanoid robots. Summary of the Invention

[0007] This specification provides a cooperative control method, system, computer device, computer-readable storage medium, and computer program product for dual humanoid robots, in order to solve the technical problems in cooperative control of dual humanoid robots, such as the disruption of interaction geometry after motion redirection due to differences in human and robot body size, and the difficulty in training cooperative strategies, poor robustness, and difficulty in maintaining precise physical contact caused by the lack of explicit modeling of physical interaction topology in traditional imitation learning and reinforcement learning.

[0008] The collaborative control method for a dual-humanoid robot provided in this application adopts the following technical solution, including: Obtain interaction data from the two presenters; Based on the interactive data, the interactive perception motion of the dual humanoid robot is redirected to generate a reference trajectory and topological prior that conform to the robot's kinematic constraints. Using the reference trajectory as the tracking target and the topological prior as guidance, reinforcement learning is performed to obtain the cooperative control strategy of the dual humanoid robot; The trained collaborative control strategy is deployed to the dual humanoid robot, which exchanges status information through real-time communication to perform physical collaborative tasks. Optionally, the step of redirecting the interactive perception motion of the dual humanoid robot based on the interactive data to generate a reference trajectory and topological prior that conform to the robot's kinematic constraints includes: Based on the interaction data of the two demonstrators and the body parameters of the two humanoid robots, individual manifolds are constructed to constrain the posture of the humanoid robots themselves and a unified manifold is constructed to constrain the relative geometric relationship between the two humanoid robots. Based on the individual manifold and the unified manifold, a nonlinear optimization problem is constructed and solved with the objective of optimizing the joint angle sequence of the two humanoid robots, and the reference trajectory is generated. The interaction graph representing spatial proximity and the contact graph representing physical contact state are extracted from the reference trajectory as the topological prior.

[0009] Optionally, the step of constructing individual manifolds for constraining the posture of the humanoid robots themselves and a unified manifold for constraining the relative geometric relationship between the two humanoid robots, based on the interaction data of the two demonstrators and the body shape parameters of the two humanoid robots, includes: The interactive data is standardized to extract 3D key point sequences; Calculate the individual scaling ratio based on the height ratio between the humanoid robot and the corresponding demonstrator; The individual scaling ratio is applied to globally scale the coordinates of all key points of the corresponding individual in the 3D key point sequence to construct the individual manifold. The average of the scaling ratios of the two individuals is taken as the uniform scaling ratio; The unified scaling ratio is applied to globally scale the relative spatial vectors representing the interaction relationship between two people in the 3D keypoint sequence to construct the unified manifold.

[0010] Optionally, the step of constructing and solving a nonlinear optimization problem based on the individual manifold and the unified manifold, with the objective of optimizing the joint angle sequence of the two humanoid robots, to generate the reference trajectory, includes: Based on the individual manifold and the unified manifold, a nonlinear optimization problem is constructed with the objective of optimizing the joint angle sequence of a dual humanoid robot. An objective function is constructed, configured to jointly optimize the self-motion term, the interaction preservation term, and the regularization term. The optimization of the self-motion term includes minimizing the difference between the self-pose of the two humanoid robots and the constraint pose defined by the individual manifold. The optimization of the interaction preservation term includes minimizing the difference between the interaction geometry between the two humanoid robots and the constraint geometry defined by the unified manifold. The optimization of the regularization term includes ensuring the smoothness of the joint angle sequence over time. The nonlinear optimization problem is solved using the objective function as the optimization objective, and a smooth joint angle sequence that satisfies the individual manifold and the unified manifold constraints is generated as the reference trajectory.

[0011] Optionally, extracting the interaction graph representing spatial proximity and the contact graph representing physical contact state from the reference trajectory as the topological prior includes: Based on the unified manifold, identify and connect spatially adjacent key point pairs in the dual humanoid robot; construct the interaction graph based on the key point pairs. Based on the physical simulation detection of the reference trajectory, robot link pairs that are in continuous physical contact are identified and marked; the contact map is constructed based on the robot link pairs.

[0012] Optionally, the step of using the reference trajectory as the tracking target and the topological prior as guidance to perform reinforcement learning to obtain the cooperative control strategy of the dual humanoid robot includes: Construct an observation space for the policy network; the observation space includes at least the on-body observation information of the humanoid robot, the relative observation information of another humanoid robot, and the historical observation information; Construct a hierarchical coding structure for the policy network; the hierarchical coding structure includes a history encoder for encoding historical action sequences, a future encoder for encoding future reference trajectories, and an action decoder for outputting target joint positions; A reinforcement learning reward function is designed based on the interaction graph and the contact graph. The reward function includes: an interaction graph reward item based on the interaction graph and a contact graph reward item based on the contact graph. The interaction graph reward item is used to penalize deviations between the relative distance between robots and the corresponding side lengths in the interaction graph. The contact graph reward item is used to reward physical contact states that conform to the contact graph and penalize states that do not conform to it. The observation space is used as the input to the policy network, the reward function is used as the training signal, and the policy network is trained using a course learning strategy. The cooperative control policy is obtained by iteratively optimizing the parameters of the policy network to minimize the negative value of the expected cumulative reward until the policy converges.

[0013] Optionally, the step of deploying the trained collaborative control strategy to the dual-humanoid robot, wherein the dual-humanoid robot exchanges status information through real-time communication to perform physical collaborative tasks, includes: The trained collaborative control strategy is deployed to the onboard computing units of the two humanoid robots respectively; The two humanoid robots exchange their respective base poses and observation information in real time through a low-latency communication link established by a local area network; For each humanoid robot, motor torque is calculated and applied based on the output of its own cooperative control strategy and the state information received from the other humanoid robot to perform a physical cooperative task.

[0014] The collaborative control system for a dual humanoid robot provided in this application adopts the following technical solution, including: The data acquisition module is used to acquire interaction data between the two presenters; The redirection module is used to redirect the interactive perception motion of the dual humanoid robot based on the interactive data, and generate a reference trajectory and topological prior that conform to the robot's kinematic constraints. The strategy generation module is used to perform reinforcement learning with the reference trajectory as the tracking target and the topological prior as the guide to obtain the cooperative control strategy of the dual humanoid robot. An execution module is used to deploy the trained collaborative control strategy to the dual humanoid robot, which exchanges status information through real-time communication to perform physical collaborative tasks. The redirection module includes: The dual-manifold construction submodule is used to construct, based on the interaction data of the two demonstrators and the body parameters of the two humanoid robots, an individual manifold for constraining the posture of the humanoid robot itself and a unified manifold for constraining the relative geometric relationship between the two humanoid robots. The trajectory generation submodule is used to construct and solve a nonlinear optimization problem with the objective of optimizing the joint angle sequence of the two humanoid robots based on the individual manifold and the unified manifold, and generate the reference trajectory. The graph structure extraction submodule is used to extract an interaction graph representing spatial proximity relationships and a contact graph representing physical contact states from the reference trajectory as the topological prior.

[0015] Optionally, the dual-manifold construction submodule includes: The interactive data is standardized to extract 3D key point sequences; Calculate the individual scaling ratio based on the height ratio between the humanoid robot and the corresponding demonstrator; The individual scaling ratio is applied to globally scale the coordinates of all key points of the corresponding individual in the 3D key point sequence to construct the individual manifold. The average of the scaling ratios of the two individuals is taken as the uniform scaling ratio; The unified scaling ratio is applied to globally scale the relative spatial vectors representing the interaction relationship between two people in the 3D keypoint sequence to construct the unified manifold.

[0016] Optionally, the trajectory generation submodule includes: Based on the individual manifold and the unified manifold, a nonlinear optimization problem is constructed with the objective of optimizing the joint angle sequence of a dual humanoid robot. An objective function is constructed, configured to jointly optimize the self-motion term, the interaction preservation term, and the regularization term. The optimization of the self-motion term includes minimizing the difference between the self-pose of the two humanoid robots and the constraint pose defined by the individual manifold. The optimization of the interaction preservation term includes minimizing the difference between the interaction geometry between the two humanoid robots and the constraint geometry defined by the unified manifold. The optimization of the regularization term includes ensuring the smoothness of the joint angle sequence over time. The nonlinear optimization problem is solved using the objective function as the optimization objective, and a smooth joint angle sequence that satisfies the individual manifold and the unified manifold constraints is generated as the reference trajectory.

[0017] Optionally, the graph structure extraction submodule includes: Based on the unified manifold, identify and connect spatially adjacent key point pairs in the dual humanoid robot; construct the interaction graph based on the key point pairs. Based on the physical simulation detection of the reference trajectory, robot link pairs that are in continuous physical contact are identified and marked; the contact map is constructed based on the robot link pairs.

[0018] Optionally, the strategy generation module includes: Construct an observation space for the policy network; the observation space includes at least the on-body observation information of the humanoid robot, the relative observation information of another humanoid robot, and the historical observation information; Construct a hierarchical coding structure for the policy network; the hierarchical coding structure includes a history encoder for encoding historical action sequences, a future encoder for encoding future reference trajectories, and an action decoder for outputting target joint positions; A reinforcement learning reward function is designed based on the interaction graph and the contact graph. The reward function includes: an interaction graph reward item based on the interaction graph and a contact graph reward item based on the contact graph. The interaction graph reward item is used to penalize deviations between the relative distance between robots and the corresponding side lengths in the interaction graph. The contact graph reward item is used to reward physical contact states that conform to the contact graph and penalize states that do not conform to it. The observation space is used as the input to the policy network, the reward function is used as the training signal, and the policy network is trained using a course learning strategy. The cooperative control policy is obtained by iteratively optimizing the parameters of the policy network to minimize the negative value of the expected cumulative reward until the policy converges.

[0019] Optionally, the execution module includes: The trained collaborative control strategy is deployed to the onboard computing units of the two humanoid robots respectively; The two humanoid robots exchange their respective base poses and observation information in real time through a low-latency communication link established by a local area network; For each humanoid robot, motor torque is calculated and applied based on the output of its own cooperative control strategy and the state information received from the other humanoid robot to perform a physical cooperative task.

[0020] This specification also provides a computer device, wherein the computer device includes: Processor; and, A memory that stores computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0021] This specification also provides a computer-readable storage medium that stores one or more programs / instructions that, when executed by a processor, implement any of the methods described above.

[0022] This specification also provides a computer program product, wherein the computer program product includes: a computer program / instruction, which, when executed by a processor, implements any of the methods described above.

[0023] This invention effectively resolves kinematic conflicts such as "air handshake" and limb clipping caused by body size differences during the retargeting phase through "dual-scale manifold" optimization, ensuring that interactive actions are naturally and accurately transferred to the robot. By extracting "interaction graphs" and "contact graphs" as topological priors and incorporating them into the reward function, a clear objective is provided for policy learning, significantly alleviating the problems of sparse rewards and difficulty in exploration in high-dimensional collaborative tasks. This enables the policy to efficiently learn robust collaborative behaviors that focus on relative relationships, improving control stability. Based on the above method, the policy trained in simulation can be directly (zero-shot) deployed to a real robot for stable operation without the need for parameter tuning on actual machines, greatly reducing the application and deployment costs of two-person collaborative skills and promoting their implementation in practical scenarios such as rehabilitation and transportation. Attached Figure Description

[0024] Figure 1 A flowchart illustrating a collaborative control method for two humanoid robots provided in an embodiment of this specification; Figure 2 A schematic diagram illustrating the principle of a cooperative control method for two humanoid robots provided in the embodiments of this specification; Figure 3 A schematic diagram illustrating the overall application scenario of a collaborative control method for two humanoid robots provided in the embodiments of this specification; Figure 4 A schematic diagram of the structure of a collaborative control system for a dual humanoid robot provided in an embodiment of this specification; Figure 5 A schematic diagram of the structure of a computer device provided in the embodiments of this specification; Figure 6 This is a schematic diagram of a computer-readable storage medium provided for an embodiment of this specification. Detailed Implementation

[0025] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0026] Exemplary embodiments of the invention will now be described more fully with reference to the accompanying drawings. While conforming to the inventive concept, the features, structures, characteristics, or other details described in a particular embodiment may be combined in one or more other embodiments in a suitable manner.

[0027] The terms “and / or” or “and / or” include all combinations of any one or more of the listed items.

[0028] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0029] Figure 1 A flowchart illustrating a cooperative control method for a dual humanoid robot provided in this specification includes: S1 acquires the interaction data between the two presenters; S2 performs interactive perception motion redirection on the dual humanoid robot based on the interactive data, generating a reference trajectory and topological prior that conform to the robot's kinematic constraints; S3 uses the reference trajectory as the tracking target and the topological prior as the guide to perform reinforcement learning to obtain the cooperative control strategy of the dual humanoid robot; S4 deploys the trained collaborative control strategy to the dual humanoid robot, which exchanges status information through real-time communication to perform physical collaborative tasks.

[0030] This invention addresses the kinematic conflicts and lack of interaction inherent in existing dual-humanoid robot cooperative control technologies by proposing a full-body cooperative control method based on interactive topology priors. The physical execution end consists of two isomorphic humanoid robots. Each robot is equipped with an onboard computing unit, and the two robots establish a low-latency communication link via a local area network using the Lightweight Communications and Marshalling (LCM) protocol to share base pose (global pose) and observation information in real time. The offline training end is a physical simulation simulator deployed on a high-performance workstation, used to process interactive data and run the physical simulation environment (Isaac Lab).

[0031] This invention achieves the transfer from heterogeneous human data to robot collaborative control through two core stages: Interactive Aware Motion Redirection (IAMR) and Interactive Graph-Based Reinforcement Learning (IGRL). Specifically: S1 acquires the interaction data between the two presenters; The demonstrators are human demonstrators. Specifically, there are two demonstrators: Demonstrator X and Demonstrator Y.

[0032] The interaction data is a dataset of original human-human interaction.

[0033] S2 performs interactive perception motion redirection on the dual humanoid robot based on the interactive data, generating a reference trajectory and topological prior that conform to the robot's kinematic constraints; As shown in the Interactive Aware Motion Redirection (IAMR) processing flow on the left side of Figure 2, this invention addresses the kinematic conflicts caused by body size differences (Embodiment Mismatch) between humans and robots through decoupled optimization for heterogeneous two-person interaction data, generating reference trajectories and topological priors that conform to robot kinematics.

[0034] S21 constructs individual manifolds for constraining the posture of the humanoid robots themselves and a unified manifold for constraining the relative geometric relationship between the two humanoid robots based on the interaction data of the two demonstrators and the body parameters of the two humanoid robots. Existing human-to-robot motion retargeting methods typically treat two robots as independent entities, ignoring their spatial interaction relationships. Due to the significant differences in size and proportion between humans and robots (embodiment mismatch), simply scaling or optimizing independently can disrupt the interaction geometry; for example, what would normally be a handshake becomes a "handshake in mid-air" or limb clipping on the robot.

[0035] Based on the kinematic conflict in this motion reorientation, this invention addresses the significant differences in height and size between humans and robots by employing dual-manifold construction and parallel computation of two scaling scales.

[0036] S211 standardizes the interactive data and extracts a 3D key point sequence; Interaction data from different sources (SMPL, BVH, etc.) are uniformly extracted into 3D keypoint sequences.

[0037] S212 Calculates the individual scaling ratio based on the height ratio between the humanoid robot and the corresponding demonstrator; The dual humanoid robot consists of a first humanoid robot (Robot A) and a second humanoid robot (Robot B).

[0038] Each demonstrator corresponds to a humanoid robot. In one embodiment of this specification, the first demonstrator (Demonstrator X) corresponds to the first humanoid robot (Robot A); the second demonstrator (Demonstrator Y) corresponds to the second humanoid robot (Robot B).

[0039] The height ratio of each robot to its corresponding demonstrator is used as the individual scale to construct the individual manifold.

[0040] S213 Apply the individual scaling ratio to globally scale the coordinates of all key points of the corresponding individual in the 3D key point sequence to construct the individual manifold; This individual manifold is used to constrain the self-motion of the humanoid robot, ensuring the rationality of individual movement and ensuring that the feet do not slip and the limbs are proportionally coordinated during individual movement.

[0041] In one embodiment of this specification, the height ratio of the first humanoid robot to the first demonstrator is calculated. This serves as the first volume scaling factor; key points for the first presenter in the interactive data. Perform global scaling, that is, The individual manifold M constraining the posture of the first humanoid robot is obtained. Calculate the height ratio between the second humanoid robot and the second demonstrator. As a second volume scaling factor; for the second presenter's key points in the interactive data. Perform global scaling, that is, The individual manifold M constraining the posture of the second humanoid robot is obtained. .

[0042] S214 takes the average of the scaling ratios of the two individuals as the uniform scaling ratio; The average of the two individual scaling ratios is taken as the unified scaling ratio.

[0043] S215 applies the unified scaling ratio to globally scale the relative spatial vectors representing the interaction relationship between the two people in the 3D keypoint sequence, and constructs the unified manifold.

[0044] A unified manifold is constructed by scaling the two demonstrators as a whole. This unified manifold is used to constrain the relative interaction geometry between the humanoid robots, ensuring that interaction intentions (such as hand-to-hand distance) are not distorted.

[0045] In one embodiment of this specification, a uniform scaling factor is calculated. By scaling the spatial vectors representing the interaction between the two robots (such as the wrist connection vector), a unified manifold constraining the relative geometry between the two robots is obtained. .

[0046] S22 Based on the individual manifold and the unified manifold, construct and solve a nonlinear optimization problem with the objective of optimizing the joint angle sequence of the two humanoid robots, and generate the reference trajectory; This invention constructs a nonlinear optimization problem (SQP) to find the optimal sequence of robot joint angles in each frame. The optimization objective is to "approach the 'individual manifold' as closely as possible in Laplace coordinate space to maintain the naturalness of the movements, while strictly satisfying the relative position constraints defined by the 'unified manifold' in Euclidean space," thereby achieving decoupled optimization and reference generation.

[0047] S221 Based on the individual manifold and the unified manifold, a nonlinear optimization problem is constructed with the objective of optimizing the joint angle sequence of the dual humanoid robot; Nonlinear optimization problems are used to solve for the joint angle sequence of a robot.

[0048] S222 constructs an objective function, which is configured to jointly optimize its own motion term, interaction-preserving term, and regularization term: The objective function of a nonlinear optimization problem includes: a self-motion term to constrain its own attitude, an interaction preservation term to constrain the interaction geometry, and a regularization term to ensure smoothness of motion.

[0049] The optimization of the self-motion term is used to constrain the self-motion fidelity. Specifically, the optimization of the self-motion term includes: minimizing the difference between the self-pose of the dual humanoid robot and the constrained pose defined by the individual manifold; in one embodiment of this specification, it specifically involves: minimizing the difference between the Laplacian coordinates of the humanoid robot's key points and the corresponding points in the "individual manifold".

[0050] The optimization of the interaction preservation term is used to constrain the interaction geometry. Specifically, the optimization of the interaction preservation term includes: minimizing the difference between the interaction geometry between the two humanoid robots and the constraint geometry defined by the unified manifold; in one embodiment of this specification, it specifically involves: minimizing the difference between the relative distance vector between the robots and the corresponding vector in the "unified manifold".

[0051] The optimization of the regularization term includes ensuring the smoothness of the joint angle sequence as it changes over time.

[0052] S223 solves the nonlinear optimization problem using the objective function as the optimization objective, generating a smooth joint angle sequence that satisfies the individual manifold and the unified manifold constraints, as the reference trajectory.

[0053] In one embodiment of this specification, a nonlinear optimization problem is constructed, with the objective function... .in, For its own motion terms, calculate the robot's key points and corresponding individual manifolds. Difference in Laplacian coordinates at the midpoint; For the interaction preservation term, calculate the relative distance vector between robots and the unified manifold. The difference between corresponding vectors; To ensure smoothness for the regularization term. Weights , , To balance various factors, the Sequential Quadratic Programming (SQP) algorithm is used to solve for the optimal joint angles in each frame, thus obtaining the reference trajectory.

[0054] The "dual-scale scaling" and "dual-manifold optimization" strategies proposed in this invention cleverly decouple the robot's own motion stability from the consistency of its interaction geometry. This allows the robot to maintain its own balance and reasonable posture while accurately maintaining the required relative spatial relationships (such as holding hands or hugging), effectively solving the kinematic conflicts in heterogeneous repositioning and overcoming the common contact loss or clipping problems in existing technologies.

[0055] S23 extracts an interaction graph representing spatial proximity and a contact graph representing physical contact state from the reference trajectory as the topological prior. Traditional imitation learning only tracks joint angles or end-effector positions, lacking explicit modeling of "interaction topology" (such as relative distances and contact states). This results in the robot's inability to maintain a stable relative posture when performing tasks requiring precise physical contact. Therefore, this invention explicitly extracts two types of graph structures from the optimized reference trajectory as priors for subsequent training, as shown in the middle part of Figure 2.

[0056] S231 Based on the unified manifold, identify and connect spatially adjacent key point pairs in the dual humanoid robot; construct the interaction graph based on the key point pairs; An interaction graph is a graph structure built on a unified manifold. It represents the spatial proximity and collaborative interaction intention between two humanoid robots by connecting key pairs of spatially adjacent points (such as the right hand of the first humanoid robot and the left hand of the second humanoid robot), thereby providing explicit geometric constraints for collaborative control.

[0057] S232 identifies and marks robot link pairs that are in continuous physical contact based on physical simulation detection of the reference trajectory; and constructs the contact map based on the robot link pairs.

[0058] A contact graph is a graph structure built on physical collision detection. It explicitly represents the physical contact state and physical coupling between robots by labeling and associating robot link pairs that have continuous physical contact (such as the palm and back), thereby providing stable physical interaction constraints priors for cooperative control.

[0059] In one embodiment of this specification, based on a unified manifold Connect adjacent robot keypoint pairs in space, with edge weights based on reference distances, to construct an interaction graph. In physical simulation, the reference trajectory The humanoid robot linkage pairs that are in continuous contact are detected, marked, and recorded.

[0060] S233 outputs motion reference information; As shown at the bottom left of Figure 2, motion references are used as input for subsequent reinforcement learning. These motion references include: a trajectory, an interaction graph, and a contact graph.

[0061] S3 uses the reference trajectory as the tracking target and the topological prior as the guide to perform reinforcement learning to obtain the cooperative control strategy of the dual humanoid robot; Two-person collaboration involves high-dimensional state spaces and complex dynamic coupling. Existing reinforcement learning reward function designs are often too sparse or only focus on individual stability, making it difficult to guide policies to learn complex collaborative behaviors, resulting in low training efficiency and poor robustness of collaborative policies. Based on this, this invention proposes an interaction graph-based collaborative policy learning (IGRL) process. After obtaining the reference trajectory and topology graph, reinforcement learning is performed in a simulator on the offline training end, as shown on the right side of Figure 2.

[0062] S31 constructs the observation space of the policy network; The policy network is the neural network model for the entire reinforcement learning process.

[0063] The observation space includes at least the on-body observation information of the humanoid robot, the relative observation information of another humanoid robot, and the historical observation information; The body observation information includes: the joint positions and velocities of the human robot body.

[0064] Peer observation information includes the ontological observation information of the humanoid robot's teammate. It should be understood that the two humanoid robots in a dual-humanoid robot are teammates.

[0065] Historical observation information includes both extrinsic historical observation information and relative historical observation information.

[0066] S32 constructs the hierarchical coding structure of the policy network; The hierarchical coding structure includes a history encoder for encoding historical motion sequences, a future encoder for encoding future reference trajectories, and a motion decoder for outputting target joint positions. As a preferred approach, both the historical encoder and the future encoder employ a one-dimensional convolutional neural network (1D-CNN).

[0067] The history encoder is used to process the ontology observation information from the past t frames. Relative observation information with teammates This is done to extract temporal control features. Specifically, a historical encoder performs state encoding on historical observation information to obtain historical action sequences, which serve as temporal control features. In one embodiment of this specification, t = 20 frames, then .

[0068] The Future Encoder processes reference trajectories from future K-frames to extract interaction intent features. Specifically, the Future Encoder analyzes the reference trajectory (instruction) information. Perform trajectory encoding (Command Encoder) to obtain future reference trajectories, which serve as interactive intent features. In one embodiment of this specification, t = 20 frames, then .

[0069] The motion decoder (MLP decoder) is used to fuse the above features and output the target joint positions of the two robots. Specifically, the motion decoder fuses the temporal control features, interaction intent features, and current observation information. The target joint positions of the dual humanoid robot are obtained through action decoding. The current observation information includes both current body observation information and current relative observation information.

[0070] S33 Designs a reinforcement learning reward function based on the interaction graph and the contact graph; This invention introduces a graph-guided reward mechanism, designing a reinforcement learning reward function based on the interaction graph and the contact graph to guide a dual-humanoid robot in learning collaborative behavior. Specifically: The reward function includes: a basic tracking reward item, an interaction graph reward item based on the interaction graph, and a contact graph reward item based on the contact graph; Among them, the basic tracking reward item is used to encourage joint angles to approach the reference trajectory.

[0071] The Interaction Graph Reward (IPR) This is used to penalize deviations between the relative distances between robots and the corresponding side lengths in the interaction diagram. In one embodiment of this specification, based on the reference spatial relationships defined in the interaction diagram, the actual relative distances between corresponding key points of the two humanoid robots in the simulation environment are calculated in real time. If the relative distance deviates from the corresponding reference side length in the interaction diagram, a penalty is applied, thereby driving the robots to actively maintain the correct relative spatial pose and ensuring the geometric consistency of spatial cooperation.

[0072] The Contact Graph Reward, This system is used to reward physical contact states that conform to the contact diagram and penalize those that do not. In one embodiment of this specification, based on the physical contact states defined in the contact diagram, the actual contact between links of the humanoid robot in the simulation environment is monitored in real time. If the actual contact state conforms to the physical contact state in the contact diagram, a reward is given. The actual contact state conforming to the physical contact state in the contact diagram includes, but is not limited to, correct physical connection (such as a continuous handshake). If the actual contact state does not conform to the physical contact state in the contact diagram, a penalty is applied to ensure the stability and safety of physical interaction. The actual contact state not conforming to the physical contact diagram includes, but is not limited to, failure to establish a specified connection (such as detachment), undefined abnormal contact, or limb penetration.

[0073] S34 uses the observation space as the input to the policy network, the reward function as the training signal, and a course learning strategy to train the policy network. The curriculum learning strategy is implemented by dynamically adjusting the weight coefficients of each reward in the reward function. The training process comprises multiple sequentially occurring stages with increasing difficulty, each defined by a different combination of weight coefficients. The training process is divided into three stages: the initial training stage, the intermediate training stage, and the later training stage.

[0074] In the initial training phase, the focus is on survival (maintaining a standing position); specifically, a higher weight is set for the basic survival reward and a lower weight is set for the motion tracking and topology reward, so that the policy network prioritizes learning to maintain the robot's balance.

[0075] The mid-term training phase focuses on individual trajectory tracking; specifically, once the robot is able to stand stably, the weight of the action tracking reward item is significantly increased, enabling the policy network to learn to track individual movements in the reference trajectory.

[0076] The later training phase focuses on maintaining a refined interaction topology; specifically, based on the humanoid robot's ability to perform individual motion tracking well, the weights of the interaction graph reward items and contact graph reward items are gradually increased, enabling the policy network to further learn and maintain the accurate spatial and physical interaction topology between the two robots.

[0077] In one embodiment of this specification, the switching of stages is automatically triggered based on a preset number of training iterations or the average performance score of the strategy in the simulation environment.

[0078] S35 iteratively optimizes the parameters of the policy network to minimize the negative value of the expected cumulative reward until the policy converges, thereby obtaining the cooperative control policy.

[0079] In the initial training phase, a certain amount of interaction error is allowed to ensure the survival of the humanoid robot (avoiding falls). In the later training phase, the constraint accuracy on the interaction graph and contact graph is gradually tightened.

[0080] This invention explicitly extracts interaction and contact graphs as topological priors and integrates these priors into the reward function of reinforcement learning. By guiding the policy network to focus on the relative relationships between robots rather than their absolute positions, the trained policy can handle complex physically coupled tasks (such as two-person hugging and high-dynamic dance), exhibits extremely high anti-interference capabilities, and significantly improves the robustness of cooperative control.

[0081] S4 deploys the trained collaborative control strategy to the dual humanoid robot, which exchanges status information through real-time communication to perform physical collaborative tasks.

[0082] The trained collaborative control strategy is directly deployed to the dual humanoid robot through a Simulation to Reality (Sim2Real) process. The dual humanoid robot exchanges status information in real time and performs collaborative tasks in the physical world.

[0083] S41 deploys the trained collaborative control strategy to the onboard computing units of the two humanoid robots respectively; S42 The two humanoid robots mentioned above exchange their respective base poses and observation information in real time through a low-latency communication link established by the local area network; Specifically, localization is performed based on the point cloud data (Lidar Points) collected by the lidar to obtain the base pose (Global Pose) in order to achieve state estimation (State Estimation).

[0084] In one embodiment of this specification, the two humanoid robots exchange observation information, namely, the base pose and observation information of the first humanoid robot. ) and the base pose and observation information of the second humanoid robot ( They can share information with each other. The preferred operating frequency for state estimation is 10 Hz.

[0085] S43 For each of the humanoid robots, based on the output of its own cooperative control strategy and the received state information of the other humanoid robot, calculate and apply motor torque to perform a physical cooperative task.

[0086] In one embodiment of this specification, the humanoid robot control execution module converts the cooperative control strategy output into motor torque. During execution, the two humanoid robots exchange their respective state information in real time via a local area network using the Wireless LCM communication protocol. Based on the exchanged information, relative pose calculation is performed, and inter-agent synchronization is achieved, thereby reproducing highly dynamic and strongly coupled cooperative operations in physical space.

[0087] The converged training strategy is directly deployed to the robot's onboard computing unit. During actual operation, the two humanoid robots synchronously reason at a frequency of 50Hz and exchange states in real time via LCM, thereby reproducing the highly dynamic, strongly coupled collaborative tasks shown in Figure 3 in the physical world. Examples include complex interactive tasks such as close contact (e.g., hugging) and spatial collaboration (e.g., transport).

[0088] This invention first transforms multimodal, raw human-to-human interaction data into reference trajectories conforming to the humanoid robot's body structure through Interactive Awareness Redirection (IAMR), resolving the "air handshake" or clipping issues caused by size differences. Subsequently, it guides the training of a policy network using Interaction Graph-Based Reinforcement Learning (IGRL) with extracted topological priors (interaction and contact graphs). Finally, the trained policy is deployed on a dual-humanoid robot (such as the Unitree G1) for real-time collaborative control via a local area network (such as LCM communication). Furthermore, with a robust training strategy and curriculum learning mechanism, policies trained in a simulation environment can be directly deployed to the physical robot (such as the Unitree G1) without cumbersome real-world fine-tuning, achieving zero-shot simulation-to-real transfer. Figure 4 This is a schematic diagram of a collaborative control system for a dual humanoid robot provided in an embodiment of this specification. The system includes: Data acquisition module 410 is used to acquire interaction data between the two presenters; The redirection module 420 is used to redirect the interactive perception motion of the dual humanoid robot based on the interactive data, and generate a reference trajectory and topological prior that conform to the robot's kinematic constraints. The strategy generation module 430 is used to perform reinforcement learning with the reference trajectory as the tracking target and the topological prior as the guide to obtain the cooperative control strategy of the dual humanoid robot. The execution module 440 is used to deploy the trained collaborative control strategy to the dual humanoid robot, which exchanges status information through real-time communication to perform physical collaborative tasks. The redirection module 420 includes: The dual-manifold construction submodule 421 is used to construct, based on the interaction data of the two demonstrators and the body parameters of the two humanoid robots, an individual manifold for constraining the posture of the humanoid robot itself and a unified manifold for constraining the relative geometric relationship between the two humanoid robots. The trajectory generation submodule 422 is used to construct and solve a nonlinear optimization problem with the objective of optimizing the joint angle sequence of the two humanoid robots based on the individual manifold and the unified manifold, and generate the reference trajectory; Graph structure extraction submodule 423 is used to extract an interaction graph representing spatial proximity and a contact graph representing physical contact state from the reference trajectory as the topological prior.

[0089] Optionally, the dual-manifold construction submodule 421 includes: The interactive data is standardized to extract 3D key point sequences; Calculate the individual scaling ratio based on the height ratio between the humanoid robot and the corresponding demonstrator; The individual scaling ratio is applied to globally scale the coordinates of all key points of the corresponding individual in the 3D key point sequence to construct the individual manifold. The average of the scaling ratios of the two individuals is taken as the uniform scaling ratio; The unified scaling ratio is applied to globally scale the relative spatial vectors representing the interaction relationship between two people in the 3D keypoint sequence to construct the unified manifold.

[0090] Optionally, the trajectory generation submodule 422 includes: Based on the individual manifold and the unified manifold, a nonlinear optimization problem is constructed with the objective of optimizing the joint angle sequence of a dual humanoid robot. An objective function is constructed, configured to jointly optimize the self-motion term, the interaction preservation term, and the regularization term. The optimization of the self-motion term includes minimizing the difference between the self-pose of the two humanoid robots and the constraint pose defined by the individual manifold. The optimization of the interaction preservation term includes minimizing the difference between the interaction geometry between the two humanoid robots and the constraint geometry defined by the unified manifold. The optimization of the regularization term includes ensuring the smoothness of the joint angle sequence over time. The nonlinear optimization problem is solved using the objective function as the optimization objective, and a smooth joint angle sequence that satisfies the individual manifold and the unified manifold constraints is generated as the reference trajectory.

[0091] Optionally, the graph structure extraction submodule 423 includes: Based on the unified manifold, identify and connect spatially adjacent key point pairs in the dual humanoid robot; construct the interaction graph based on the key point pairs. Based on the physical simulation detection of the reference trajectory, robot link pairs that are in continuous physical contact are identified and marked; the contact map is constructed based on the robot link pairs.

[0092] Optionally, the strategy generation module 430 includes: Construct an observation space for the policy network; the observation space includes at least the on-body observation information of the humanoid robot, the relative observation information of another humanoid robot, and the historical observation information; Construct a hierarchical coding structure for the policy network; the hierarchical coding structure includes a history encoder for encoding historical action sequences, a future encoder for encoding future reference trajectories, and an action decoder for outputting target joint positions; A reinforcement learning reward function is designed based on the interaction graph and the contact graph. The reward function includes: an interaction graph reward item based on the interaction graph and a contact graph reward item based on the contact graph. The interaction graph reward item is used to penalize deviations between the relative distance between robots and the corresponding side lengths in the interaction graph. The contact graph reward item is used to reward physical contact states that conform to the contact graph and penalize states that do not conform to it. The observation space is used as the input to the policy network, the reward function is used as the training signal, and the policy network is trained using a course learning strategy. The cooperative control policy is obtained by iteratively optimizing the parameters of the policy network to minimize the negative value of the expected cumulative reward until the policy converges.

[0093] Optionally, the execution module 440 includes: The trained collaborative control strategy is deployed to the onboard computing units of the two humanoid robots respectively; The two humanoid robots exchange their respective base poses and observation information in real time through a low-latency communication link established by a local area network; For each humanoid robot, motor torque is calculated and applied based on the output of its own cooperative control strategy and the state information received from the other humanoid robot to perform a physical cooperative task.

[0094] The functions of the system in this embodiment have been described in the above method embodiments. Therefore, for any parts not detailed in this embodiment, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0095] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this specification. The computer device includes a memory 501 and a processor 502. The memory 501 is used to store computer-executable instructions. When the computer-executable instructions are executed by the processor 502, they can implement the steps of the above-described method embodiments.

[0096] Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of this specification. The computer-readable storage medium 600 stores one or more computer programs, which, when executed by a processor, can implement the steps of the above-described method embodiments.

[0097] This specification also provides a computer program product, including a computer program / computer executable instructions, which, when executed by a processor, can implement the steps of the above-described method embodiments.

[0098] Those skilled in the art will understand that all or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. When the computer program is executed, it may include the processes of the embodiments of the above methods.

[0099] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A cooperative control method for two humanoid robots, characterized in that, include: Obtain interaction data from the two presenters; Based on the interactive data, the interactive perception motion of the two humanoid robots is redirected to generate a reference trajectory and topological prior that conform to the robot's kinematic constraints. Using the reference trajectory as the tracking target and the topological prior as guidance, reinforcement learning is performed to obtain the cooperative control strategy of the dual humanoid robot; The trained collaborative control strategy is deployed to the dual humanoid robot, which exchanges status information through real-time communication to perform physical collaborative tasks. The step of redirecting the interactive perception motion of the dual humanoid robot based on the interactive data to generate a reference trajectory and topological prior that conform to the robot's kinematic constraints includes: Based on the interaction data of the two demonstrators and the body parameters of the two humanoid robots, individual manifolds are constructed to constrain the posture of the humanoid robots themselves and a unified manifold is constructed to constrain the relative geometric relationship between the two humanoid robots. Based on the individual manifold and the unified manifold, a nonlinear optimization problem is constructed and solved with the objective of optimizing the joint angle sequence of the two humanoid robots, and the reference trajectory is generated. The interaction graph representing spatial proximity and the contact graph representing physical contact state are extracted from the reference trajectory as the topological prior.

2. The cooperative control method for two humanoid robots as described in claim 1, characterized in that, The step of constructing individual manifolds for constraining the posture of the humanoid robots themselves and a unified manifold for constraining the relative geometric relationship between the two humanoid robots, based on the interaction data of the two demonstrators and the body parameters of the two humanoid robots, includes: The interactive data is standardized to extract 3D key point sequences; Calculate the individual scaling ratio based on the height ratio between the humanoid robot and the corresponding demonstrator; The individual scaling ratio is applied to globally scale the coordinates of all key points of the corresponding individual in the 3D key point sequence to construct the individual manifold. The average of the scaling ratios of the two individuals is taken as the uniform scaling ratio; The unified scaling ratio is applied to globally scale the relative spatial vectors representing the interaction relationship between two people in the 3D keypoint sequence to construct the unified manifold.

3. The cooperative control method for two humanoid robots as described in claim 1, characterized in that, The process of constructing and solving a nonlinear optimization problem based on the individual manifold and the unified manifold, with the objective of optimizing the joint angle sequence of the two humanoid robots, and generating the reference trajectory, includes: Based on the individual manifold and the unified manifold, a nonlinear optimization problem is constructed with the objective of optimizing the joint angle sequence of a two-person humanoid robot. An objective function is constructed, configured to jointly optimize the self-motion term, the interaction preservation term, and the regularization term. The optimization of the self-motion term includes minimizing the difference between the self-pose of the two humanoid robots and the constraint pose defined by the individual manifold. The optimization of the interaction preservation term includes minimizing the difference between the interaction geometry between the two humanoid robots and the constraint geometry defined by the unified manifold. The optimization of the regularization term includes ensuring the smoothness of the joint angle sequence over time. The nonlinear optimization problem is solved using the objective function as the optimization objective, and a smooth joint angle sequence that satisfies the individual manifold and the unified manifold constraints is generated as the reference trajectory.

4. The cooperative control method for two humanoid robots as described in claim 1, characterized in that, The extraction of an interaction graph representing spatial proximity and a contact graph representing physical contact states from the reference trajectory as the topological prior includes: Based on the unified manifold, identify and connect spatially adjacent key point pairs in the dual humanoid robot; construct the interaction graph based on the key point pairs. Based on the physical simulation detection of the reference trajectory, robot link pairs that are in continuous physical contact are identified and marked; the contact map is constructed based on the robot link pairs.

5. The cooperative control method for two humanoid robots as described in claim 1, characterized in that, The cooperative control strategy for the dual humanoid robots, which uses the reference trajectory as the tracking target and the topological prior as guidance for reinforcement learning, includes: Construct an observation space for the policy network; the observation space includes at least the on-body observation information of the humanoid robot, the relative observation information of another humanoid robot, and the historical observation information; Construct a hierarchical coding structure for the policy network; the hierarchical coding structure includes a history encoder for encoding historical action sequences, a future encoder for encoding future reference trajectories, and an action decoder for outputting target joint positions; A reinforcement learning reward function is designed based on the interaction graph and the contact graph. The reward function includes: an interaction graph reward item based on the interaction graph and a contact graph reward item based on the contact graph. The interaction graph reward item is used to penalize deviations between the relative distance between robots and the corresponding side lengths in the interaction graph. The contact graph reward item is used to reward physical contact states that conform to the contact graph and penalize states that do not conform to it. The observation space is used as the input to the policy network, the reward function is used as the training signal, and the policy network is trained using a course learning strategy. The cooperative control policy is obtained by iteratively optimizing the parameters of the policy network to minimize the negative value of the expected cumulative reward until the policy converges.

6. The cooperative control method for two humanoid robots as described in claim 5, characterized in that, The step of deploying the trained collaborative control strategy to the dual humanoid robots, wherein the dual humanoid robots exchange status information through real-time communication to perform physical collaborative tasks, includes: The trained collaborative control strategy is deployed to the onboard computing units of the two humanoid robots respectively; The two humanoid robots exchange their respective base poses and observation information in real time through a low-latency communication link established by a local area network; For each humanoid robot, motor torque is calculated and applied based on the output of its own cooperative control strategy and the state information received from the other humanoid robot to perform a physical cooperative task.

7. A collaborative control system for two humanoid robots, characterized in that, include: The data acquisition module is used to acquire interaction data between the two presenters; The redirection module is used to redirect the interactive perception motion of the dual humanoid robot based on the interactive data, and generate a reference trajectory and topological prior that conform to the robot's kinematic constraints. The strategy generation module is used to perform reinforcement learning with the reference trajectory as the tracking target and the topological prior as the guide to obtain the cooperative control strategy of the dual humanoid robot. An execution module is used to deploy the trained collaborative control strategy to the dual humanoid robot, which exchanges status information through real-time communication to perform physical collaborative tasks. The redirection module includes: The dual-manifold construction submodule is used to construct, based on the interaction data of the two demonstrators and the body parameters of the two humanoid robots, an individual manifold for constraining the posture of the humanoid robot itself and a unified manifold for constraining the relative geometric relationship between the two humanoid robots. The trajectory generation submodule is used to construct and solve a nonlinear optimization problem with the objective of optimizing the joint angle sequence of the two humanoid robots based on the individual manifold and the unified manifold, and generate the reference trajectory. The graph structure extraction submodule is used to extract an interaction graph representing spatial proximity relationships and a contact graph representing physical contact states from the reference trajectory as the topological prior.

8. A computer device, characterized in that, The computer device includes: Processor; and, A memory storing computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs / instructions, which, when executed by a processor, implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program / instruction, which, when executed by a processor, implements the method as described in any one of claims 1-6.

Images