Robot teleoperation method and system, storage medium, equipment and program product

Abstract

The invention provides a robot teleoperation method and system, a storage medium, equipment and a program product. The method comprises the following steps: receiving skeleton movement data of an operator, selecting pose information of part of skeleton nodes from the skeleton movement data, and taking the pose information as reference movement data; mapping the reference motion data into target motion information of the robot; and solving based on the target motion information to generate a control instruction for driving the robot joint. Therefore, the pose information of part of the skeleton nodes is selected from the skeleton motion data and is taken as the reference motion data, so that core motion characteristics can be reserved, the complexity of mapping from the skeleton motion data to the target motion information of the robot in the related technology is simplified, redundant data interference is reduced, and the reliability of the robot is improved. And therefore, the limited joint freedom degree of the robot can be adapted.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of robotics technology, and in particular to a robot teleoperation method and system, storage medium, device, and program product. Background Art

[0002] Related robotic teleoperation methods are typically implemented using handles or specific controllers, making it difficult to accurately map the operator's full-body movements. Optical motion capture-based systems are subject to environmental constraints and high latency. Furthermore, the human skeleton has far more degrees of freedom (e.g., approximately 200) than robot joints (e.g., 30-50). Direct mapping can lead to inverse kinematics solution failures or motion distortion.

[0003] Based on this, embodiments of the present application provide a robot teleoperation method and system, storage medium, device, and program product to improve related technologies. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a robot teleoperation method and system, storage medium, device and program product to adapt to the limited joint freedom of the robot.

[0005] The purpose of the embodiments of the present application is achieved by using the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a robot remote operation method, the method comprising: receiving skeletal motion data of an operator, the skeletal motion data comprising pose information of multiple skeletal nodes of the operator; the pose information comprising position information and / or posture information; selecting pose information of some skeletal nodes from the skeletal motion data as reference motion data; mapping the reference motion data into target motion information of the robot, the target motion information comprising target pose information corresponding to at least one robot joint of the robot; the target pose information comprising target position information and / or target posture information; and performing a solution based on the target motion information to generate control instructions for driving the robot joints.

[0007] In some possible implementations, the skeletal motion data is acquired through an inertial motion capture device.

[0008] In some possible implementations, the selected partial skeletal nodes include one or more of a head node, a neck node, a hip node, a shoulder node, an elbow node, a wrist node, and a finger node.

[0009] In some possible implementations, mapping the reference motion data into the target motion information of the robot includes: determining the posture information of the selected skeletal node as the target posture information of the robot joint; and / or, based on the relative posture relationship between two or more skeletal nodes, using the posture information of the corresponding skeletal nodes to calculate the target posture information of the robot joint.

[0010] In some possible implementations, the solving based on the target motion information includes: modeling the physical constraints that the robot joints need to satisfy as constraint equations in the form of inequalities; the physical constraints include one or more of angle constraints, velocity constraints, acceleration constraints, and collision avoidance constraints; modeling the target motion information as a target equation in the form of an equation; performing optimal solving based on the task equation to obtain control instructions for driving the robot joints; the task equation is determined based on the constraint equation and the target equation.

[0011] In some possible implementations, the optimal solution based on the task equation includes: layering multiple task equations according to priority, wherein the priority of the constraint equation is higher than the priority of the target equation; solving each layer of task equations layer by layer in order from high to low priority, wherein the low-priority task equations are optimally solved within the solution space determined by the high-priority task equations.

[0012] In some possible implementations, the robot control instruction includes joint angle information for indicating a rotation angle of a robot joint.

[0013] In some possible implementations, the solving based on the target motion information includes: solving, based on the target motion information, first joint angle information for indicating the rotation angle of the robot joint; and performing low-pass filtering on the first joint angle information to obtain the joint angle information.

[0014] In some possible implementations, the robot includes a real robot and / or a virtual robot in a simulation environment; or, the robot includes a humanoid robot and / or an animal-like robot.

[0015] In a second aspect, an embodiment of the present application provides a robot teleoperation system, the system comprising: a receiving module for receiving skeletal motion data of an operator, the skeletal motion data including posture information of multiple skeletal nodes of the operator; the posture information including position information and / or attitude information; a selection module for selecting the posture information of some skeletal nodes from the skeletal motion data as reference motion data; a mapping module for mapping the reference motion data into target motion information of the robot, the target motion information including target posture information corresponding to at least one robot joint of the robot; the target posture information including target position information and / or target attitude information; a solver for performing a solution based on the target motion information to generate control instructions for driving the robot joints.

[0016] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above methods is implemented.

[0017] In a fourth aspect, an embodiment of the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any one of the above methods when executing the computer program.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements any of the above methods.

[0019] The present application provides a robot teleoperation method and system, storage medium, device and program product, which includes: receiving the operator's skeletal motion data, selecting the position information of some skeletal nodes from the skeletal motion data, and using it as reference motion data; mapping the reference motion data to the robot's target motion information; solving based on the target motion information to generate control instructions for driving the robot's joints. In this way, selecting the position information of some skeletal nodes from the skeletal motion data and using it as reference motion data can retain the core motion characteristics, simplify the complexity of mapping between skeletal motion data and the robot's target motion information in related technologies, reduce redundant data interference, and thus adapt to the robot's limited joint degrees of freedom. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The embodiments of the present application are further described below with reference to the accompanying drawings and specific implementation methods.

[0021] Figure 1 This is a flow chart of a robot teleoperation method provided in an embodiment of the present application.

[0022] Figure 2 This is a flowchart of the solution based on target motion information provided in an embodiment of the present application.

[0023] Figure 3 This is a module diagram of the robot teleoperation system provided in an embodiment of the present application.

[0024] Figure 4 This is a structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of the embodiments of the present application.

[0026] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0027] The applicant conducted an in-depth analysis of relevant robot remote operation methods and found that: since the degrees of freedom of the human skeleton are much higher than the degrees of freedom of the robot joints, direct mapping will lead to solution failure or unnatural movement; that is, in the process of real-time control of the robot's whole-body joint movement through the operator's skeletal motion data, there is a mapping contradiction between the robot's limited degrees of freedom and the operator's complex skeletal motion data.

[0028] See also Figure 1 , Figure 1 This is a flow chart of a robot teleoperation method provided in an embodiment of the present application.

[0029] In order to improve the mapping contradiction between the limited degrees of freedom of the robot and the complex skeletal motion data of the operator, the embodiment of the present application provides a robot teleoperation method. Figure 1 The method includes steps S100 to S400.

[0030] Step S100: Receive the operator's skeletal motion data. The skeletal motion data includes the position and posture information of multiple skeletal nodes of the operator; the position and posture information includes position information and / or posture information.

[0031] Step S200: Selecting the posture information of some skeleton nodes from the skeleton motion data as reference motion data.

[0032] Step S300: Mapping the reference motion data into target motion information of the robot. The target motion information includes target posture information corresponding to at least one robot joint of the robot; the target posture information includes target position information and / or target posture information.

[0033] Step S400: performing a solution based on the target motion information to generate a control instruction for driving the robot joint.

[0034] In some embodiments, in step S100, skeletal motion data may refer to motion information of a human body (such as an operator) joint in three-dimensional space recorded by corresponding sensors. For example, skeletal motion data may include the posture information of multiple skeletal nodes of an operator. Pose information may refer to a set of spatial states of some or all skeletal nodes of a human body (such as an operator) or a character. Pose information may define the position and / or orientation of relevant joints in the skeletal system, together describing the instantaneous form of the entire body. Pose information may be used to calculate the target motion information of the robot. Skeletal nodes may include any one or more of the head node, neck node, hip node, shoulder node, elbow node, wrist node, and finger node. Skeletal nodes may also include spine nodes, knee nodes, ankle nodes, and foot nodes, etc. Those skilled in the art may edit (such as add, delete, modify, etc.) the types of skeletal nodes involved in skeletal motion data according to actual scenario requirements. Pose information may include position information and / or posture information. For example, pose information may also include the change in position information and / or the change in posture information. Position information, for example, refers to the absolute or relative coordinate values of skeletal nodes in three-dimensional space. Position information can be used to determine the spatial position of a skeletal node. Pose information, for example, refers to the rotational state of a skeletal node in three-dimensional space. Pose information can be used to describe the orientation or angle of a skeletal node. Pose information can be represented using quaternions or Euler angles.

[0035] In some embodiments, in step S100, the skeletal motion data may include the position information of all skeletal nodes of the operator that can be collected. For example, the skeletal motion data may include the position information of the head node, neck node, hip node, shoulder node, elbow node, wrist node, finger node, spine node, knee node, ankle node, and foot node.

[0036] It is understandable that the core of step S200 is data screening and key information extraction. The specific process can be: 1) Input: operator's whole body skeletal motion data (including multiple nodes, such as head, hands, feet, etc.); 2) Operation: Selectively extract the pose information of some skeletal nodes from all skeletal nodes; 3) Output: streamlined reference motion data for subsequent robot motion mapping. In other words, the role of step S200 may include: by selecting key nodes (such as the above-mentioned partial skeletal nodes), reducing redundant data interference and adapting to the robot's limited joint degrees of freedom; reducing computational complexity: for example, the motion capture device may output the pose information of more than 100 skeletal nodes per second. Through step S200, only 10-20 key nodes can be retained, reducing the computational burden of subsequent inverse kinematics (IK) or hierarchical quadratic programming (HQP) algorithms; In addition, by dynamically selecting key nodes, the robot teleoperation method of the embodiment of the present application can cover multiple types of robots (such as real robots and / or virtual robots in simulation environments).

[0037] It can be understood that, in step S200, the reference motion data is a simplified representation of the original skeletal motion data.

[0038] It can be seen that, in some embodiments, the method can achieve approximate tracking of complex human motion by a robot with limited degrees of freedom through the above steps S100 to S400, thereby reducing computational complexity.

[0039] In some embodiments, in step S100, the skeletal motion data may be acquired by an optical motion capture device. The skeletal motion data may also be acquired by other devices (such as a visual motion capture device).

[0040] In order to obtain the operator's skeletal motion data with high precision and low latency, in some embodiments, in step S100 , the skeletal motion data may be obtained by an inertial motion capture device.

[0041] As you can understand, an inertial motion capture device (i.e., inertial motion capture device) is a system that uses inertial sensors (such as accelerometers, gyroscopes, and magnetometers) to track the movement of a person (such as an operator) or an object. It relies on wearable sensors to achieve real-time motion capture without the need for external cameras or markers, allowing for flexible use in various scenarios.

[0042] In some embodiments, the inertial motion capture device can provide real-time full-body motion data of a human body (e.g., an operator). To achieve robot mapping, the requirements for the inertial motion capture device may include one or more of the following: 1) It must be able to collect one or more of the following data: the posture of the head relative to the torso, the position of the upper and lower arms relative to the torso, the posture of the wrist relative to the torso, the position of the fingertips relative to the wrist, and the posture of the waist; 2) It must be able to be mapped into unified skeletal motion data when worn by different operators; 3) It must be able to achieve stable real-time data streaming, and the delay must not exceed a preset threshold (e.g., 100ms).

[0043] In some embodiments, skeletal motion data can be collected and processed by a motion capture device. For example, the motion capture device can collect positional information of the head node and the neck node. The motion capture device can then calculate the relative positional relationship between the head node and the neck node based on the positional information of the head node and the neck node. In other words, the relative positional relationship can be obtained by the motion capture device.

[0044] In some embodiments, skeletal motion data can be collected by a motion capture device and processed by other devices. For example, the motion capture device can collect the position information of the head node and the position information of the neck node. Then, other devices can calculate the relative position relationship between the head node and the neck node based on the position information of the head node and the neck node. In other words, the relative position relationship can be obtained by other devices besides the motion capture device.

[0045] In some embodiments, a motion capture device (such as an inertial motion capture device) and a robot can communicate directly using a built-in or external communication device (such as a network card or router) to transmit skeletal motion data streams calculated from inertial sensors to the robot's industrial computer via UDP (User Datagram Protocol). Within a local area network (LAN), the two devices can communicate directly within the same network segment. Within a public network, the two devices can communicate via public network RTC (Real-Time Communication).

[0046] It can be seen that in some embodiments, the robot teleoperation method can use inertial motion capture equipment (such as a motion capture suit) to collect skeletal motion data in real time, thereby reducing environmental restrictions, adapting to multi-scenario use, and reducing delays.

[0047] Skeletal motion data provided by motion capture equipment and other devices can directly describe the relative positions and posture relationships of the nodes of a complete human skeleton. However, the joint degrees of freedom of a real robot are limited. To achieve the most approximate posture tracking within the limited solution space, in some embodiments, this method can select key information from the skeletal motion data as reference motion data to obtain the robot's target motion information.

[0048] In order to reasonably select key skeletal nodes to balance data simplification and motion expression integrity, in some embodiments, in step S200, the selected partial skeletal nodes may include one or more of head nodes, neck nodes, hip nodes, shoulder nodes, elbow nodes, wrist nodes and finger nodes.

[0049] Those skilled in the art can determine on their own, based on actual scenario requirements, that the selected partial skeletal nodes include one or more of the head node, neck node, hip node, shoulder node, elbow node, wrist node, and finger node.

[0050] It is understandable that the reason for “selecting the posture information of some skeletal nodes from the skeletal motion data” in step S200 is to match the essential requirements of the robot’s degrees of freedom with human motion. In other words, when the joint degrees of freedom of the humanoid robot are much lower than those of the human skeletal nodes (such as fingers with only 6 degrees of freedom and waist with 4 degrees of freedom), approximate control needs to be achieved through key skeletal node mapping. If the posture information of non-key skeletal nodes (such as knees and ankles) is selected from the skeletal motion data to be mapped as the target motion information of the robot, redundant calculations will be increased, reducing the real-time performance of the subsequent generation of corresponding control instructions.

[0051] In some embodiments, the selected partial skeletal nodes can cover the core chain structure of human movement (such as head-torso-arm-fingers), and then the target posture information of the robot joint can be derived through the relative posture relationship (such as the vector obtained from the elbow node and the shoulder node can be mapped to the target orientation of the robot's upper arm).

[0052] It can be seen that in some embodiments, the method can simplify the mapping through step S200, retain the core motion features, reduce redundant data interference, and adapt to the limited joint freedom of the robot; it can reduce the computational complexity and reduce the computational burden of subsequent inverse kinematics (IK) or HQP algorithms; in addition, by dynamically selecting key nodes, the robot teleoperation method of the embodiment of the present application can cover multiple types of robots (such as real robots and / or virtual robots in simulation environments).

[0053] In order to improve the problem of adapting human skeleton data to the differences in robot joint structure, in some embodiments, in step S300, mapping the reference motion data to the target motion information of the robot may include: determining the posture information of the selected skeleton node as the target posture information of the robot joint; and / or, based on the relative posture relationship between two or more skeleton nodes, using the posture information of the corresponding skeleton nodes to calculate the target posture information of the robot joint.

[0054] In some embodiments, in step S300 , the target posture information may include target position information and / or target posture information.

[0055] In some embodiments, in step S200, the posture information of the head node and the neck node can be extracted from the original skeletal motion data. The posture information of the head node and the neck node is used as reference motion data. Thereafter, in step S300, the target posture information of the robot head can be calculated using the posture information of the head node and the neck node. For example, if the extracted position information of the head node is (0.1, 1.75, 0.05) and the extracted position information of the neck node is (0.1, 1.6, 0.05), then the direction vector of the head posture is: (0, 0.15, 0). It can be understood that the direction vector of the head posture can be used as the target posture information of the robot head.

[0056] In some embodiments, in step S200, the pose information of the wrist node, shoulder node, and elbow node can be extracted from the original skeletal motion data. The pose information of the wrist node, shoulder node, and elbow node is used as reference motion data. Thereafter, in step S300, the pose information of the wrist node, shoulder node, and elbow node can be used to calculate the target pose information of the robot's upper arm joint and / or the robot's lower arm joint. For example, if the extracted elbow node position information is (0.6, 1.4, 0.15), the shoulder node pose information is (0.3, 1.5, 0.1), the wrist node position information is (0.8, 1.3, 0.2), and the elbow node position information is (0.6, 1.4, 0.15), then the arm vector can be expressed as: (0.6, 1.4, 0.15) - (0.3, 1.5, 0.1) = (0.3, -0.1, 0.05), and the forearm vector can be expressed as: (0.8, 1.3, 0.2) - (0.6, 1.4, 0.15) = (0.2, -0.1, 0.05). It can be understood that the above arm vector can be used as the target pose information of the robot's arm joint, and the above forearm vector can be used as the target pose information of the robot's forearm joint.

[0057] In some embodiments, in step S200, the posture information of the wrist node can be extracted from the original skeletal motion data. The posture information of the wrist node can include the change in the position information of the wrist node, and / or the change in the posture information of the wrist node. The posture information of the above-mentioned wrist node is used as reference motion data. Thereafter, in step S300, mapping the reference motion data to the target motion information of the robot can be: the posture information of the selected wrist node can be used as the target posture information of the robot wrist joint. In other words, the change in the position information of the wrist node can be used as the target position information of the robot wrist joint, and / or the change in the posture information can be used as the target posture information of the robot wrist joint.

[0058] In some embodiments, in step S200, the posture information of the hip node can be extracted from the original skeletal motion data, and the posture information of the hip node can be used as reference motion data. For example, the posture information of the hip node here can include the initial position information of the hip node (0.1, 1.0, 0) and its current position information (0.1, 0.9, 0), then the displacement vector of the hip node can be expressed as (0, 0.1, 0); similarly, the change in the posture information of the hip node can be obtained based on the initial posture information of the hip node and its current posture information. Afterwards, in step S300, mapping the reference motion data to the target motion information of the robot can be: the displacement vector of the hip node (0, 0.1, 0) can be used as the target position information of the robot's waist joint (such as moving 0.1m downward in the vertical direction), and the change in the posture information can be used as the target posture information of the robot's waist joint.

[0059] In some embodiments, the initial pose and position information of the operator's (e.g., motion capture operator) hip joint can be determined based on the operator's initial calibration pose. For example, when the inertial motion capture device is started, the initial quaternion Q0 and height coordinate y0 of the operator's hip joint can be recorded. The change in pose information can then be calculated using the following expression: , Q_current represents the current posture information of the hip node; the vertical displacement is calculated using the following expression: Δy = y_current - y0, where y_current represents the current height coordinate of the hip node. The yaw and pitch components can then be extracted from the above ΔQ and combined with Δy to form a 4-dimensional control quantity [Δyaw, Δpitch, Δy, 0]. This 4-dimensional control quantity can correspond to the four degrees of freedom (e.g., 2 rotations, 1 translation, and 1 redundant) actuators of the robot's waist joint. In other words, the robot's waist rotation and squat control can be achieved using only the posture information and height coordinates of the hip node.

[0060] In some embodiments, the finger node can be divided into a fingertip sub-node and a finger root sub-node. The direction vector of the finger node can be calculated by the position information of the fingertip sub-node and the finger root sub-node. For example, the difference in the position information of the fingertip sub-node and the finger root sub-node is used as the direction vector of the finger node. The direction vector of the finger node can be used as the target posture information of the robot's finger joint. For example, when the finger node is the thumb, the corresponding control instructions for the two degrees of freedom (such as flexion and extension degrees of freedom and side-swing degrees of freedom) of the robot's thumb can be obtained by performing corresponding calculations on the direction vector of the finger node. When the finger node is one of the remaining four fingers other than the thumb, single-degree-of-freedom approximate control can be adopted, that is, the corresponding control instructions for the single degree of freedom of the corresponding finger of the robot can be obtained according to the modulus of the direction vector of the finger node. The modulus of the direction vector of the finger node can be used to describe the extension or bending posture of the corresponding finger node. That is, for the finger joints, the thumb can have two degrees of freedom to achieve full control (such as independent control of pinching and lateral gripping), while the other four fingers each have one degree of freedom (such as flexion and extension) to achieve approximate control. That is, the other four fingers of the robot can only bend or extend as a whole, and cannot independently control each knuckle. For example, when the operator can pinch a cup with the thumb and index finger, the robot's thumb can adjust the angle through two degrees of freedom to precisely fit the cup wall; the robot's index finger can bend through one degree of freedom to provide basic grip strength. For another example, when the operator makes a fist, each knuckle except the thumb bends in segments, while the four fingers of the robot only bend synchronously at one joint angle, forming an approximate fist shape. It can be seen that even though the robot's fingers have fewer degrees of freedom than human fingers, they can still use the above-mentioned fingertip direction vectors to achieve actions such as grasping.

[0061] It can be seen that, in some embodiments, the method can compress the original skeletal motion data into key motion information (such as the above-mentioned reference motion data) that adapts to the robot's degrees of freedom through step S200.

[0062] After obtaining the key posture information (such as the target motion information mentioned above), in theory, it is possible to directly solve it through inverse kinematics (IK) to calculate the actual joint angle of the robot that matches the above key posture information as much as possible. However, the actual robot system has multiple physical constraints, which makes it difficult for a simple inverse kinematics solution to meet actual needs. That is, the relevant inverse kinematics (IK) cannot handle multiple constraints at the same time, which can easily lead to solution failure or dangerous movement or discontinuous movement. For example, it can include one or more of the following physical constraints: joint limit constraints (such as each joint can only move within a preset angle range), joint speed constraints (such as the joint movement speed cannot exceed the rated upper limit) and collision avoidance constraints (such as the risk of self-collision due to the difference between the robot limb size and the motion capture skeleton). Taking into account the above physical constraints, when solving the joint angle, it cannot be considered a simple inverse kinematics calculation problem.

[0063] In order to achieve optimal motion tracking while satisfying the robot's physical constraints (such as position limits, speed, and collision avoidance), in some embodiments, step S400 is solved based on the target motion information. Figure 2 , step S400 may include steps S410 to S430.

[0064] Step S410: Modeling the physical constraints that the robot joints need to satisfy as constraint equations in the form of inequalities. The physical constraints may include one or more of angle constraints, velocity constraints, acceleration constraints, and collision avoidance constraints.

[0065] Step S420: Model the target motion information as a target equation in the form of an equation.

[0066] Step S430: Optimally solving the task equation to obtain control instructions for driving the robot joints. The task equation can be determined based on the constraint equation and the target equation.

[0067] It should be noted that the order of step S410 and step S420 can be interchanged.

[0068] In some embodiments, in step S410, those skilled in the art can determine the physical constraints that the robot joints need to meet based on the actual scenario requirements. For example, one or more of the following physical constraints may be included: angle constraint: each joint can only move within a preset angle range; speed constraint: the speed of joint movement cannot exceed the rated upper limit; acceleration constraint: the acceleration of joint movement cannot exceed the rated upper limit; collision avoidance constraint: the robot's limb size differs from that of the motion capture equipment or operator, which may lead to self-collision risks. For example, physical constraints may also include torque constraints, etc. Those skilled in the art can use relevant technologies to model physical constraints as constraint equations, and the specific method of modeling is not limited here. The specific form of the physical constraints is also not limited here.

[0069] In some embodiments, in step S410, the constraint equation refers to, for example, an equation that describes physical constraints that the robot must satisfy. For example, the constraint equation can be used to enforce joint velocity limits.

[0070] It is understandable that the technical essence of the constraint equation in the form of an inequality in step S410 is to define the boundaries of the feasible solution space in the form of an inequality. In other words, the role of step S410 is to convert the hardware safety restrictions of the robot into a mathematically solvable form to ensure that the generated joint instructions do not exceed the physically feasible range. The constraint equation in the form of an inequality can prioritize the safety of robot operation. The constraint equation can be the highest priority level in hierarchical optimization. The mathematical essence of the constraint equation here is an inequality constraint.

[0071] In some embodiments, the constraint equation has the highest priority (eg, level 0) to ensure the safety of the robot's motion.

[0072] In some embodiments, in step S420, the target equation, for example, refers to an equation that describes the desired motion tracking target. For example, the target equation can be used to optimize the tracking accuracy of the end effector. Those skilled in the art can use relevant techniques to model the target motion information into a target equation in the form of an equation, and the specific modeling method is not limited here. For example, the target equation can be in the form of: Jq˙=v desired , where J represents the Jacobian matrix of the robot kinematic chain; v desired Indicates the expected velocity corresponding to the key pose of motion capture (such as the end velocity corresponding to the target wrist pose).

[0073] It is understandable that in step S420, since it is required to consider the actual physical constraints at the same time on the basis of the original key posture information, this will make the already tight joint degrees of freedom even more insufficient, the solution space will be further compressed, and sometimes it will even be impossible to find a solution that satisfies all equations at the same time. Therefore, the idea of hierarchical optimization can be further introduced. The technical essence of the target equation in the form of an equation is to define the ideal motion trajectory through equality constraints. In other words, the effect of step S420 is to convert the desired motion target obtained by mapping (such as the above-mentioned target motion information) into a tracking target in a mathematical optimization problem. Under the premise of satisfying the constraints of step S410, the motion similarity is maximized through equation optimization. The target equation can be an optimization target of lower priority in the HQP algorithm. The mathematical essence of the target equation here is an equality constraint.

[0074] In some embodiments, in step S420, the target equation can be described by the following analytical form:

[0075] for k∈{0,1,…}:

[0076] find x k ,min||υ k ||+||w k ||

[0077]

[0078] Among them, L k Represents a layer consisting of all objective equations and constraint equations with a priority of k; the smaller the layer, the higher the priority; x represents the control variable, for example, for kinematic control, x can be generalized velocity; for dynamic control, x can be generalized acceleration and contact force; v k and w k Indicates L k The slack variables of the level, for example, when v k and w k When it is equal to 0, the optimization problem has an exact solution, otherwise it is an approximate solution; for example, when v1≠0, the goal is allowed to be achieved inaccurately; A k x=b k The objective equation can be expressed in equation form; for example, for kinematic control, A k It can be expressed as the Jacobian matrix, b k represents the desired target speed; Represents the constraint equation in the form of an inequality.

[0079] It is understandable that the meaning of hierarchical optimization is that the upper layer problem must be optimized within the scope of ensuring the results of the lower layer. For example:

[0080] find x k+1, min v k+1 ||+||w k+1 ||

[0081]

[0082] in, All indicate that L j The slack variables of a level cannot be changed when optimizing problems in subsequent levels. There is no priority difference between the constraint equations and the objective equations in the same level. If there is a conflict between the constraint equations and the objective equations, neither of them will satisfy the obtained solution. The objective equation in the form of an equation in the previous level will directly reduce the dimension of the optimization variables in the subsequent levels, while the dimension of the constraint equation increases with the level, which will cause the objective equation and constraint equations of the upper level to not be strictly satisfied.

[0083] In some embodiments, in step S430, the task equation refers to, for example, a combination of a constraint equation and a target equation in a hierarchical optimization framework.

[0084] It is understandable that the role of step S430 is to coordinate the conflict between the constraint equation and the target equation and generate feasible and approximately optimal joint instructions. The technical essence of the task equation is to use slack variables (such as v k and w k ) enables flexible handling of constraint and objective equations; solution spaces are transferred between levels, meaning that higher-level solutions serve as the boundaries of the feasible regions of lower-level solutions. The mathematical essence of the task equation here is a hierarchical optimization problem formulation.

[0085] It can be understood that the constraint equations can ensure safety, the objective equations can achieve motion tracking accuracy, and the task equations can resolve multi-objective optimization conflicts through a hierarchical strategy.

[0086] In some embodiments, step S400 of the present method combines the above-mentioned HQP algorithm to describe the physical constraints as optimal solution equations in the form of inequalities (such as constraint equations), and describes the key motion information as optimal solution equations in the form of equations (such as target equations). The two types of equations are considered simultaneously to obtain the task equations, and then the optimal solution is performed. In this way, the calculated joint angles can theoretically track the desired motion targets while satisfying the actual robot constraints.

[0087] In some embodiments, in step S430, an optimal solution is performed based on the task equation. Step S430 may include steps S431 to S432.

[0088] Step S431: multiple task equations are layered according to priority, wherein the priority of the constraint equation is higher than the priority of the target equation.

[0089] Step S432: Solve each layer of task equations layer by layer in order of priority from high to low, wherein the task equations of low priority are optimized within the solution space determined by the task equations of high priority.

[0090] It is understandable that the function of step S431 may include: sorting multiple types of task equations (such as constraint equations and target equations) according to safety importance to form a hierarchical structure (such as Layer 0>Layer 1>...).

[0091] In some embodiments, in step S431, the high-priority layer (such as Layer 0) may contain all constraint equations in the form of inequalities (such as joint limits, velocity constraints, and collision avoidance constraints in step S410). The high-priority layer (such as Layer 0) can ensure that the solution is within the physical feasible domain of the robot, reducing hardware damage or dangerous actions. The low-priority layer (such as Layer 1 to Layer N) may contain target equations in the form of equations (such as the end tracking and joint smoothness-related target equations in step S420). The low-priority layer can optimize motion performance under the premise of safety. For example, if the operator's arm moves quickly, the high-priority layer (such as Layer 0) will first limit the joint speed through the constraint equation, and the low-priority layer (such as Layer 1) will then optimize the hand position tracking through the target equation.

[0092] It is understandable that the function of step S432 may include: solving in order of priority, with the high-level solution space serving as the input constraint of the low-level.

[0093] In some embodiments, in step S432, for example, for high-level solutions, the constraint equations of Layer 0 can be rigorously solved first to obtain a feasible solution space (e.g., all q˙ that satisfy joint velocity constraints). If the constraint equations conflict (e.g., the target velocity exceeds the limit), an approximate solution can be obtained by using slack variables. For low-level optimization, the objective equations of Layer 1 can be optimized within the solution space of Layer 0 (e.g., minimizing hand pose error). This is then passed layer by layer, ultimately outputting joint commands that balance safety and tracking performance. Step S432 can reduce the problem of multiple solutions caused by redundant degrees of freedom in related IK calculations. When the objective equations cannot be fully satisfied (e.g., collision avoidance causes hand deviation), a suboptimal feasible solution can still be generated. For example, if the operator's hand's target pose requires joints to exceed the limits to be achieved, the HQP algorithm can produce the following results: Layer 0 forces the joint angles to remain within the limits; Layer 1 brings the hand as close to the target as possible within the remaining solution space; and the resulting output can be: the hand stops at the limit boundary, rather than forcibly tracking and causing mechanical damage.

[0094] As can be seen, in some embodiments, this solution can resolve the conflict between safety and tracking accuracy in robot teleoperation methods through step S430, namely, through a hierarchical priority mechanism. In other words, this method not only separates the constraint equations from the objective equation, reducing the limitations of a single optimization objective, but also adjusts the weights of each layer based on real-time conditions (e.g., temporarily reducing tracking accuracy during emergency collision avoidance). Furthermore, this method can also adapt to the physical limitations of different robot configurations (e.g., humanoid, bionic).

[0095] As can be seen, in some embodiments, this method can utilize the aforementioned HQP solution strategy, setting constraint equations that affect machine operational safety to a high priority. The solver will prioritize satisfying these physical constraints, then optimize the solution within the remaining solution space based on the objective equations determined by the motion capture data, calculating joint angles that are as close as possible to the operator's (e.g., the motion capture operator's) motion posture. Advantages of this setup may include one or more of the following: the robot's joint motion will inevitably satisfy the physical constraints, reducing the frequency of dangerous situations; the robot will closely approximate the operator's (e.g., the motion capture operator's) motion, reducing the frequency of solution failures resulting in motion failure; and the ability to adjust the weights of various objective equations at the same level (e.g., trade-offs between wrist tracking accuracy and arm tracking accuracy). In other words, this method can utilize the aforementioned HQP solution strategy, modeling the physical constraints as the aforementioned constraint equations and simultaneously transforming the key posture tracking objectives into the aforementioned objective equations. Through multi-objective collaborative optimization, the resulting joint angles can both strictly satisfy the physical constraints and maximize the approximation of the operator's motion intentions. This method can ensure motion safety (e.g., joints do not exceed limits) and approximate the motion capture posture within the remaining solution space.

[0096] Since the robot's underlying actuators (such as servo motors) usually require clear joint angle instructions (rather than abstract kinematic targets), the motion capture data must ultimately be converted into joint angle information that can be directly driven. Therefore, the relevant robot teleoperation methods have the problem of control instruction standardization; in addition, since different robot hardware (such as serial joints and parallel mechanisms) have different requirements for the format of control instructions, the joint angle information is universal, so the relevant robot teleoperation methods have compatibility requirements.

[0097] In order to improve the above-mentioned control instruction standardization and compatibility requirements, in some embodiments, the robot control instruction may include joint angle information for indicating the rotation angle of the robot joint.

[0098] It is understood that "joint angle information indicating the rotation angles of robot joints" provides a hardware-independent command output format. This joint angle information is compatible with most robot control systems and also provides a clear input data type for subsequent filtering.

[0099] Because the HQP solution strategy described above can be affected by sensor noise, network latency, and other factors, the output of the first joint angle information may contain high-frequency jitter. Direct execution can lead to discontinuous robot motion (such as "stuttering" or trembling), thus creating problems with solution noise and jitter. Furthermore, at high control frequencies (e.g., 100Hz), there's a need to balance filter smoothness with response latency, creating a conflict between real-time performance and smoothness.

[0100] In order to improve the above-mentioned solution noise and jitter problems and the contradiction between real-time and smoothness, in some embodiments, in step S400, the solution is performed based on the target motion information, which may include: according to the target motion information, solving and obtaining the first joint angle information for indicating the rotation angle of the robot joint; and performing low-pass filtering on the first joint angle information to obtain the joint angle information.

[0101] It is understandable that the role of "solving the first joint angle information for indicating the rotation angle of the robot joint according to the target motion information" may include: generating the original discrete joint angle instructions based on the hierarchical optimization results of the HQP solution strategy. For example, the input may be target motion information (such as end pose, joint constraints); the output may be an unfiltered joint angle sequence, and the joint angle sequence may contain solution noise or jumps. For example, when the operator waves his hands quickly, the above-mentioned HQP solution strategy may cause slight jitter in the output angle due to the iteration step size limit. The specific process of "solving the first joint angle information for indicating the rotation angle of the robot joint" can directly adopt the existing technology in the field, so it will not be repeated here.

[0102] It can be understood that the effect of "performing low-pass filtering on the first joint angle information to obtain joint angle information" may include: performing real-time filtering on the first joint angle information to generate a smooth final joint angle instruction.

[0103] In some embodiments, the analytical form of the low-pass filtering process can be (taking any joint angle y as an example):

[0104] y[k]=(1-α)·y[k-1]+α·x[k];

[0105] Where x[k] represents the input joint angle at the current moment; y[k] represents the output joint angle after filtering; α∈(0|1) represents the filter coefficient, which reflects the filter's response to the new input. The coefficient α can be adjusted based on the sampling period T and the desired cutoff frequency ω. c(unit is rad / s) for adjustment, for example: α = T / (T+τ), where τ = 1 / ω c The low-pass filtering process may also adopt other forms, which are not specifically limited here.

[0106] It can be seen that in some embodiments, the low-pass filter used in the present method utilizes exponential weighted averaging, which can suppress the high-frequency noise of the input signal, and can have a significant smoothing effect on the jitter of the solver solution and network packet loss. The corresponding parameters can be adjusted according to the actual movement of the robot. For example, the smaller the parameter, the smoother the output. In addition, the corresponding parameters should be made as large as possible while ensuring the relative smoothness of the joint movement to obtain better motion capture tracking effect.

[0107] It can be seen that in some embodiments, the present method can perform low-pass filtering (such as exponential weighted averaging) on the output joint angle (such as the first joint angle information mentioned above); the smoothness and delay can be balanced by adjusting the filter coefficient, thereby suppressing high-frequency noise and improving movement smoothness.

[0108] As can be seen, in some embodiments, this method can use the aforementioned HQP solution strategy to calculate discrete robot joint angles. However, due to the relatively time-consuming computation of the HQP solution strategy and communication delays, the frequency sent to the hardware layer is often insufficient or unstable, which directly affects the joint movement, resulting in stuttering or noise. To address this issue, this method can employ a low-pass filtering algorithm to low-pass filter the joint angle commands obtained by the HQP solution strategy, thereby achieving smoother and safer control.

[0109] In order to improve the problem of limited applicable scenarios (simulation / real, humanoid / non-humanoid), in some embodiments, the robot may include a real robot and / or a virtual robot in a simulation environment; or, the robot may include a humanoid robot and / or an animal bionic robot.

[0110] It is understood that any of the above-described robot teleoperation methods can seamlessly switch to a virtual robot in a simulation environment, reducing development costs. For example, an operator (such as a motion capture actor) can control a virtual robot to perform movie special effects movements; the virtual robot can be trained in simulation to adapt to complex gaits and then deployed to a real robot.

[0111] It is understood that the robot teleoperation method in any of the above embodiments can be adapted to humanoid robots and / or animal-inspired robots. Animal-inspired robots can include quadrupedal robot dogs, snake-like robots, and the like. Humanoid robots can focus more on whole-body coordination (e.g., walking + arm manipulation); animal-inspired robots can focus more on specific movement patterns (e.g., quadrupedal running, bird flight). For example, a humanoid robot can use motion capture to control its arms to grasp objects; a robot dog can achieve crawling or jumping by mirroring the operator's limb movements.

[0112] In some embodiments, the HQP solution strategy can adapt to the degree of freedom differences of a humanoid robot or an animal-inspired robot by adjusting the Jacobian matrix and physical constraints.

[0113] The following is an introduction to the general process of the robot teleoperation method of an embodiment of the present application. The motion capture device (such as an inertial motion capture device) can collect the skeletal motion data generated by the operator's movement in real time, and send the skeletal motion data to the robot's industrial computer through UDP network communication. The original motion capture skeletal data is mapped and converted to obtain key motion posture information. The key posture information is handed over to the HQP solver for algorithm solution. During the solution process, in addition to meeting the expected motion goals as much as possible, priority is given to ensuring that the actual physical constraints of the robot are met. The discrete joint angles obtained by the HQP solution are low-pass filtered, and the joint angle motion commands that are both smooth and highly followable are calculated by adjusting the appropriate low-pass filter parameters. The motion commands (such as the above-mentioned control instructions) are sent to the robot hardware layer for execution, thereby realizing real-time, physically constrained, and highly similar humanoid robot motion capture teleoperation.

[0114] It can be seen that, in some embodiments, the present method can support real robots, virtual robots, humanoid robots and animal bionic robots, which can enhance the universality of the present method and cover multiple purposes such as training and testing.

[0115] The above is a description of a method for remote operation of a robot. Figure 3, an embodiment of the present application also provides a robot teleoperation system. The system includes a receiving module 100, a selection module 200, a mapping module 300 and a solver 400. The receiving module 100 is used to receive the operator's skeletal motion data, and the skeletal motion data includes the posture information of multiple skeletal nodes of the operator; the posture information includes position information and / or posture information. The selection module 200 is used to select the posture information of some skeletal nodes from the skeletal motion data as reference motion data. The mapping module 300 is used to map the reference motion data into the target motion information of the robot, and the target motion information includes the target posture information corresponding to at least one robot joint of the robot; the target posture information includes target position information and / or target posture information. The solver 400 is used to solve based on the target motion information to generate control instructions for driving the robot joints.

[0116] It should be noted that the relevant content of each module in this robotic teleoperation system has already been discussed in the robotic teleoperation method described above. The technical effects achieved by this robotic teleoperation system are essentially the same as those of the robotic teleoperation method described above, so we will not elaborate on them here.

[0117] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any one of the robot teleoperation methods in the above embodiments is implemented.

[0118] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements any one of the robot teleoperation methods in the above embodiments.

[0119] The computer program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the computer program product of the embodiments of the present application is not limited thereto, and the computer program product may be any combination of one or more computer-readable media.

[0120] See also Figure 4 , Figure 4 This is a structural block diagram of a computer device provided in an embodiment of the present application.

[0121] An embodiment of the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the above-mentioned robot teleoperation methods when executing the computer program.

[0122] The embodiments of the present application do not limit the computer device, which may be, for example, a local computer device, a cloud computer device, a distributed computer device, etc.

[0123] The computer device may include: a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected via an internal connection path.

[0124] The memory 110 is used to store computer programs. In some implementations, the computer programs may include codes for implementing the methods of the embodiments of the present application.

[0125] The processor 120 is configured to execute the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information and output data such as operation results. In some implementations, when the solutions of the embodiments of the present application are implemented through software or firmware, the computer program for implementing the solutions of the embodiments of the present application may be stored in the processor 120 and executed by the processor 120.

[0126] The memory 110 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, any memory of these and other suitable types. As an example, the memory 110 includes a random access memory (RAM), a cache memory and a read-only memory (ROM). Among them, the memory 110 stores a computer program, and the computer program can be executed by the processor 120 so that the processor 120 implements the steps of any of the above methods.

[0127] The processor 120 may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor, or the processor 120 may be any conventional processor.

[0128] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 120 or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor 120. The software module can be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 110, and the processor 120 reads the information in the memory 110 and completes the steps of the above method in combination with its hardware. To reduce repetition, it will not be described in detail here.

[0129] In some implementations, in addition to the hardware units described above, the computer device may also include software modules, where the software modules may be, for example, an operating system, a basic input and output system (BIOS), application software, etc.

[0130] An operating system manages the hardware and / or software resources of a computer device and is the core and cornerstone of the computer. It handles basic tasks such as managing and allocating memory, prioritizing the supply and demand of system resources, controlling input and output devices, operating the network, and managing the file system. To facilitate user operation, most operating systems provide an interface for users to interact with the system.

[0131] The BIOS is used to run hardware initialization during the power-on boot phase and provide runtime services for the operating system and applications. In some implementations, the BIOS can also monitor and display the processor temperature and execute functions such as adjusting temperature protection strategies.

[0132] Application software, also known as an application program, is software written for a specific user purpose. It is a major category of computer software. For example, application software might be a program used for power control, temperature management, and other purposes.

[0133] It should be understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation methods of this application, rather than to limit the scope of protection of this application.

[0134] It can be understood that in various implementations of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the present application.

[0135] It can be understood that the various embodiments described in this application can be implemented individually or in combination, and this application is not limited to this.

[0136] Unless otherwise indicated, all technical and scientific terms used in this application have the same meaning as those commonly understood by those skilled in the art in the art of this application. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit the scope of this application. The singular forms "a," "above," and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0137] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0138] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described embodiments may refer to the corresponding processes in other embodiments and will not be repeated here.

[0139] In the embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0140] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the objectives of the technical solutions of this application.

[0141] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0142] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0143] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A robot teleoperation method, characterized in that: The method comprises: Receive skeletal motion data of an operator, wherein the skeletal motion data includes position information of multiple skeletal nodes of the operator; the position information includes position information and / or posture information; Selecting the pose information of some skeletal nodes from the skeletal motion data as reference motion data; Mapping the reference motion data into target motion information of the robot, wherein the target motion information includes target posture information corresponding to at least one robot joint of the robot; the target posture information includes target position information and / or target posture information; A solution is performed based on the target motion information to generate control instructions for driving the robot joints.

2. The robot teleoperation method according to claim 1, characterized in that: The skeletal motion data is acquired through an inertial motion capture device.

3. The robot teleoperation method according to claim 1, characterized in that: The selected partial skeleton nodes include one or more of a head node, a neck node, a hip node, a shoulder node, an elbow node, a wrist node, and a finger node.

4. The robot teleoperation method according to claim 1, characterized in that: Mapping the reference motion data into target motion information of the robot includes: Determining the pose information of the selected skeletal node as the target pose information of the robot joint; and / or, Based on the relative pose relationship between two or more skeletal nodes, the target pose information of the robot joint is calculated using the pose information of the corresponding skeletal nodes.

5. The robot teleoperation method according to claim 1, characterized in that: The solving based on the target motion information includes: Modeling physical constraints that the robot joints need to satisfy as constraint equations in the form of inequalities; the physical constraints include one or more of angle constraints, velocity constraints, acceleration constraints, and collision avoidance constraints; Modeling the target motion information as a target equation in equation form; An optimal solution is performed based on a task equation to obtain a control instruction for driving a robot joint; the task equation is determined based on the constraint equation and the target equation.

6. The robot teleoperation method according to claim 5, characterized in that: The optimal solution based on the task equation includes: Layering the multiple task equations according to priority, wherein the priority of the constraint equation is higher than the priority of the target equation; The task equations of each layer are solved layer by layer in order of priority from high to low, wherein the task equations of low priority are solved optimally within the solution space determined by the task equations of high priority.

7. The robot teleoperation method according to claim 1, characterized in that: The robot control instruction includes joint angle information for indicating a rotation angle of a robot joint.

8. The robot teleoperation method according to claim 7, characterized in that: The solving based on the target motion information includes: Obtaining, based on the target motion information, first joint angle information indicating a rotation angle of a robot joint; Low-pass filtering is performed on the first joint angle information to obtain the joint angle information.

9. The robot teleoperation method according to claim 1, characterized in that: The robot includes a real robot and / or a virtual robot in a simulation environment; or, The robots include humanoid robots and / or animal-like robots.

10. A robot teleoperation system, characterized in that: The system comprises: A receiving module, configured to receive skeletal motion data of an operator, wherein the skeletal motion data includes position information of multiple skeletal nodes of the operator; the position information includes position information and / or posture information; A selection module, configured to select the posture information of some skeletal nodes from the skeletal motion data as reference motion data; a mapping module, configured to map the reference motion data into target motion information of the robot, wherein the target motion information includes target posture information corresponding to at least one robot joint of the robot; the target posture information includes target position information and / or target posture information; A solver is used to solve based on the target motion information to generate control instructions for driving the robot joints.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

12. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 9 when executing the computer program.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images