Robotic teleoperation method and system, storage medium, device and program product

Abstract

The application provides a robot teleoperation method and system, a storage medium, equipment and a program product. The method receives bone motion data of an operator, selects pose information of part of bone nodes from the bone motion data as reference motion data, maps the reference motion data into target motion information of a robot, and solves based on the target motion information to generate a control instruction for driving a joint of the robot. In this way, the pose information of part of the bone nodes is selected from the bone motion data and used as the reference motion data, the core motion characteristics can be retained, the complexity of the mapping between the bone motion data and the target motion information of the robot in the related technology is simplified, the redundant data interference is reduced, and the limited joint degrees of freedom of the robot can be adapted.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and more particularly to robot teleoperation methods and systems, storage media, devices, and program products. Background Technology

[0002] Related robot teleoperation methods are typically implemented through handles or specific controllers, making it difficult to accurately map the operator's full-body movements. Optical motion capture-based systems are limited by environmental conditions and suffer from high latency. Furthermore, the degrees of freedom of the human skeleton (approximately 200) far exceed the degrees of freedom of robot joints (approximately 30-50), and direct mapping may lead to failure in inverse kinematics solutions or motion distortion.

[0003] Based on this, embodiments of this application provide robot teleoperation methods and systems, storage media, devices, and program products to improve related technologies. Summary of the Invention

[0004] The purpose of this application is to provide a robot teleoperation method and system, storage medium, device and program product to adapt to the limited joint degrees of freedom of a robot.

[0005] The objective of this application embodiment is achieved using the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a robot teleoperation method, the method comprising: receiving skeletal motion data of an operator, the skeletal motion data including pose information of multiple skeletal nodes of the operator; the pose information including position information and / or posture information; selecting pose information of a portion of skeletal nodes from the skeletal motion data as reference motion data; mapping the reference motion data to target motion information of the robot, the target motion information including target pose information corresponding to at least one robot joint of the robot; the target pose information including target position information and / or target posture information; and solving based on the target motion information to generate control commands for driving robot joints.

[0007] In some possible implementations, the skeletal motion data is acquired via an inertial motion capture device.

[0008] In some possible implementations, the selected skeletal nodes include one or more of the following: head node, neck node, hip node, shoulder node, elbow node, wrist node, and finger node.

[0009] In some possible implementations, mapping the reference motion data to the robot's target motion information includes: determining the pose information of selected skeletal nodes as the target pose information of the robot joint; and / or, calculating the target pose information of the robot joint using the pose information of the corresponding skeletal nodes based on the relative pose relationship between two or more skeletal nodes.

[0010] In some possible implementations, the solution based on the target motion information includes: modeling the physical constraints that the robot joints need to satisfy as constraint equations in inequality form; the physical constraints include one or more of angle constraints, velocity constraints, acceleration constraints, and collision avoidance constraints; modeling the target motion information as a target equation in equation form; performing optimal solution based on the task equation to obtain control commands for driving the robot joints; the task equation is determined based on the constraint equation and the target equation.

[0011] In some possible implementations, the optimal solution based on the task equations includes: layering multiple task equations according to priority, wherein the priority of the constraint equations is higher than that of the objective equations; solving each layer of task equations in descending order of priority, wherein, within the solution space determined by the high-priority task equations, the low-priority task equations are optimized.

[0012] In some possible implementations, the robot control commands include joint angle information for indicating the rotation angle of the robot joints.

[0013] In some possible implementations, the step of solving based on the target motion information includes: obtaining first joint angle information for indicating the rotation angle of the robot joints based on the target motion information; and performing low-pass filtering on the first joint angle information to obtain the joint angle information.

[0014] In some possible implementations, the robot includes a real robot and / or a virtual robot in a simulated environment; or, the robot includes a humanoid robot and / or an animal-inspired robot.

[0015] Secondly, embodiments of this application provide a robot teleoperation system, the system comprising: a receiving module for receiving skeletal motion data of an operator, the skeletal motion data including pose information of multiple skeletal nodes of the operator; the pose information including position information and / or posture information; a selection module for selecting pose information of a portion of skeletal nodes from the skeletal motion data as reference motion data; a mapping module for mapping the reference motion data to target motion information of the robot, the target motion information including target pose information corresponding to at least one robot joint of the robot; the target pose information including target position information and / or target posture information; and a solver for solving based on the target motion information to generate control commands for driving robot joints.

[0016] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described above.

[0017] Fourthly, embodiments of this application provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement any of the methods described above.

[0018] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements any of the methods described above.

[0019] This application provides a robot teleoperation method and system, storage medium, device, and program product. The method includes: receiving skeletal motion data from an operator; selecting pose information of a portion of skeletal nodes from the skeletal motion data and using it as reference motion data; mapping the reference motion data to target motion information of the robot; and solving based on the target motion information to generate control commands for driving robot joints. By selecting pose information of a portion of skeletal nodes from the skeletal motion data and using it as reference motion data, core motion features can be preserved, simplifying the complexity of mapping from skeletal motion data to target motion information of the robot in related technologies, reducing redundant data interference, and thus adapting to the limited joint degrees of freedom of the robot. Attached Figure Description

[0020] The embodiments of this application are further described below with reference to the accompanying drawings and specific implementation details.

[0021] Figure 1 This is a flowchart illustrating a robot teleoperation method provided in an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of the process for solving based on target motion information provided in an embodiment of this application.

[0023] Figure 3 This is a schematic diagram of the robot teleoperation system provided in the embodiments of this application.

[0024] Figure 4 This is a structural block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this application.

[0026] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0027] The applicant conducted an in-depth analysis of relevant robot teleoperation methods and found that: since the degrees of freedom of the human skeleton are much higher than the degrees of freedom of the robot joints, direct mapping will lead to solution failure or unnatural movement; that is, in the process of controlling the movement of the robot's whole body joints in real time through the operator's skeletal motion data, there is a mapping contradiction between the robot's limited degrees of freedom and the operator's complex skeletal motion data.

[0028] See Figure 1 , Figure 1 This is a flowchart illustrating a robot teleoperation method provided in an embodiment of this application.

[0029] To address the mapping discrepancy between the limited degrees of freedom of a robot and the complex skeletal motion data of an operator, this application provides a robot teleoperation method. See also... Figure 1 The method includes steps S100 to S400.

[0030] Step S100: Receive the operator's skeletal motion data. The skeletal motion data includes the pose information of multiple skeletal nodes of the operator; the pose information includes position information and / or posture information.

[0031] Step S200: Select the pose information of some bone nodes from the skeletal motion data as reference motion data.

[0032] Step S300: Map the reference motion data to the robot's target motion information. The target motion information includes target pose information corresponding to at least one robot joint; the target pose information includes target position information and / or target orientation information.

[0033] Step S400: Solve based on target motion information to generate control commands for driving robot joints.

[0034] In some embodiments, in step S100, skeletal motion data can refer to motion information of human joints (such as an operator) in three-dimensional space recorded by corresponding sensors. For example, skeletal motion data can include pose information of multiple skeletal nodes of the operator. Pose information can refer to a set of spatial states of some or all skeletal nodes of a human body (such as an operator) or character. Pose information can define the position and / or orientation of related joints in the skeletal system, collectively describing the instantaneous shape of the overall body. Pose information can be used to calculate the target motion information of the robot. Skeletal nodes can include any one or more of head nodes, neck nodes, hip nodes, shoulder nodes, elbow nodes, wrist nodes, and finger nodes. Skeletal nodes can also include spine nodes, knee nodes, ankle nodes, and foot nodes, etc. Those skilled in the art can edit (e.g., add, delete, modify, etc.) the types of skeletal nodes involved in the skeletal motion data according to the actual needs of the scenario. Pose information can include position information and / or posture information. For example, pose information can also include the amount of change in position information and / or the amount of change in posture information. Position information, for example, refers to the absolute or relative coordinate values ​​of skeletal nodes in three-dimensional space. Position information can be used to determine the spatial location of skeletal nodes. Pose information, for example, refers to the rotational state of a skeletal node in three-dimensional space. Pose information can be used to describe the orientation or angular changes of a skeletal node. Pose information can be represented using quaternions or Euler angles, etc.

[0035] In some embodiments, in step S100, the aforementioned skeletal motion data may include pose information of all skeletal nodes of the operator that can be acquired. For example, the aforementioned skeletal motion data may include pose information of head nodes, neck nodes, hip nodes, shoulder nodes, elbow nodes, wrist nodes, finger nodes, spine nodes, knee nodes, ankle nodes, and foot nodes.

[0036] It is understandable that the core of step S200 is data filtering and key information extraction. The specific process can be as follows: 1) Input: Operator's whole-body skeletal motion data (including multiple nodes, such as head, hands, feet, etc.); 2) Operation: Selectively extract the pose information of some skeletal nodes from all skeletal nodes; 3) Output: Simplified reference motion data for subsequent robot motion mapping. In other words, the role of step S200 can include: reducing redundant data interference by selecting key nodes (such as the aforementioned partial skeletal nodes), adapting to the robot's limited joint degrees of freedom; reducing computational complexity: for example, the motion capture device may output pose information of more than 100 skeletal nodes per second, but step S200 can retain only 10-20 key nodes, reducing the computational burden of subsequent inverse kinematics (IK) or hierarchical quadratic programming (HQP) algorithms; in addition, by dynamically selecting key nodes, the robot teleoperation method of this application embodiment can cover multiple types of robots (such as real robots and / or virtual robots in simulation environments).

[0037] Understandably, in step S200, the reference motion data is a simplified representation of the original skeletal motion data.

[0038] As can be seen, in some embodiments, this method can achieve approximate tracking of complex human motion by a robot with finite degrees of freedom through the above steps S100 to S400, thereby reducing computational complexity.

[0039] In some embodiments, in step S100, skeletal motion data can be acquired using an optical motion capture device. Skeletal motion data can also be acquired using other devices (such as a visual motion capture device).

[0040] In order to acquire the operator's skeletal motion data with high accuracy and low latency, in some embodiments, the skeletal motion data can be acquired by an inertial motion capture device in step S100.

[0041] As can be understood, inertial motion capture equipment (i.e., inertial motion capture devices) is a system that tracks the movement of a human body (such as an operator) or object using inertial sensors (such as accelerometers, gyroscopes, magnetometers, etc.). It does not require external cameras or markers, and can complete real-time motion capture by relying on wearable sensors, making it suitable for flexible scenarios.

[0042] In some embodiments, the inertial motion capture device can provide real-time full-body motion data of a human body (such as an operator). To achieve mapping to a robot, the requirements for the inertial motion capture device may include one or more of the following: 1) It must be able to collect one or more of the following data: head posture relative to the torso, upper arm and forearm positions relative to the torso, wrist posture relative to the torso, fingertip positions relative to the wrist, and waist posture; 2) It must be able to map to uniform skeletal motion data when worn by different operators; 3) It must be able to achieve stable real-time data stream transmission with a latency not exceeding a preset threshold (e.g., 100ms).

[0043] In some embodiments, skeletal motion data can be acquired and processed by a motion capture device. For example, the motion capture device can acquire the position information of the head nodes and the neck nodes, and then calculate the relative pose relationship between the head nodes and the neck nodes based on the position information of the head nodes and the neck nodes. In other words, the relative pose relationship can be obtained by the motion capture device.

[0044] In some embodiments, skeletal motion data can be acquired by a motion capture device and processed by other devices. For example, the motion capture device can acquire the positional information of the head nodes and the neck nodes, and then other devices can calculate the relative pose relationship between the head nodes and the neck nodes based on the positional information of the head nodes and the neck nodes. That is, the relative pose relationship can be obtained by devices other than the motion capture device.

[0045] In some embodiments, the motion capture device (such as an inertial motion capture device) and the robot can send the skeletal motion data stream calculated from the inertial sensor to the robot's industrial control computer via UDP (User Datagram Protocol) through built-in or external communication devices (such as network cards or routers). Under local area network conditions, the two can communicate directly on the same network segment; under cross-public network conditions, the two can communicate through public network RTC (Real-Time Communication).

[0046] As can be seen, in some embodiments, the robot teleoperation method can use inertial motion capture devices (such as motion capture suits) to collect skeletal motion data in real time, thereby reducing environmental limitations, adapting to multiple scenarios, and reducing latency.

[0047] Skeletal motion data provided by motion capture devices can describe the relative positions and posture relationships between the nodes of a complete human skeleton. However, the joint degrees of freedom of a real robot are limited. In order to achieve the closest possible posture tracking within a finite solution space, in some embodiments, this method can select some key information from the skeletal motion data as reference motion data to obtain the robot's target motion information.

[0048] In order to reasonably select key skeletal nodes to balance data simplification and motion representation integrity, in some embodiments, the selected skeletal nodes in step S200 may include one or more of the following: head nodes, neck nodes, hip nodes, shoulder nodes, elbow nodes, wrist nodes, and finger nodes.

[0049] Those skilled in the art can determine the selected skeletal nodes according to the actual needs of the scenario, including one or more of the following: head nodes, neck nodes, hip nodes, shoulder nodes, elbow nodes, wrist nodes, and finger nodes.

[0050] Understandably, the reason for "selecting pose information of some skeletal nodes from the skeletal motion data" in step S200 is to match the essential requirements of robot degrees of freedom with human motion. In other words, when the joint degrees of freedom of a humanoid robot are far lower than those of human skeletal nodes (e.g., fingers have only 6 degrees of freedom, and the waist has 4 degrees of freedom), approximate control needs to be achieved through mapping of key skeletal nodes. However, selecting pose information of non-key skeletal nodes (such as knees and ankles) from the skeletal motion data to map to the robot's target motion information would increase redundant calculations and reduce the real-time performance of subsequent control command generation.

[0051] In some embodiments, the selected skeletal nodes can cover the core chain structure of human movement (such as head-torso-arm-finger), thereby enabling the derivation of the target pose information of the robot joints through relative pose relationships (such as the vectors obtained from elbow nodes and shoulder nodes can be mapped to the target orientation of the robot's upper arm).

[0052] As can be seen, in some embodiments, this method can simplify the mapping through step S200, retain the core motion features, reduce redundant data interference, and adapt to the limited joint degrees of freedom of the robot; it can reduce the computational complexity and reduce the computational burden of subsequent inverse kinematics (IK) or HQP algorithms; in addition, by dynamically selecting key nodes, the robot teleoperation method of this application embodiment can cover multiple types of robots (such as real robots and / or virtual robots in simulation environments).

[0053] To address the issue of adapting human skeletal data to differences in robot joint structures, in some embodiments, step S300, mapping reference motion data to target motion information for the robot, may include: determining the pose information of selected skeletal nodes as the target pose information for the robot joint; and / or, calculating the target pose information for the robot joint using the pose information of the corresponding skeletal nodes based on the relative pose relationship between two or more skeletal nodes.

[0054] In some embodiments, in step S300, the target pose information may include target position information and / or target pose information.

[0055] In some embodiments, in step S200, the pose information of the head node and neck node can be extracted from the original skeletal motion data. This pose information is used as reference motion data. Then, in step S300, the target pose information of the robot head can be calculated using this pose information. For example, if the extracted head node position information is (0.1, 1.75, 0.05) and the extracted neck node position information is (0.1, 1.6, 0.05), then the head pose direction vector is (0, 0.15, 0). It can be understood that the head pose direction vector can be used as the target pose information of the robot head.

[0056] In some embodiments, in step S200, the pose information of the wrist node, shoulder node, and elbow node can be extracted from the original skeletal motion data. This pose information is used as reference motion data. Then, in step S300, the target pose information of the robot's upper arm joint and / or forearm joint can be calculated using this pose information. For example, if the extracted elbow node position information is (0.6, 1.4, 0.15), the shoulder node pose information is (0.3, 1.5, 0.1), the wrist node position information is (0.8, 1.3, 0.2), and the elbow node position information is (0.6, 1.4, 0.15), then the upper arm vector can be represented as: (0.6, 1.4, 0.15) - (0.3, 1.5, 0.1) = (0.3, -0.1, 0.05), and the forearm vector can be represented as: (0.8, 1.3, 0.2) - (0.6, 1.4, 0.15) = (0.2, -0.1, 0.05). It can be understood that the above upper arm vector can be used as the target pose information for the robot's upper arm joint, and the above forearm vector can be used as the target pose information for the robot's forearm joint.

[0057] In some embodiments, in step S200, the pose information of the wrist node can be extracted from the original skeletal motion data. The pose information of the wrist node may include the change in the position information of the wrist node and / or the change in the posture information of the wrist node. This wrist node pose information is used as reference motion data. Then, in step S300, mapping the reference motion data to the robot's target motion information may involve using the selected wrist node pose information as the target pose information of the robot's wrist joint. That is, the change in the position information of the wrist node can be used as the target position information of the robot's wrist joint, and / or, the change in posture information can be used as the target posture information of the robot's wrist joint.

[0058] In some embodiments, in step S200, the pose information of the hip node can be extracted from the original skeletal motion data, and the pose information of the hip node can be used as reference motion data. For example, the pose information of the hip node may include the initial position information (0.1, 1.0, 0) and its current position information (0.1, 0.9, 0) of the hip node, and the displacement vector of the hip node can be represented as (0, 0.1, 0). Similarly, the change in the pose information of the hip node can be obtained based on the initial pose information and its current pose information. Then, in step S300, mapping the reference motion data to the target motion information of the robot can be done by using the displacement vector (0, 0.1, 0) of the hip node as the target position information of the robot's lumbar joint (e.g., moving downwards by 0.1m in the vertical direction), and using the change in the pose information as the target pose information of the robot's lumbar joint.

[0059] In some embodiments, the initial attitude and position information of the operator's (e.g., a motion capture operator's) hip node can be determined through the operator's initial calibration posture. For example, when the inertial motion capture device is started, the initial quaternion Q0 and height coordinate y0 of the operator's hip node can be recorded. Then, the change in attitude information can be calculated using the following expression: Q_current represents the current attitude information of the hip node. The vertical displacement is calculated using the following expression: Δy = y_current - y0, where y_current represents the current height coordinate of the hip node. Then, the yaw and pitch components can be extracted from ΔQ and combined with Δy to form a 4D control variable [Δyaw, Δpitch, Δy, 0]. This 4D control variable corresponds to the four degrees of freedom (e.g., two rotations, one translation, and one redundant) actuator of the robot's waist joint. In other words, waist rotation and squatting / standing control of the robot can be achieved using only the attitude information and height coordinates of the hip node.

[0060] In some embodiments, finger nodes can be divided into fingertip sub-nodes and finger root sub-nodes. The direction vector of the finger node can be calculated using the position information of the fingertip and finger root sub-nodes. For example, the difference between the position information of the fingertip and finger root sub-nodes can be used as the direction vector of the finger node. The direction vector of the finger node can serve as the target pose information of the robot's finger joint. For example, when the finger node is the thumb, the corresponding control commands for the two degrees of freedom (such as flexion-extension and lateral swing) of the robot's thumb can be obtained by calculating the direction vector of the finger node. When the finger node is one of the other four fingers besides the thumb, a single degree of freedom approximation control can be used, that is, the corresponding control command for the single degree of freedom of the robot's corresponding finger can be obtained based on the magnitude of the direction vector of the finger node. The magnitude of the direction vector of the finger node can be used to describe the extension or flexion posture of the corresponding finger node. In other words, for the finger joints, the thumb can have two degrees of freedom to achieve full control (such as independent control of pinching and lateral gripping), while the other four fingers each have one degree of freedom (such as flexion and extension) to achieve approximate control. That is, the robot's other four fingers can only bend or extend as a whole, and cannot independently control each joint. For example, when an operator can pinch a cup with their thumb and forefinger, the robot's thumb can adjust its angle using two degrees of freedom to precisely fit the cup wall; the robot's forefinger can bend using one degree of freedom to provide basic grip strength. As another example, when an operator makes a fist, the operator's finger joints (excluding the thumb) bend segmentally, while the robot's four fingers bend synchronously at only one joint angle, forming an approximate fist shape. It can be seen that even though the robot's fingers have fewer degrees of freedom than human fingers, the robot's fingers can still use the aforementioned fingertip direction vectors to achieve grasping and other actions.

[0061] As can be seen, in some embodiments, this method can compress the original skeletal motion data into key motion information (such as the reference motion data mentioned above) that is adapted to the robot's degrees of freedom through step S200.

[0062] After obtaining key posture information (such as the target motion information mentioned above), theoretically, inverse kinematics (IK) can be used to directly calculate the actual joint angles of the robot to best match the aforementioned key posture information. However, real-world robot systems have multiple physical constraints, making simple inverse kinematics solutions insufficient for practical needs. In other words, related inverse kinematics (IK) cannot handle multiple constraints simultaneously, easily leading to solution failures, dangerous movements, or discontinuous motion. For example, these constraints may include one or more of the following physical constraints: joint limit constraints (e.g., each joint can only move within a preset angle range), joint velocity constraints (e.g., joint movement speed cannot exceed the rated upper limit), and collision avoidance constraints (e.g., due to differences between the robot's limb size and the motion-captured skeleton, there may be a risk of self-collision). Considering these physical constraints, solving for joint angles cannot be considered a simple inverse kinematics calculation problem.

[0063] To achieve optimal motion tracking while satisfying robot physical constraints (such as limits, velocity, and collision avoidance), in some embodiments, step S400 involves solving based on target motion information. See also Figure 2 Step S400 may include steps S410 to S430.

[0064] Step S410: Model the physical constraints that the robot joints need to satisfy as constraint equations in the form of inequalities. Physical constraints may include one or more of the following: angle constraints, velocity constraints, acceleration constraints, and collision avoidance constraints.

[0065] Step S420: Model the target motion information into an equation in the form of a target equation.

[0066] Step S430: Solve the optimal solution based on the task equations to obtain control commands for driving the robot joints. The task equations can be determined based on the constraint equations and the objective equations.

[0067] It should be noted that the order of steps S410 and S420 can be interchanged.

[0068] In some embodiments, in step S410, those skilled in the art can determine the physical constraints that the robot joints need to meet according to the actual needs of the scenario. For example, these constraints may include one or more of the following: angle constraints: each joint can only move within a preset angle range; velocity constraints: the speed of joint movement cannot exceed the rated upper limit; acceleration constraints: the acceleration of joint movement cannot exceed the rated upper limit; collision avoidance constraints: the robot's limb dimensions differ from the motion capture device or operator, which may lead to a risk of self-collision. For example, physical constraints may also include torque constraints, etc. Those skilled in the art can use relevant technologies to model the physical constraints into constraint equations; the specific modeling method is not limited here. Nor is the specific form of the physical constraints limited here.

[0069] In some embodiments, in step S410, the constraint equation refers to an equation describing the physical constraints that the robot must satisfy. For example, the constraint equation can be used to enforce joint velocity limits.

[0070] It is understandable that in step S410, the technical essence of the constraint equations in inequality form is to define the boundary of the feasible solution space through inequality. In other words, the role of step S410 is to transform the robot's hardware safety constraints into a mathematically solvable form, ensuring that the generated joint commands do not exceed the physically feasible range. The constraint equations in inequality form prioritize ensuring the robot's operational safety. The constraint equations can be the highest priority level in hierarchical optimization. The mathematical essence of the constraint equations here is inequality constraints.

[0071] In some embodiments, the constraint equations have the highest priority (e.g., level 0) to ensure the safety of robot movement.

[0072] In some embodiments, in step S420, the target equation refers to, for example, an equation describing the desired motion tracking objective. For instance, the target equation can be used to optimize the tracking accuracy of the end effector. Those skilled in the art can use relevant techniques to model the target motion information into an equation of equal form; the specific modeling method is not limited here. For example, the target equation could be in the form: Jq˙=v desired Where J represents the Jacobian matrix of the robot's kinematic chain; v desired This indicates the desired velocity corresponding to the key motion capture posture (such as the end effector velocity corresponding to the wrist target pose).

[0073] Understandably, in step S420, since actual physical constraints are required to be considered in addition to the existing key attitude information, the already strained joint degrees of freedom become even more insufficient, further compressing the solution space. Sometimes, it may even be impossible to find a solution that simultaneously satisfies all equations. Therefore, the idea of ​​hierarchical optimization can be further introduced. The technical essence of the objective equation in the form of equations is to define the ideal motion trajectory through equality constraints. That is, the role of step S420 is to transform the mapped desired motion target (such as the target motion information mentioned above) into a tracking target in a mathematical optimization problem. Under the premise of satisfying the constraints of step S410, motion similarity is maximized through equality optimization. The objective equation can be a lower-priority optimization objective in the HQP algorithm. The mathematical essence of the objective equation here is equality constraints.

[0074] In some embodiments, in step S420, the objective equation can be described by the following analytical form:

[0075] for k∈{0,1,…}:

[0076] find x k ,min||υ k ||+||w k ||

[0077]

[0078] Among them, L k This represents the hierarchy of all objective and constraint equations with priority k; the smaller the hierarchy, the higher the priority; x represents the control variables, for example, for kinematic control, x can be generalized velocity; for dynamic control, x can be generalized acceleration and contact force; v k and w k L represents k Hierarchical slack variables, for example, when v k and w k When v1 equals 0, the optimization problem has a strict solution; otherwise, an approximate solution is obtained. For example, when v1 ≠ 0, the goal can be achieved imprecisely. k x = b k The objective equation can be expressed in equation form; for example, for kinematic control, A k It can represent the Jacobian matrix, b k Indicates the desired target speed; Represent the constraint equations in inequality form.

[0079] It's understandable that hierarchical optimization means that the upper-level problem must be optimized within the scope of ensuring the results of the lower-level problem. For example:

[0080] find x k+1,min v k+1 ||+||w k+1 ||

[0081]

[0082] in, All indicate that L is satisfied j The slack variables at each level cannot be changed when optimizing problems at subsequent levels; there is no priority distinction between constraint equations and objective equations at the same level. If there is a conflict between the constraint equations and the objective equations, neither will be satisfied in the resulting solution; the objective equation in the form of equality at the previous level will directly reduce the dimension of optimization variables at subsequent levels, while the dimension of constraint equations increases with level, which may lead to the objective equations and constraint equations at the higher level not being strictly satisfied.

[0083] In some embodiments, in step S430, the task equation refers to, for example, the combination of the constraint equation and the objective equation within a hierarchical optimization framework.

[0084] Understandably, the purpose of step S430 is to reconcile the conflict between the constraint equations and the objective equations, generating feasible and near-optimal joint commands. The technical essence of the task equations is to relax variables (such as v...) k and w k This allows for flexible handling of constraint equations and objective equations; it also enables solution space transfer between levels, where higher-level solutions serve as the boundaries of feasible regions for lower levels. The mathematical essence of the task equations here is a hierarchical optimization problem formulation.

[0085] Understandably, the constraint equations ensure safety. The objective equations achieve motion tracking accuracy. The task equations can resolve multi-objective optimization conflicts through a hierarchical strategy.

[0086] In some embodiments, step S400 of this method combines the above-mentioned HQP algorithm to describe physical constraints as optimal solution equations in inequality form (such as constraint equations) and key motion information as optimal solution equations in equality form (such as target equations). By considering both types of equations simultaneously, the task equation is obtained, and then the optimal solution is performed. In this way, the calculated joint angles can theoretically track the desired motion target while satisfying the actual robot constraints.

[0087] In some embodiments, step S430 involves finding the optimal solution based on the task equation. Step S430 may include steps S431 to S432.

[0088] Step S431: Hierarchically classify multiple task equations according to their priority, where the priority of the constraint equations is higher than that of the objective equations.

[0089] Step S432: Solve the task equations at each level in descending order of priority. Within the solution space determined by the high-priority task equations, optimize the solution of the low-priority task equations.

[0090] It is understandable that step S431 may include: sorting multiple types of task equations (such as constraint equations and objective equations) according to their safety importance to form a hierarchical structure (such as Layer 0>Layer 1>…).

[0091] In some embodiments, in step S431, high-priority layers (such as Layer 0) may contain all constraint equations in inequality form (such as joint limiting, velocity constraints, and collision avoidance constraints in step S410). High-priority layers (such as Layer 0) can ensure that the solution is within the robot's physically feasible region, reducing hardware damage or dangerous actions. Low-priority layers (such as Layer 1 to Layer N) may contain objective equations in equation form (such as the end-effector tracking and joint smoothness-related objective equations in step S420). Low-priority layers can optimize motion performance while ensuring safety. For example, if the operator's arm moves rapidly, high-priority layers (such as Layer 0) prioritize limiting joint velocity through constraint equations, while low-priority layers (such as Layer 1) then optimize hand position tracking through objective equations.

[0092] It is understandable that step S432 may include solving the problem in order of priority, with the solution space of higher levels serving as the input constraints of lower levels.

[0093] In some embodiments, in step S432, for example, for high-level solutions, the constraint equations of Layer 0 can be rigorously solved first to obtain a feasible solution space (such as all q˙ that satisfy the joint velocity limits); if the constraint equations conflict (such as the target velocity exceeding the limit), an approximate solution can be obtained by relaxing variables. For low-level optimization, the objective equation of Layer 1 (such as minimizing the hand pose error) can be optimized within the solution space of Layer 0; this is passed layer by layer, and the final output is a joint command that balances safety and tracking performance. Step S432 can reduce the multi-solution problem caused by redundant degrees of freedom in related IK calculations; when the objective equation is difficult to fully satisfy (such as hand displacement caused by collision avoidance), a suboptimal feasible solution can still be generated. For example, the operator's hand target pose requires joints to exceed the limits to reach; the result of the above HQP algorithm can be: Layer 0 forces the joint angles not to exceed the limits; Layer 1 makes the hand as close to the target as possible within the remaining solution space; the output result can be: the hand stops at the limit boundary, rather than being forcibly tracked and causing mechanical damage.

[0094] As can be seen, in some embodiments, this solution can resolve the conflict between safety and tracking accuracy in robot teleoperation methods through step S430, i.e., through a hierarchical priority mechanism. In other words, this method not only separates the constraint equations from the objective equations, reducing the limitations of a single optimization objective, but also adjusts the weights of each layer according to the real-time state (e.g., temporarily reducing tracking accuracy during emergency collision avoidance). Furthermore, this method can adapt to the physical limitations of different robot configurations (e.g., humanoid, biomimetic).

[0095] As can be seen, in some embodiments, this method can utilize the aforementioned HQP solution strategy to set the constraint equations affecting machine operation safety as high priority. The solver will prioritize satisfying these physical constraints and then optimize the solution in the remaining solution space based on the objective equations determined by the motion capture data. This calculates joint angle results that are as close as possible to the operator's (e.g., motion capture operator's) motion posture. The advantages of this setup include one or more of the following: robot joint movements will always satisfy physical constraints, reducing the frequency of dangerous situations; the robot will approximate the operator's (e.g., motion capture operator's) movements as closely as possible, reducing the frequency of solution failures leading to inability to move; and it allows adjustment of the weights of various objective equations at the same level (e.g., adjusting wrist tracking accuracy and arm tracking accuracy). In other words, this method can adopt the aforementioned HQP solution strategy, modeling physical constraints as the aforementioned constraint equations, while simultaneously transforming key posture tracking targets into the aforementioned objective equations; through multi-objective collaborative optimization, the final generated joint angles can both strictly satisfy physical constraints and approximate the operator's motion intentions to the greatest extent. This method can ensure motion safety (e.g., joints do not exceed limits) and approximate the motion capture posture within the remaining solution space.

[0096] Because the underlying actuators of robots (such as servo motors) usually require explicit joint angle commands (rather than abstract kinematic targets), motion capture data must be ultimately converted into directly driveable joint angle information. Therefore, there is a standardization problem for control commands in related robot teleoperation methods. In addition, because different robot hardware (such as serial joints and parallel mechanisms) have different requirements for the format of control commands, and joint angle information has universality, there is a compatibility problem in related robot teleoperation methods.

[0097] To address the aforementioned issues of standardization and compatibility in control commands, in some embodiments, robot control commands may include joint angle information to indicate the rotation angle of robot joints.

[0098] Understandably, "joint angle information used to indicate the rotation angle of robot joints" provides a hardware-independent instruction output format. This joint angle information is compatible with most robot control systems. It also provides a clear input data type for subsequent filtering processes.

[0099] Because the aforementioned HQP solution strategy may be affected by sensor noise, network latency, etc., the output first joint angle information may have high-frequency jitter; if executed directly, it will cause discontinuous robot motion (such as "stuttering" or trembling), thus there are problems of solution noise and jitter. In addition, under high control frequency requirements (such as 100Hz), it is necessary to balance the smoothness of filtering and response delay, that is, there is a contradiction between real-time performance and smoothness.

[0100] To improve the aforementioned problems of noise and jitter in the solution process, as well as the contradiction between real-time performance and smoothness, in some embodiments, step S400, which involves solving based on the target motion information, may include: obtaining first joint angle information for indicating the rotation angle of the robot joints based on the target motion information; and performing low-pass filtering on the first joint angle information to obtain the joint angle information.

[0101] It is understandable that the function of "obtaining the first joint angle information for indicating the robot joint rotation angle based on the target motion information" can include: generating the original discrete joint angle command based on the hierarchical optimization results of the HQP solution strategy. For example, the input can be target motion information (such as end-effector pose, joint constraints); the output can be an unfiltered joint angle sequence, which may contain solution noise or jumps. For example, when the operator waves their hand quickly, the above HQP solution strategy may slightly jitter the output angle due to the limitation of the iteration step size. The specific process of "obtaining the first joint angle information for indicating the robot joint rotation angle" can directly adopt the existing technology in this field, so it will not be described in detail here.

[0102] It is understandable that the function of "low-pass filtering the first joint angle information to obtain joint angle information" can include: real-time filtering of the first joint angle information to generate a smooth final joint angle command.

[0103] In some embodiments, the analytical form of low-pass filtering can be (taking arbitrary joint angle y as an example):

[0104] y[k]=(1-α)·y[k-1]+α·x[k];

[0105] Where x[k] represents the input joint angle at the current moment; y[k] represents the filtered output joint angle; α∈(0|1) represents the filtering coefficient, reflecting the filter's response to new inputs. The coefficient α can be determined based on the sampling period T and the desired cutoff frequency ω. cAdjustments are made (in rad / s), for example: α = T / (T + τ), where τ = 1 / ω c Low-pass filtering can also take other forms, and no specific restrictions are imposed here.

[0106] As can be seen, in some embodiments, the low-pass filter used in this method utilizes exponential weighted averaging, which can suppress high-frequency noise in the input signal and has a significant smoothing effect on jitter in the solver's solution and network packet loss. The corresponding parameters can be adjusted according to the actual robot's motion; for example, the smaller the parameters, the smoother the output. In addition, while ensuring relatively smooth joint motion, the corresponding parameters should be made as large as possible to obtain better motion capture tracking results.

[0107] As can be seen, in some embodiments, this method can perform low-pass filtering (such as exponential weighted averaging) on ​​the output joint angles (such as the first joint angle information mentioned above); by adjusting the filtering coefficients, smoothness and delay can be balanced, thereby suppressing high-frequency noise and improving motion smoothness.

[0108] As can be seen, in some embodiments, this method can use the aforementioned HQP solution strategy to calculate discrete robot joint angles. However, due to the relatively time-consuming computation of the HQP solution strategy, coupled with communication delays, the frequency sent to the hardware layer is often insufficient or unstable. The direct impact is that the joints exhibit stuttering or noise during movement. To address this issue, this method employs a low-pass filtering algorithm to perform low-pass filtering on the joint angle commands obtained from the aforementioned HQP solution strategy, thereby achieving smoother and safer control.

[0109] To address the limitation of applicable scenarios (simulation / physical, humanoid / non-humanoid), in some embodiments, the robot may include a real robot and / or a virtual robot in a simulated environment; or, the robot may include a humanoid robot and / or an animal-inspired robot.

[0110] It is understood that the robot teleoperation method in any of the above embodiments can be seamlessly switched to a virtual robot in a simulation environment, reducing development costs. For example, an operator (such as a motion capture actor) can control a virtual robot to perform movie special effects actions; the virtual robot can be trained in simulation to adapt to complex gaits and then deployed to a real robot.

[0111] It is understood that the robot teleoperation method in any of the above embodiments can be adapted to humanoid robots and / or animal-like robots. Animal-like robots can be quadrupedal robotic dogs, snake-like robots, etc. Humanoid robots can focus more on overall coordination (such as walking + arm manipulation); animal-like robots can focus more on specific movement patterns (such as quadrupedal running, bird flight). For example, a humanoid robot can use motion capture to control its arm to grasp objects; a robotic dog can crawl or jump by mapping the operator's limb movements.

[0112] In some embodiments, the above-described HQP solution strategy can be adapted to the differences in degrees of freedom between humanoid robots or animal-inspired robots by adjusting the Jacobian matrix and physical constraints.

[0113] The following is a general description of the robot teleoperation method according to one embodiment of this application. A motion capture device (such as an inertial motion capture device) can collect skeletal motion data generated by the operator's movements in real time and send the skeletal motion data to the robot's industrial control computer via UDP network communication. The original motion capture skeletal data is mapped and transformed to obtain key motion posture information. The key posture information is then passed to an HQP solver for algorithmic solving. During the solving process, in addition to satisfying the desired motion target as much as possible, priority is given to ensuring that the actual physical constraints of the robot are met. The discrete joint angles obtained from the HQP solution are filtered using a low-pass algorithm. By adjusting appropriate low-pass filter parameters, a smooth and highly responsive joint angle motion command is calculated. The motion command (such as the control instructions mentioned above) is sent to the robot hardware layer for execution, realizing real-time, physically constrained, and highly similar humanoid robot motion capture teleoperation.

[0114] As can be seen, in some embodiments, this method can support real robots, virtual robots, humanoid robots and animal bionic robots, which can enhance the universality of this method and cover multiple uses such as training and testing.

[0115] The above is a description of a robot teleoperation method. See also... Figure 3This application also provides a robot teleoperation system. The system includes a receiving module 100, a selection module 200, a mapping module 300, and a solver 400. The receiving module 100 receives skeletal motion data from an operator, including pose information of multiple skeletal nodes; the pose information includes position information and / or posture information. The selection module 200 selects pose information of a subset of skeletal nodes from the skeletal motion data as reference motion data. The mapping module 300 maps the reference motion data to target motion information for the robot, including target pose information corresponding to at least one robot joint; the target pose information includes target position information and / or target posture information. The solver 400 solves based on the target motion information to generate control commands for driving the robot joints.

[0116] It should be noted that the relevant content of each module in the robot teleoperation system described here has already been discussed in the previous section on robot teleoperation methods. The technical effects achievable by this robot teleoperation system are basically the same as those of the robot teleoperation methods described earlier. Therefore, they will not be repeated here.

[0117] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the robot teleoperation methods described above.

[0118] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements any of the robot teleoperation methods described above.

[0119] The computer program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the computer program product of the embodiments of this application is not limited thereto, and the computer program product may be any combination of one or more computer-readable media.

[0120] See Figure 4 , Figure 4 This is a structural block diagram of a computer device provided in an embodiment of this application.

[0121] This application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the above-described robot teleoperation methods.

[0122] The embodiments of this application do not limit the computer device, which may be, for example, a local computer device, a cloud computer device, a distributed computer device, etc.

[0123] The computer device may include: a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected through internal connection paths.

[0124] The memory 110 is used to store computer programs, which in some implementations may include code for implementing the methods of the embodiments of this application.

[0125] The processor 120 executes the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information, and output operation results and other data. In some implementations, when the solutions of the embodiments of this application are implemented by software or firmware, the computer program used to implement the solutions of the embodiments of this application can be stored in the processor 120 and executed by the processor 120.

[0126] The memory 110 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, any memory of these and other suitable types. As an example, the memory 110 includes random access memory (RAM), cache memory, and read-only memory (ROM). The memory 110 stores a computer program that can be executed by processor 120, causing processor 120 to implement the steps of any of the methods described above.

[0127] The processor 120 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 120 can be any conventional processor.

[0128] In implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 120 or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor 120. The software modules can be located in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in the memory 110, and the processor 120 reads the information in the memory 110 and, in conjunction with its hardware, completes the steps of the above method. To reduce repetition, detailed descriptions are not provided here.

[0129] In some implementations, in addition to the hardware units described above, computer devices may also include software modules, such as operating systems, basic input / output systems (BIOS), and application software.

[0130] An operating system is used to manage the hardware and / or software resources of a computer device; it is the kernel and foundation of the computer. The operating system handles fundamental tasks such as managing and configuring memory, determining the priority of system resource allocation, controlling input and output devices, operating the network, and managing the file system. To facilitate user operation, most operating systems provide a user interface for interaction with the system.

[0131] The BIOS is used to perform hardware initialization during the power-on boot phase and to provide runtime services for the operating system and applications. In some implementations, the BIOS can also monitor and display processor temperature and execute temperature protection strategies.

[0132] Application software, also known as an application program, can be understood as software written for a specific user application purpose, and is one of the main categories of computer software. For example, application software can be a program used to achieve purposes such as power control and temperature management.

[0133] It is understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation of this application, and are not intended to limit the scope of protection of this application.

[0134] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0135] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and this application does not limit them.

[0136] Unless otherwise stated, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0137] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0138] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the embodiments described above can be referred to the corresponding processes in other embodiments, and will not be repeated here.

[0139] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0140] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the technical solution in this application, depending on actual needs.

[0141] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0142] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to related technologies, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for teleoperating a robot, characterized in that, The method includes: Receive skeletal motion data of the operator, the skeletal motion data including pose information of multiple skeletal nodes of the operator; the pose information includes position information and / or posture information; The pose information of some bone nodes is selected from the bone motion data as reference motion data; The reference motion data is mapped to the robot's target motion information, which includes target pose information corresponding to at least one robot joint; the target pose information includes target position information and / or target posture information. The target motion information is used to solve the problem and generate control commands for driving the robot joints. The step of solving based on the target motion information includes: The physical constraints that the robot joints need to satisfy are modeled as constraint equations in the form of inequalities. The physical constraints include one or more of the following: angle constraints, velocity constraints, acceleration constraints, and collision avoidance constraints. The target motion information is modeled as a target equation in equation form; The multiple task equations determined based on the constraint equations and the target equations are hierarchically arranged according to priority, wherein the priority of the constraint equations is higher than that of the target equations. The task equations at each level are solved in descending order of priority. Within the solution space determined by the high-priority task equations, the low-priority task equations are optimized to obtain the first joint angle information used to indicate the rotation angle of the robot joints. The first joint angle information is low-pass filtered to obtain joint angle information, and the control command includes the joint angle information.

2. The robot teleoperation method according to claim 1, characterized in that, The skeletal motion data is acquired through an inertial motion capture device.

3. The robot teleoperation method according to claim 1, characterized in that, The selected skeletal nodes include one or more of the following: head nodes, neck nodes, hip nodes, shoulder nodes, elbow nodes, wrist nodes, and finger nodes.

4. The robot teleoperation method according to claim 1, characterized in that, The step of mapping the reference motion data to the robot's target motion information includes: The pose information of the selected skeletal nodes is determined as the target pose information of the robot joints; and / or, Based on the relative pose relationship between two or more skeletal nodes, the target pose information of the robot joint is calculated using the pose information of the corresponding skeletal nodes.

5. The robot teleoperation method according to claim 1, characterized in that, The robot includes real robots and / or virtual robots in a simulated environment; or, The robots include humanoid robots and / or animal-inspired robots.

6. A robot teleoperation system, characterized in that, The system includes: A receiving module is used to receive skeletal motion data of the operator, the skeletal motion data including the pose information of multiple skeletal nodes of the operator; the pose information includes position information and / or posture information; The selection module is used to select the pose information of a portion of the bone nodes from the bone motion data as reference motion data. A mapping module is used to map the reference motion data into target motion information of the robot, wherein the target motion information includes target pose information corresponding to at least one robot joint of the robot; the target pose information includes target position information and / or target posture information; A solver is used to solve based on the target motion information to generate control commands for driving robot joints; The step of solving based on the target motion information includes: The physical constraints that the robot joints need to satisfy are modeled as constraint equations in the form of inequalities. The physical constraints include one or more of the following: angle constraints, velocity constraints, acceleration constraints, and collision avoidance constraints. The target motion information is modeled as a target equation in equation form; The multiple task equations determined based on the constraint equations and the target equations are hierarchically arranged according to priority, wherein the priority of the constraint equations is higher than that of the target equations. The task equations at each level are solved in descending order of priority. Within the solution space determined by the high-priority task equations, the low-priority task equations are optimized to obtain the first joint angle information used to indicate the rotation angle of the robot joints. The first joint angle information is low-pass filtered to obtain joint angle information, and the control command includes the joint angle information.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 5.

8. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method of any one of claims 1 to 5.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 5.

Images