Humanoid robot teleoperation method and device, computer equipment and storage medium

Abstract

The invention relates to a humanoid robot teleoperation method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring original data of an original action of a simulated target; acquiring joint position parameters according to the original data; according to the joint position parameters, a movement track is obtained, smooth optimization is conducted on the movement track, and an optimized track is obtained; and according to the optimized track, a motor real-time control instruction is generated to drive joints of the humanoid robot. The method has high accuracy and real-time performance.

Description

Technical Field

[0001] The present invention relates to the field of robots, and in particular to a method, device, computer equipment and storage medium for remote operation of a humanoid robot. Background Art

[0002] With the development of computer technology and artificial intelligence, humanoid robots have become an important research direction in the field of humanoid robots. The remote operation of humanoid robots has important strategic significance in data collection and model optimization. It can provide strong support for the development of embodied intelligence, general artificial intelligence (AGI) and the progress of the humanoid robot industry.

[0003] Humanoid robot teleoperation refers to the real-time collection of human joint values, the generation of joint space instructions suitable for the humanoid robot model, and then the execution of real-time motion control of the humanoid robot through trajectory planning, that is, the realization of the humanoid robot's "imitation" of human movements.

[0004] There are many challenges in implementing humanoid robot teleoperation. First, teleoperation requires real-time response, especially in high-precision scenarios. Whether the action is real-time will directly affect its success rate. Second, the accuracy of the action is also critical. Humanoid robot teleoperation involves a large number of actions that interact with objects, and these actions require good accuracy to be achieved. At the same time, the actions of humanoid robots need to be highly similar to those of real humans.

[0005] However, the methods in the prior art are difficult to meet the requirements of real-time performance, accuracy and high similarity at the same time. Summary of the invention

[0006] In order to solve the above technical problems or at least partially solve the above technical problems, the present invention provides a method, apparatus, computer equipment and storage medium for remote operation of a humanoid robot.

[0007] In a first aspect, the present invention provides a method for remote operation of a humanoid robot, the method comprising:

[0008] Obtaining raw data of the original action of the imitation target;

[0009] Acquire joint position parameters according to the original data;

[0010] According to the joint position parameters, a motion trajectory is obtained, and the motion trajectory is smoothly optimized to obtain an optimized trajectory;

[0011] According to the optimized trajectory, real-time motor control instructions are generated to drive the joints of the humanoid robot.

[0012] Optionally, the humanoid robot includes a humanoid robot head, the joint position parameters of the humanoid robot head include a yaw angle of the humanoid robot head, and acquiring the joint position parameters according to the raw data includes:

[0013] The yaw angle of the humanoid robot head is obtained in the following manner:

[0014] J robot_head_yaw =w 1 ·J robot_head_yaw_head +w 2 ·J robot_head_yaw_neck

[0015]

[0016] Among them, w 1 and w 2 is the adjustment proportionality factor, w 1 +w 2 =1, J robot_head_yaw is the total yaw angle of the humanoid robot head, J robot_head_yaw_head is the head component of the yaw angle of the humanoid robot head, J robot_head_yaw_neck is the neck component of the yaw angle of the humanoid robot head, J human_neck_yaw To simulate the actual angle of the target neck yaw angle, J human_neck_yaw_min To simulate the minimum yaw angle of the target neck, J human_neck_yaw_max is the maximum yaw angle of the target’s neck, J robot_head_yaw_max is the maximum yaw angle of the humanoid robot head, J robot_head_yaw_min is the minimum yaw angle of the humanoid robot head, J human_neck_yaw To simulate the actual angle of the target neck yaw angle, J human_neck_yaw_min To simulate the minimum yaw angle of the target neck, J human_neck_yaw_max To simulate the maximum yaw angle of the target neck, J robot_head_yaw_max is the maximum yaw angle of the humanoid robot head, J robot_head_yaw_min The minimum yaw angle of the humanoid robot's head.

[0017] Optionally, the joint position parameters of the head further include a pitch angle of the humanoid robot head, and obtaining the joint position parameters according to the raw data includes:

[0018] The pitch angle of the humanoid robot head is obtained in the following manner:

[0019] J robot_head_pitch =w 1 ·J robot_head_pitch_head +w 2 ·J robot_head_pitch_neck

[0020]

[0021] Among them, w 1 and w 2 is the adjustment proportionality factor, w 1 +w 2 =1, J robot_head_pitch is the total pitch angle of the humanoid robot head, J robot_head_pitch_head is the head component of the pitch angle of the humanoid robot head, J robot_head_pitch_neck is the neck component of the pitch angle of the humanoid robot head, J human_head_pitch To simulate the actual pitch angle of the target head, J human_head_pitch_min To simulate the minimum pitch angle of the target head, J robot_head_pitch_max is the maximum pitch angle of the humanoid robot neck, J robot_head_pitch_min is the minimum pitch angle of the humanoid robot neck, J human_neck_pitch To simulate the actual pitch angle of the target neck, J human_neck_pitch_min To simulate the minimum pitch angle of the target neck, J human_neck_pitch_max To simulate the maximum pitch angle of the target neck, J robot_head_pitch_max is the maximum pitch angle of the humanoid robot head, J robot_head_pitch_min The minimum pitch angle of the humanoid robot's head.

[0022] Optionally, the humanoid robot further comprises humanoid robot arms, the humanoid robot arms comprise humanoid robot shoulders, humanoid robot elbows and humanoid robot wrists, and the humanoid robot arm joints comprise humanoid robot shoulder joints, humanoid robot elbow joints and humanoid robot wrist joints.

[0023] The step of obtaining joint position parameters according to the original data comprises:

[0024] Obtaining a dependency relationship of both arms of the humanoid robot, wherein the dependency relationship of both arms includes: a dependency relationship of the shoulder of the humanoid robot on the shoulder joint of the humanoid robot, a dependency relationship of the elbow of the humanoid robot on the elbow joint of the humanoid robot, and a dependency relationship of the wrist of the humanoid robot on the elbow joint and the wrist joint of the humanoid robot;

[0025] According to the double-arm dependency, the correspondence between the double-arm joints of the humanoid robot and the double-arm joints of the imitation target is obtained.

[0026] Optionally, the corresponding relationship between the double-arm joints of the humanoid robot and the double-arm joints of the imitation target is obtained according to the double-arm dependency in the following manner:

[0027]

[0028] Among them, J robot is the value of the j-th humanoid robot joint, i is the total number of dependent joints of the j-th humanoid robot joint, J human_dependency_j_i is the value of the i-th dependent joint of the j-th humanoid robot joint, W j_i is the weight value of the i-th dependent joint of the j-th humanoid robot joint, offset j_i is the offset value of the i-th dependent joint of the j-th humanoid robot joint.

[0029] Optionally, the humanoid robot comprises a humanoid robot hand, and the humanoid robot hand comprises a humanoid robot thumb;

[0030] Acquiring joint position parameters according to the raw data includes:

[0031] Get the quaternion of the thumb joint of the humanoid robot;

[0032] normalizing the quaternion;

[0033] Extract axis angle based on normalized quaternion;

[0034] The yaw angle of the thumb joint of the humanoid robot is obtained according to the axis angle.

[0035] Optionally, the quaternion of the thumb joint of the humanoid robot is obtained in the following manner:

[0036] q′=Quaternion(ω′,x′,y′,z′)

[0037] The quaternion is normalized as follows:

[0038]

[0039] The axis angle is extracted according to the normalized quaternion in the following manner:

[0040]

[0041] θ=2arccos(w)

[0042]

[0043] The yaw angle of the thumb joint of the humanoid robot is obtained according to the axis angle in the following manner:

[0044]

[0045] Wherein, q′ is the quaternion of the thumb joint of the humanoid robot, q is the normalized quaternion, is the axis angle, θ and a are intermediate parameters, J robot_yaw Yaw angle of the thumb joint of the humanoid robot.

[0046] Optionally, if the original action is a fine action, after acquiring the joint position parameters according to the original data and before acquiring the motion trajectory according to the joint position parameters and smoothly optimizing the motion trajectory to obtain the optimized trajectory, the method further includes:

[0047] The joint position parameters are optimized according to the reference position at the current sampling moment, the target position at the current sampling moment, and the constraint conditions.

[0048] Optionally, the joint position parameters are optimized according to the reference position at the current sampling moment, the target position at the current sampling moment, and the constraint conditions in the following manner:

[0049]

[0050] Among them, J i+1 is the joint position parameter at the next sampling moment, The reference position at the current sampling moment, The target position at the current sampling time, ∈ is the distance threshold, The representation is solved by the representation inverse kinematics solver.

[0051] Optionally, acquiring a motion trajectory according to the joint position parameters, and performing smooth optimization on the motion trajectory to obtain an optimized trajectory includes:

[0052] The optimization trajectory is obtained as follows:

[0053]

[0054] Among them, P smooth (t) is the path position of the interpolated and smoothed path point at sampling time t, S() represents the smoothing function, ω i ′ is the interpolation point weight, P(t k ) is the path point sequence before optimization, f(t,t k ) is the interpolation function.

[0055] In a second aspect, a humanoid robot teleoperation device is provided, the device comprising:

[0056] Motion capture device, used to obtain raw data of the original motion of the imitation target;

[0057] A motion redirector, used for obtaining joint position parameters according to the raw data;

[0058] A trajectory planner, used to obtain a motion trajectory according to the joint position parameters, and to smoothly optimize the motion trajectory to obtain an optimized trajectory;

[0059] The actuator is used to generate real-time motor control instructions according to the optimized trajectory to drive the joints of the humanoid robot.

[0060] According to a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the computer program.

[0061] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method as described in any one of the above items is implemented.

[0062] The present invention provides a method, device, computer equipment and storage medium for remote operation of a humanoid robot, the method comprising: obtaining raw data of the original action of the imitation target; obtaining joint position parameters according to the raw data; obtaining a motion trajectory according to the joint position parameters, and smoothly optimizing the motion trajectory to obtain an optimized trajectory; generating a motor real-time control instruction according to the optimized trajectory to drive the joints of the humanoid robot. The method of the embodiment of the present invention obtains joint position parameters according to the raw data, and then obtains the motion trajectory, and after optimizing the motion trajectory, generates a motor real-time control instruction to directly drive the joints of the humanoid robot to realize remote operation of the humanoid robot. The method of the embodiment of the present invention is not a direct imitation of the whole body action of the imitation object, but an operation performed by obtaining the joint position parameters, and the data collection amount and processing amount are small, so that the control of the humanoid robot has a high real-time performance. At the same time, since the imitation object, that is, the action of the human body, is also realized through the human body joints, the method of the embodiment of the present invention also has a high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0065] Figure 1 The figure shows an application environment diagram of the humanoid robot teleoperation method according to an embodiment of the present invention;

[0066] Figure 2 FIG. 1 is a flow chart of a method for remotely operating a humanoid robot according to an embodiment of the present invention;

[0067] Figure 3 Shown is a schematic diagram of a humanoid robot according to an embodiment of the present invention;

[0068] Figure 4 FIG. 1 is a structural block diagram of a humanoid robot teleoperation device according to an embodiment of the present invention;

[0069] Figure 5 FIG. 2 is a diagram showing the internal structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] Figure 1 FIG. 1 is an application environment diagram of a humanoid robot teleoperation method in an embodiment. Figure 1 , the humanoid robot teleoperation method is applied to a humanoid robot teleoperation system. The humanoid robot teleoperation method includes a terminal 110 and / or a server 120. The terminal 110 and the server 120 are connected via a network. The terminal 110 may be a desktop terminal or a mobile terminal, and the mobile terminal may be at least one of a mobile phone, a tablet computer, a laptop computer, etc. The server 120 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0072] The humanoid robot teleoperation method of the present invention is applied to the terminal 110 and / or the server 120 .

[0073] Figure 2 FIG. 1 is a flow chart of a method for remotely operating a humanoid robot according to an embodiment of the present invention. Figure 2 As shown, the method includes:

[0074] Step 210, obtaining original data of the original action of the imitation target;

[0075] Step 220, obtaining joint position parameters according to the original data;

[0076] Step 230, obtaining a motion trajectory according to the joint position parameters, and performing smooth optimization on the motion trajectory to obtain an optimized trajectory;

[0077] Step 240: Generate real-time motor control instructions based on the optimized trajectory to drive the joints of the humanoid robot.

[0078] The method of the embodiment of the present invention obtains joint position parameters based on raw data, and then obtains the motion trajectory. After optimizing the motion trajectory, it generates real-time motor control instructions to directly drive the joints of the humanoid robot to achieve remote control of the humanoid robot. The method of the embodiment of the present invention is not a direct imitation of the whole body movements of the imitation object, but an operation performed by obtaining the joint position parameters. The amount of data collected and the amount of processing are relatively small, so that the control of the humanoid robot has high real-time performance. At the same time, since the imitation object, that is, the movement of the human body, is also achieved through the joints of the human body, the method of the embodiment of the present invention also has high accuracy.

[0079] In the embodiment of the present invention, the humanoid robot includes multiple segments and multiple joints, and the raw data of the raw action obtained includes:

[0080] Obtaining the position and rotation orientation of each of the segments;

[0081] Obtaining the linear velocity, angular velocity, linear acceleration and angular acceleration of each of the segments;

[0082] Get the position component of each of the joints.

[0083] The segmentation in the embodiment of the present invention is divided according to the imitation object, that is, the human body, and its main purpose is to facilitate the classification control of the humanoid robot.

[0084] Figure 3 FIG. 1 is a schematic diagram of a humanoid robot according to an embodiment of the present invention. Figure 3 As shown, the segments of the humanoid robot may include the head, neck, arms, feet, hands, waist, etc. of the humanoid robot. The acquisition of joint position parameters of different segments is not exactly the same. The following will take some segments as an example for explanation.

[0085] In the embodiment of the present invention, the original data can be obtained by various data acquisition devices in the prior art, which will not be described in detail here.

[0086] In the embodiment of the present invention, the humanoid robot includes a humanoid robot head, and the joint position parameters of the humanoid robot head include a yaw angle of the humanoid robot head. Step 220, obtaining the joint position parameters according to the raw data, includes:

[0087] The yaw angle of the humanoid robot head is obtained in the following manner:

[0088] J robot_head_yaw =w 1 ·J robot_head_yaw_head +w 2 ·J robot_head_yaw_neck

[0089]

[0090] Among them, w 1 and w 2 is the adjustment proportionality factor, w 1 +w 2 =1, J robot_head_yaw is the total yaw angle of the humanoid robot head, J robot_head_yaw_head is the head component of the yaw angle of the humanoid robot head, J robot_head_yaw_neck is the neck component of the yaw angle of the humanoid robot head, J human_neck_yaw To simulate the actual angle of the target neck yaw angle, J human_neck_yaw_min To simulate the minimum yaw angle of the target neck, J human_neck_yaw_max is the maximum yaw angle of the target’s neck, J robot_head_yaw_max is the maximum yaw angle of the humanoid robot head, J robot_head_yaw_min is the minimum yaw angle of the humanoid robot head, J human_neck_yaw To simulate the actual angle of the target neck yaw angle, J human_neck_yaw_min To simulate the minimum yaw angle of the target neck, J human_neck_yaw_max To simulate the maximum yaw angle of the target neck, J robot_head_yaw_max is the maximum yaw angle of the humanoid robot head, J robot_head_yaw_min The minimum yaw angle of the humanoid robot's head.

[0091] In the embodiment of the present invention, the joint position parameters of the head also include the pitch angle of the humanoid robot head. Step 220, obtaining the joint position parameters according to the raw data, includes:

[0092] The pitch angle of the humanoid robot head is obtained in the following manner:

[0093] J robot_head_pitch =w 1 ·J robot_head_pitch_head +w 2 ·J robot_head_pitch_neck

[0094]

[0095] Among them, w 1 and w 2 is the adjustment proportionality factor, w 1 +w2 =1, J robot_head_pitch is the total pitch angle of the humanoid robot head, J robot_head_pitch_head is the head component of the pitch angle of the humanoid robot head, J robot_head_pitch_neck is the neck component of the pitch angle of the humanoid robot head, J human_head_pitch To simulate the actual pitch angle of the target head, J human_head_pitch_min To simulate the minimum pitch angle of the target head, J robot_head_pitch_max is the maximum pitch angle of the humanoid robot neck, J robot_head_pitch_min is the minimum pitch angle of the humanoid robot neck, J human_neck_pitch To simulate the actual pitch angle of the target neck, J human_neck_pitch_min To simulate the minimum pitch angle of the target neck, J human_neck_pitch_max is the maximum pitch angle of the target neck, J robot_head_pitch_max is the maximum pitch angle of the humanoid robot head, J robot_head_pitch_min The minimum pitch angle of the humanoid robot's head.

[0096] In the embodiment of the present invention, the waist rotation angle and yaw angle of the humanoid robot can be generated by compounding the rotation angles of the hip, spine, and sternum in the same manner as the head, and will not be described in detail here.

[0097] In the embodiment of the present invention, the yaw angle and the pitch angle can also be considered as two degrees of freedom, so the head has two degrees of freedom and the waist also has two degrees of freedom.

[0098] In an embodiment of the present invention, the humanoid robot further comprises humanoid robot arms, the humanoid robot arms comprise humanoid robot shoulders, humanoid robot elbows and humanoid robot wrists, and the humanoid robot arm joints comprise humanoid robot shoulder joints, humanoid robot elbow joints and humanoid robot wrist joints.

[0099] Step 220, obtaining joint position parameters according to the original data, includes:

[0100] Obtaining a dependency relationship of both arms of the humanoid robot, wherein the dependency relationship of both arms includes: a dependency relationship of the shoulder of the humanoid robot on the shoulder joint of the humanoid robot, a dependency relationship of the elbow of the humanoid robot on the elbow joint of the humanoid robot, and a dependency relationship of the wrist of the humanoid robot on the elbow joint and the wrist joint of the humanoid robot;

[0101] According to the double-arm dependency, the correspondence between the double-arm joints of the humanoid robot and the double-arm joints of the imitation target is obtained.

[0102] In the embodiment of the present invention, the corresponding relationship between the two-arm joints of the humanoid robot and the two-arm joints of the imitation target is obtained according to the two-arm dependency relationship in the following manner:

[0103]

[0104] Among them, J roboy is the value of the j-th humanoid robot joint, i is the total number of dependent joints of the j-th humanoid robot joint, J human_dependency_j_i is the value of the i-th dependent joint of the j-th humanoid robot joint, W j_i is the weight value of the i-th dependent joint of the j-th humanoid robot joint, offset j_i is the offset value of the i-th dependent joint of the j-th humanoid robot joint.

[0105] In the embodiment of the present invention, the limbs of the humanoid robot can be considered as typical linkage structure movements, generally including 7 degrees of freedom for each arm, namely, the humanoid robot shoulder rotation angle, the humanoid robot shoulder yaw angle, the humanoid robot shoulder pitch angle, the humanoid robot elbow, the humanoid robot wrist rotation angle, the humanoid robot wrist yaw angle, and the humanoid robot wrist pitch angle; the two feet each have 6 degrees of freedom, namely, the humanoid robot thigh hip joint rotation angle, the humanoid robot thigh hip joint yaw angle, the humanoid robot thigh hip joint pitch angle, the humanoid robot knee rotation angle, and the humanoid robot ankle pitch angle.

[0106] The imitation object, that is, the human limbs, is different from the robot structure. The joints of the human body are mainly ball and socket joints, such as the human shoulder, human hip, and human ankle; the human joints also have rotational pivot / hinge joints, such as the human elbow, and the human joints also have pivot / hinge joints, such as the human knee. The basic motion models of different joints are different. The ball and socket joint has 3 degrees of freedom, the rotational pivot / hinge joint has 1 degree of freedom, and the pivot / hinge joint has 3 degrees of freedom.

[0107] The legs of the embodiment of the present invention can obtain joint position parameters in the same manner as the arms, which will not be described in detail here.

[0108] In an embodiment of the present invention, the humanoid robot comprises a humanoid robot hand, and the humanoid robot hand comprises a humanoid robot thumb;

[0109] Step 220, obtaining joint position parameters according to the original data, includes:

[0110] Get the quaternion of the thumb joint of the humanoid robot;

[0111] normalizing the quaternion;

[0112] Extract axis angle based on normalized quaternion;

[0113] The yaw angle of the thumb joint of the humanoid robot is obtained according to the axis angle.

[0114] In the embodiment of the present invention, the quaternion q′ of the thumb joint of the humanoid robot is obtained in the following manner:

[0115] q′=Quaternion(ω′,x′,y′,z′)

[0116] The quaternion is normalized as follows:

[0117]

[0118] The axis angle is extracted according to the normalized quaternion in the following manner:

[0119]

[0120] θ=2arccos(w)

[0121]

[0122] The yaw angle of the thumb joint of the humanoid robot is obtained according to the axis angle in the following manner:

[0123]

[0124] Wherein, q′ is the quaternion of the thumb joint of the humanoid robot, q is the normalized quaternion, is the axis angle, θ and a are intermediate parameters, J robot_yaw Yaw angle of the thumb joint of the humanoid robot.

[0125] In the embodiment of the present invention, only the thumb is used as an example for description. The joint position parameters of other finger joints can refer to the above method, which will not be described in detail here.

[0126] In the embodiment of the present invention, after obtaining the yaw angle of the thumb joint of the humanoid robot, the method further includes: applying restrictions and obtaining a control percentage:

[0127]

[0128] Among them, percent is the control percentage, applyLimit represents the application limit, J min The minimum limiting angle of the thumb joint of the humanoid robot, J max The maximum limit angle of the thumb joint of the humanoid robot.

[0129] The above maximum and minimum limiting angles are because the bending angle of the thumb of the human body to be simulated is limited, approximately between 0 and 180. Obtaining the control percentage is convenient for the subsequent control of the motor.

[0130] In the embodiment of the present invention, if the original action is a fine action, after acquiring the joint position parameters according to the original data and before acquiring the motion trajectory according to the joint position parameters and smoothly optimizing the motion trajectory to obtain the optimized trajectory, the method further includes:

[0131] The joint position parameters are optimized according to the reference position at the current sampling moment, the target position at the current sampling moment, and the constraint conditions.

[0132] In the embodiment of the present invention, the joint position parameters are optimized according to the reference position at the current sampling moment, the target position at the current sampling moment, and the constraint conditions in the following manner:

[0133]

[0134] Among them, J i+1 is the joint position parameter at the next sampling moment, The reference position at the current sampling moment, The target position at the current sampling time, ∈ is the distance threshold, The representation is solved by the representation inverse kinematics solver.

[0135] The above ∈ is used to select the inverse solution that satisfies a specific distance constraint.

[0136] In the embodiment of the present invention, if the original action is a fine action, that is, more attention is paid to the accuracy of the original action and more precise coordination of both hands is required, then the joint position parameters need to be optimized.

[0137] In an embodiment of the present invention, the inverse kinematics solver can perform inverse kinematics solution on the joint space of the robot's two arms through the end position output by the motion redirector, and at the same time perform selection with reference to the expected position of the joint space output by the motion redirector, and finally output real-time joint position parameters that prioritize position and take anthropomorphism into consideration.

[0138] In the embodiment of the present invention, if more attention is paid to the efficiency and human-like degree of control, and the operation is less sophisticated, such as carrying work, the trajectory planning can be performed directly according to the joint position parameters.

[0139] In the embodiment of the present invention, for fine movements, the output accuracy is higher after optimization; non-fine movements do not need to be optimized and are directly output, with high frequency and better real-time performance.

[0140] In the embodiment of the present invention, the motion trajectory is obtained and smoothly optimized for the continuity of the humanoid robot's movements in space. The discontinuous motion in the path is eliminated by smoothing to avoid mechanical impact of the humanoid robot.

[0141] Through optimization such as interpolation and smoothing, action subdivision can be achieved, so that the control system of the humanoid robot can accurately execute in each small step. This can achieve precise, efficient and stable real-time control, make the robot move more naturally, reduce errors and energy consumption, and improve the response speed and reliability of the system.

[0142] In the embodiment of the present invention, in step 230, the motion trajectory is obtained according to the joint position parameters, and the motion trajectory is smoothly optimized to obtain the optimized trajectory, including:

[0143] The optimization trajectory is obtained as follows:

[0144]

[0145] Among them, P smooth (t) is the path position of the interpolated and smoothed path point at sampling time t, S() represents the smoothing function, ω i ′ is the interpolation point weight, P(t k ) is the path point sequence before optimization, f(t,t k ) is the interpolation function.

[0146] The present invention has higher accuracy and real-time performance.

[0147] like Figure 4 As shown, the present invention also provides a humanoid robot teleoperation device, the device comprising:

[0148] Motion capture device 410, used to obtain original data of the original motion of the imitation target;

[0149] A motion redirector 420, for obtaining joint position parameters according to the raw data;

[0150] A trajectory planner 430 is used to obtain a motion trajectory according to the joint position parameters, and to perform smooth optimization on the motion trajectory to obtain an optimized trajectory;

[0151] The actuator 440 is used to generate real-time motor control instructions according to the optimized trajectory to drive the joints of the humanoid robot.

[0152] In the embodiment of the present invention, the humanoid robot includes a humanoid robot head, and the joint position parameters of the humanoid robot head include a yaw angle of the humanoid robot head.

[0153] The motion redirector 420 is also used to:

[0154] The yaw angle of the humanoid robot head is obtained in the following manner:

[0155] J robot_head_yaw =w 1 ·J robot_head_yaw_head +w 2 ·J robot_head_yaw_neck

[0156]

[0157] Among them, w 1 and w 2 is the adjustment proportionality factor, w 1 +w 2 =1, J robot_head_yaw is the total yaw angle of the humanoid robot head, J robot_head_yaw_head is the head component of the yaw angle of the humanoid robot head, J robot_head_yaw_neck is the neck component of the yaw angle of the humanoid robot head, J human_neck_yaw To simulate the actual angle of the target neck yaw angle, J human_neck_yaw_min To simulate the minimum yaw angle of the target neck, J human_neck_yaw_max is the maximum yaw angle of the target’s neck, J robot_head_yaw_max is the maximum yaw angle of the humanoid robot head, J robot_head_yaw_min is the minimum yaw angle of the humanoid robot head, J human_neck_yaw To simulate the actual angle of the target neck yaw angle, J human_neck_yaw_min To simulate the minimum yaw angle of the target neck, J human_neck_yaw_max To simulate the maximum yaw angle of the target neck, J robot_head_yaw_max is the maximum yaw angle of the humanoid robot head, J robot_head_yaw_min The minimum yaw angle of the humanoid robot's head.

[0158] In the embodiment of the present invention, the joint position parameters of the head further include the pitch angle of the humanoid robot head, and the motion redirector 420 is further used for:

[0159] The pitch angle of the humanoid robot head is obtained in the following manner:

[0160] J robot_head_pitch =w 1 ·J robot_head_pitch_head +w 2 ·J robot_head_pitch_neck

[0161]

[0162] Among them, w 1 and w2 is the adjustment proportionality factor, w 1 +w 2 =1, J robot_head_pitch is the total pitch angle of the humanoid robot head, J robot_head_pitch_head is the head component of the pitch angle of the humanoid robot head, J robot_head_pitch_neck is the neck component of the pitch angle of the humanoid robot head, J human_head_pitch To simulate the actual pitch angle of the target head, J human_head_pitch_min To simulate the minimum pitch angle of the target head, J robot_head_pitch_max is the maximum pitch angle of the humanoid robot neck, J robot_head_pitch_min is the minimum pitch angle of the humanoid robot neck, J human_neck_pitch To simulate the actual pitch angle of the target neck, J human_neck_pitch_min To simulate the minimum pitch angle of the target neck, J human_neck_pitch_max To simulate the maximum pitch angle of the target neck, J robot_head_pitch_max is the maximum pitch angle of the humanoid robot head, J robot_head_pitch_min The minimum pitch angle of the humanoid robot's head.

[0163] In an embodiment of the present invention, the humanoid robot further comprises humanoid robot arms, the humanoid robot arms comprise humanoid robot shoulders, humanoid robot elbows and humanoid robot wrists, and the humanoid robot arm joints comprise humanoid robot shoulder joints, humanoid robot elbow joints and humanoid robot wrist joints.

[0164] The motion redirector 420 is also used to:

[0165] Obtaining a dependency relationship of both arms of the humanoid robot, wherein the dependency relationship of both arms includes: a dependency relationship of the shoulder of the humanoid robot on the shoulder joint of the humanoid robot, a dependency relationship of the elbow of the humanoid robot on the elbow joint of the humanoid robot, and a dependency relationship of the wrist of the humanoid robot on the elbow joint and the wrist joint of the humanoid robot;

[0166] According to the double-arm dependency, the correspondence between the double-arm joints of the humanoid robot and the double-arm joints of the imitation target is obtained.

[0167] In the embodiment of the present invention, the motion redirector 420 is further used for:

[0168] According to the dual-arm dependency, the corresponding relationship between the dual-arm joints of the humanoid robot and the dual-arm joints of the imitation target is obtained in the following manner:

[0169]

[0170] Among them, Jrobot is the value of the j-th humanoid robot joint, i is the total number of dependent joints of the j-th humanoid robot joint, J human_dependency_j_i is the value of the i-th dependent joint of the j-th humanoid robot joint, W j_i is the weight value of the i-th dependent joint of the j-th humanoid robot joint, offset j_i is the offset value of the i-th dependent joint of the j-th humanoid robot joint.

[0171] In an embodiment of the present invention, the humanoid robot comprises a humanoid robot hand, and the humanoid robot hand comprises a humanoid robot thumb;

[0172] The motion redirector 420 is also used to:

[0173] Get the quaternion of the thumb joint of the humanoid robot;

[0174] normalizing the quaternion;

[0175] Extract axis angle based on normalized quaternion;

[0176] The yaw angle of the thumb joint of the humanoid robot is obtained according to the axis angle.

[0177] The motion redirector 420 is also used to obtain the quaternion of the thumb joint of the humanoid robot in the following manner:

[0178] q′=Quaternion(ω′,x′,y′,z′)

[0179] The motion redirector 420 is further configured to normalize the quaternion in the following manner:

[0180]

[0181] The motion redirector 420 is further configured to extract the axis angle from the normalized quaternion in the following manner:

[0182]

[0183] The motion redirector 420 is further configured to obtain the yaw angle of the thumb joint of the humanoid robot according to the axis angle in the following manner:

[0184]

[0185] Wherein, q′ is the quaternion of the thumb joint of the humanoid robot, q is the normalized quaternion, is the axis angle, θ and a are intermediate parameters, J robot_yaw Yaw angle of the thumb joint of the humanoid robot.

[0186] The device of the embodiment of the present invention further includes:

[0187] The optimization inverse solver is used for: if the original action is a fine action, after obtaining the joint position parameters according to the original data, and before obtaining the motion trajectory according to the joint position parameters and smoothly optimizing the motion trajectory to obtain the optimized trajectory:

[0188] The joint position parameters are optimized according to the reference position at the current sampling moment, the target position at the current sampling moment, and the constraint conditions.

[0189] In summary, in the embodiment of the present invention, the optimized inverse solver is further used for:

[0190] The joint position parameters are optimized according to the reference position at the current sampling moment, the target position at the current sampling moment, and the constraints in the following manner:

[0191]

[0192] Among them, J i+1 is the joint position parameter at the next sampling moment, The reference position at the current sampling moment, The target position at the current sampling time, ∈ is the distance threshold, The representation is solved by the representation inverse kinematics solver.

[0193] In the embodiment of the present invention, the trajectory planner 430 is also used to obtain the optimized trajectory in the following manner:

[0194]

[0195] Among them, P smooth (t) is the path position of the interpolated and smoothed path point at sampling time t, S() represents the smoothing function, ω i ′ is the interpolation point weight, P(t k ) is the path point sequence before optimization, f(t,t k ) is the interpolation function.

[0196] The present invention has higher accuracy and real-time performance.

[0197] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following method when executing the computer program: obtaining original data of the original action of the imitation target; obtaining joint position parameters according to the original data; obtaining a motion trajectory according to the joint position parameters, and smoothly optimizing the motion trajectory to obtain an optimized trajectory; generating real-time motor control instructions according to the optimized trajectory to drive the joints of the humanoid robot.

[0198] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following method is implemented: obtaining original data of the original action of the imitation target; obtaining joint position parameters based on the original data; obtaining a motion trajectory based on the joint position parameters, and smoothly optimizing the motion trajectory to obtain an optimized trajectory; generating real-time motor control instructions based on the optimized trajectory to drive the joints of the humanoid robot.

[0199] The above-mentioned humanoid robot remote operation method achieves the beneficial effect of being able to solve the technical problems raised in the background technology.

[0200] Figure 2 FIG. 1 is a flow chart of a method for remotely operating a humanoid robot in one embodiment. Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0201] Figure 5 The internal structure diagram of a computer device in one embodiment is shown. The computer device may specifically be Figure 1 The server 120 in FIG. Figure 5As shown, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the humanoid robot remote operation method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the humanoid robot remote operation method. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0202] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0203] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0204] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0205] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for remote operation of a humanoid robot, characterized in that: The method comprises: Obtaining raw data of the original action of the imitation target; Acquire joint position parameters according to the original data; According to the joint position parameters, a motion trajectory is obtained, and the motion trajectory is smoothly optimized to obtain an optimized trajectory; According to the optimized trajectory, real-time motor control instructions are generated to drive the joints of the humanoid robot.

2. The method according to claim 1, characterized in that The humanoid robot includes a humanoid robot head, the joint position parameters of the humanoid robot head include a yaw angle of the humanoid robot head, and the step of acquiring the joint position parameters according to the raw data includes: The yaw angle of the humanoid robot head is obtained in the following manner: J robot_head_yaw =w1·J robot_head_yaw_head +w2·J robot_head_yaw_neck Among them, w1 and w2 are adjustment ratio coefficients, w1+w2=1, J robot_head_yaw is the total yaw angle of the humanoid robot head, J robot_head_yaw_head is the head component of the yaw angle of the humanoid robot head, J robot_head_yaw_neck is the neck component of the yaw angle of the humanoid robot head, J human_neck_yaw To simulate the actual angle of the target neck yaw angle, J human_neck_yaw_min To simulate the minimum yaw angle of the target neck, J human_neck_yaw_max is the maximum yaw angle of the target’s neck, J robot_head_yaw_max is the maximum yaw angle of the humanoid robot head, J robot_head_yaw_min is the minimum yaw angle of the humanoid robot head, J human_neck_yaw To simulate the actual angle of the target neck yaw angle, J human_neck_yaw_min To simulate the minimum yaw angle of the target neck, J human_neck_yaw_max To simulate the maximum yaw angle of the target neck, J robot_head_yaw_max is the maximum yaw angle of the humanoid robot head, J robot_head_yaw_min The minimum yaw angle of the humanoid robot's head.

3. The method according to claim 2, characterized in that The joint position parameters of the head also include the pitch angle of the humanoid robot head, and the step of obtaining the joint position parameters according to the raw data includes: The pitch angle of the humanoid robot head is obtained in the following manner: J robot_head_pitch =w1·J robot_head_pitch_head +w2·J robot_head_pitch_neck Among them, w1 and w2 are adjustment ratio coefficients, w1+w2=1, J robot_head_pitch is the total pitch angle of the humanoid robot head, J robot_head_pitch_head is the head component of the pitch angle of the humanoid robot head, J robot_head_pitch_neck is the neck component of the pitch angle of the humanoid robot head, J human_head_pitch To simulate the actual pitch angle of the target head, J human_head_pitch_min To simulate the minimum pitch angle of the target head, J robot_head_pitch_max is the maximum pitch angle of the humanoid robot neck, J robot_head_pitch_min is the minimum pitch angle of the humanoid robot neck, J human_neck_pitch To simulate the actual pitch angle of the target neck, J human_neck_pitch_min To simulate the minimum pitch angle of the target neck, J human_neck_pitch_max To simulate the maximum pitch angle of the target neck, J robot_head_pitch_max is the maximum pitch angle of the humanoid robot head, J robot_head_pitch_min The minimum pitch angle of the humanoid robot's head.

4. The method according to claim 1, characterized in that: The humanoid robot further comprises humanoid robot arms, wherein the humanoid robot arms comprise humanoid robot shoulders, humanoid robot elbows and humanoid robot wrists, and the humanoid robot arm joints comprise humanoid robot shoulder joints, humanoid robot elbow joints and humanoid robot wrist joints. The step of obtaining joint position parameters according to the original data comprises: Obtaining a dependency relationship of both arms of the humanoid robot, wherein the dependency relationship of both arms includes: a dependency relationship of the shoulder of the humanoid robot on the shoulder joint of the humanoid robot, a dependency relationship of the elbow of the humanoid robot on the elbow joint of the humanoid robot, and a dependency relationship of the wrist of the humanoid robot on the elbow joint and the wrist joint of the humanoid robot; According to the double-arm dependency, the correspondence between the double-arm joints of the humanoid robot and the double-arm joints of the imitation target is obtained.

5. The method according to claim 4, characterized in that The corresponding relationship between the double-arm joints of the humanoid robot and the double-arm joints of the imitation target is obtained according to the double-arm dependency relationship in the following manner: Among them, J robot is the value of the j-th humanoid robot joint, i is the total number of dependent joints of the j-th humanoid robot joint, J human_dependency_j_i is the value of the i-th dependent joint of the j-th humanoid robot joint, W j_i is the weight value of the i-th dependent joint of the j-th humanoid robot joint, offset j_i is the offset value of the i-th dependent joint of the j-th humanoid robot joint.

6. The method according to claim 1, characterized in that The humanoid robot comprises a humanoid robot hand, and the humanoid robot hand comprises a humanoid robot thumb; Acquiring joint position parameters according to the raw data includes: Get the quaternion of the thumb joint of the humanoid robot; normalizing the quaternion; Extract axis angle based on normalized quaternion; The yaw angle of the thumb joint of the humanoid robot is obtained according to the axis angle.

7. The method according to claim 6, characterized in that The quaternion of the thumb joint of the humanoid robot is obtained in the following manner: q′=Quaternion(ω′,x′,y′,z′) The quaternion is normalized as follows: The axis angle is extracted according to the normalized quaternion in the following manner: The yaw angle of the thumb joint of the humanoid robot is obtained according to the axis angle in the following manner: Wherein, q′ is the quaternion of the thumb joint of the humanoid robot, q is the normalized quaternion, is the axis angle, θ and a are intermediate parameters, J robot_yaw Yaw angle of the thumb joint of the humanoid robot.

8. The method according to claim 1, characterized in that If the original action is a fine action, after acquiring the joint position parameters according to the original data and before acquiring the motion trajectory according to the joint position parameters and performing smooth optimization on the motion trajectory to obtain the optimized trajectory, the method further includes: The joint position parameters are optimized according to the reference position at the current sampling moment, the target position at the current sampling moment, and the constraint conditions.

9. The method according to claim 8, characterized in that The joint position parameters are optimized according to the reference position at the current sampling moment, the target position at the current sampling moment, and the constraints, in the following manner: Among them, J i+1 is the joint position parameter at the next sampling moment, The reference position at the current sampling moment, The target position at the current sampling time, ∈ is the distance threshold, The representation is solved by the representation inverse kinematics solver.

10. The method according to claim 1, characterized in that The step of obtaining a motion trajectory according to the joint position parameters and performing smooth optimization on the motion trajectory to obtain an optimized trajectory includes: The optimization trajectory is obtained as follows: Among them, P smooth (t) is the path position of the interpolated and smoothed path point at sampling time t, S() represents the smoothing function, ω i ′ is the interpolation point weight, P(t k ) is the path point sequence before optimization, f(t,t k ) is the interpolation function.

11. A humanoid robot teleoperation device, characterized in that: The device comprises: Motion capture device, used to obtain raw data of the original motion of the imitation target; A motion redirector, used for obtaining joint position parameters according to the raw data; A trajectory planner, used to obtain a motion trajectory according to the joint position parameters, and to smoothly optimize the motion trajectory to obtain an optimized trajectory; The actuator is used to generate real-time motor control instructions according to the optimized trajectory to drive the joints of the humanoid robot.

12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images