Motion retargeting method and apparatus for full humanoid robots based on motion capture devices

Abstract

The application relates to a full human-shaped robot motion reorientation method and device based on a motion capture device, the method comprising: converting human body posture quaternions collected by the motion capture device into a rotation matrix, and combining an Euler angle decomposition strategy to realize accurate mapping from a Cartesian space to a robot joint space; taking joint angles decomposed by the quaternions as initial solutions, introducing various physical constraints in real time according to an actual support state of the robot, constructing and solving inverse kinematics problems with various constraints, and correcting lower limb joint angles on line, so that the robot can strictly meet joint limiting, self-collision avoidance and dynamic balance requirements while replicating operator actions, thereby effectively unifying the accuracy of motion simulation and system stability.

Description

Technical Field

[0001] This application relates to the field of robot motion control, specifically to a method and apparatus for redirecting the motion of a humanoid robot based on motion capture equipment. Background Technology

[0002] Human motion capture technology has become a key tool in fields such as medical rehabilitation and human-computer interaction. In particular, systems based on inertial sensors are gradually replacing traditional optical solutions due to their advantages of low cost, strong environmental adaptability, and ease of remote application. Motion capture equipment uses multi-axis inertial measurement units (IMUs) to capture acceleration, angular velocity, and magnetic field data, enabling accurate description and real-time reconstruction of motion posture, providing an important data foundation for human motion modeling.

[0003] In terms of motion posture description, although various optimization methods based on pre-recorded marker data exist, most current research focuses on upper body posture decomposition and mapping, while research on lower limb movement and whole-body coordination redirection remains relatively limited. Furthermore, due to the large range of motion and complex joint structures of the human body, using Euler angles for posture representation can easily produce singularities during large-amplitude movements, affecting the accuracy and stability of motion data. Summary of the Invention

[0004] To overcome at least one deficiency in the prior art, this application provides a method and apparatus for redirecting the motion of a humanoid robot based on motion capture equipment.

[0005] Firstly, a method for motion redirection of a humanoid robot based on motion capture equipment is provided, including:

[0006] The rotational posture data of each joint of the human body is acquired by the motion capture device. The rotational posture data is represented in the form of quaternions. Each joint of the human body includes single-degree-of-freedom joints and multi-degree-of-freedom joints.

[0007] Based on the rotational posture data of each joint of the human body, the joint angles of each joint of the robot are determined; for single-degree-of-freedom joints, the angle of rotation of the quaternion around the rotation axis is determined based on the quaternion, and the angle is used as the joint angle; for multi-degree-of-freedom joints, the multiple joints of the robot decomposed by the multi-degree-of-freedom joint and the rotation direction corresponding to each joint are determined, and the joint angle of each joint of the robot obtained by the decomposition is calculated based on the quaternion; each joint of the robot includes upper body joints and lower body joints.

[0008] For single-degree-of-freedom joints of the upper body, the difference between the current joint angle and the initial joint angle is used as the incremental control command for the corresponding joint; for multi-degree-of-freedom joints of the upper body, the current joint angle is used as the incremental control command for the corresponding joint.

[0009] For the lower body joints, the distance between the robot's center of mass and the support point is calculated using forward kinematics based on the joint angles. The robot's support state is determined based on the distance and the robot's support state determination rules. The robot's support states include bipedal support, unipedal support, and transitional states. Based on the robot's support state, corresponding constraints are applied to the robot, and the corrected joint angles are obtained using inverse kinematics. The corrected joint angles are then used as incremental control commands for the corresponding joints.

[0010] In one embodiment, for a single-degree-of-freedom joint, the angle of rotation of the quaternion about the rotation axis is determined based on the quaternion using the following formula:

[0011]

[0012] in, It is a quaternion. For quaternions in Rotational component in the axial direction, For quaternions in Rotational component in the axial direction, For quaternions in Rotational component in the axial direction, Let be the angle of rotation of the quaternion about the rotation axis. The axis of rotation for quaternions. Indicates the size of a quaternion.

[0013] In one embodiment, the joint angles of each joint of the robot, calculated and decomposed using quaternions, include:

[0014] The internal rotation direction of a multi-degree-of-freedom joint is determined based on the rotation direction corresponding to each joint.

[0015] Based on the quaternions before and after the rotation of the multi-degree-of-freedom joint, construct the relative rotation matrix in the internal rotation direction;

[0016] Construct a relative rotation matrix for the external rotation direction based on the internal rotation direction;

[0017] Based on the relative rotation matrices in the inward rotation direction and the relative rotation matrices in the outward rotation direction, the joint angles of each joint of the robot obtained from the decomposition are determined.

[0018] In one embodiment, the robot's support state is determined based on distance and robot support state determination rules, including:

[0019] like Then the robot's supporting state is a bipedal supporting state; where, The distance between the robot's center of mass and the support point. For the height of the center of mass, It is the acceleration due to gravity. The distance between the zero torque point ZMP and the support point;

[0020] like ,and Then the robot's supported state is a transitional state; where, The length of the robot's foot. The width of the robot's foot;

[0021] like If so, the robot's support state is a single-leg support state.

[0022] In one embodiment, based on the robot's support state, corresponding constraints are applied to the robot, including:

[0023] If the robot's support state is a bipedal support state, then a constraint is applied to ensure that both feet are at the same height, using the following formula:

[0024]

[0025]

[0026]

[0027] in, The left foot is the center of gravity. This refers to the angle of the joint in the lower body that belongs to the left leg. The position is centered on the right foot. The angle of the joint in the lower body that belongs to the right leg; The coordinates of the center of the left foot Center around the left foot The rotation angle of the shaft, Center around the left foot The rotation angle of the shaft, Center around the left foot The rotation angle of the shaft, The coordinates of the center of the right foot. Centering around the right foot The rotation angle of the shaft, Centering around the right foot The rotation angle of the shaft, Center around the right foot The rotation angle of the shaft, This indicates transpose.

[0028] In one embodiment, based on the robot's support state, corresponding constraints are applied to the robot, including:

[0029] If the robot's supported state is a transitional state, then a posture constraint of the hips being parallel to the ground is applied to the robot using the following formula:

[0030]

[0031]

[0032] in, This is the increment of control volume when the left foot is supporting the weight. The proportional gain coefficient for the left leg. The pitch angle is the data obtained from the waist IMU. To control the cycle; This is the increment of control volume when the right foot is supporting the weight. This is the proportional gain coefficient for the right leg.

[0033] In one embodiment, based on the robot's support state, corresponding constraints are applied to the robot, including:

[0034] If the robot is in a single-leg support state, then a constraint is applied to keep the supporting leg parallel to the ground, using the following formula:

[0035]

[0036]

[0037] in, This represents the increment of the control quantity for the pitch axis. This is the proportional gain coefficient. To support the roll angle of data acquired by the leg and foot IMU, To control the cycle; This represents the control increment for the roll axis. This is the proportional gain coefficient. The pitch angle obtained by the IMU for supporting the legs and feet.

[0038] In one embodiment, single-degree-of-freedom joints include the knee joint and the elbow joint; multi-degree-of-freedom joints include the head joint, left shoulder joint, right shoulder joint, waist joint, left hip joint, right hip joint, left ankle joint, and right ankle joint.

[0039] Secondly, a motion redirection device for a humanoid robot based on motion capture equipment is provided to implement the above-mentioned motion redirection method for a humanoid robot based on motion capture equipment.

[0040] Thirdly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the aforementioned motion redirection method for a humanoid robot based on a motion capture device.

[0041] Compared with the prior art, this application has the following beneficial effects:

[0042] 1. This application converts the human posture quaternions collected by the motion capture device into rotation matrices and combines them with the Euler angle decomposition strategy to achieve accurate mapping from Cartesian space to robot joint space. This method is specifically optimized for complex joint structures with multiple degrees of freedom, such as the hip joint. By establishing a multi-rotation center kinematic model and a local coordinate system separation strategy, the decoupling problem of coupled rotational motion is effectively handled.

[0043] 2. This application fully considers real-time changes in contact and support states. Using quaternion-decomposed joint angles as the initial solution, various physical constraints are introduced in real-time based on the robot's actual support state (bipedal / monopalal / transitional state), including: balance constraints based on the zero-moment point (ZMP) to ensure the center of mass always remains within the current support polygon; foot height constraints based on forward kinematics; and horizontal constraints on the torso and supporting foot posture based on IMU data. Furthermore, by constructing and solving an inverse kinematics problem with multiple constraints, the lower limb joint angles are corrected online. This allows the robot to reproduce the operator's actions while strictly meeting requirements for joint limitation, self-collision avoidance, and dynamic balance, thus achieving an effective balance between the accuracy of motion imitation and system stability. Attached Figure Description

[0044] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings:

[0045] Figure 1 A flowchart of a motion redirection method for a humanoid robot based on motion capture equipment is shown.

[0046] Figure 2 A schematic diagram of the movement postures of the main human joints is shown;

[0047] Figure 3 A schematic diagram of each joint of the robot is shown;

[0048] Figure 4 The diagram shows the transition relationships between different states during the remapping process. Detailed Implementation

[0049] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.

[0050] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0051] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0052] This application provides a method for motion redirection of a humanoid robot based on motion capture equipment. Figure 1 A flowchart illustrating a motion retargeting method for a humanoid robot based on motion capture equipment is shown. (See attached image.) Figure 1 The method mainly includes the following steps:

[0053] Step S1: Obtain rotational posture data of each joint of the human body collected by the motion capture device. The rotational posture data is represented in the form of quaternions. Each joint of the human body includes single-degree-of-freedom joints and multi-degree-of-freedom joints.

[0054] The operator wears motion capture equipment and performs a target action; the system collects and records in real time the rotational posture data of the major joints of the human body as they change over time. This data is represented in quaternion form, denoted as . ,in, Indicate the size of a quaternion; In order to be in Rotational component in the axial direction; For quaternions in Rotational component in the axial direction, For quaternions in The rotational component along the axis. This motion capture device generates pose information for the corresponding human joints based on the SMPL (Skinned Multi-Person Linear) model. Figure 2 A schematic diagram of the motion poses of the main human joints is shown. To map human motion to the robot model, the system uses a common reference pose, the so-called T-pose. The global coordinate system is defined as follows: The axis points in the left-hand direction during the T-pose state. The axis is vertically upward. The axes are determined according to the rules of the right-hand coordinate system; the origin of the global coordinate system is set as the vertical projection point of the hip joint on the ground plane. The joints of the human body are divided into single-degree-of-freedom joints and multi-degree-of-freedom joints, as shown in Table 1.

[0055] Table 1

[0056]

[0057] Step S2: Based on the rotational posture data of each joint of the human body, determine the joint angles of each joint of the robot; for single-degree-of-freedom joints, determine the angle of rotation of the quaternion around the rotation axis based on the quaternion, and use the angle as the joint angle; for multi-degree-of-freedom joints, determine the multiple joints of the robot decomposed by the multi-degree-of-freedom joint and the rotation direction corresponding to each joint, and calculate the joint angle of each joint of the robot obtained by decomposition based on the quaternion; each joint of the robot includes upper body joints and lower body joints.

[0058] Based on the rotational posture data of each joint of the human body, an Euler angle decomposition strategy is used to map and decompose it into the corresponding joints of the robot. Figure 3 A schematic diagram of each joint of the robot is shown. Taking the left hip joint as an example, although the human hip joint captured by the motion capture device is represented by only a quaternion for its overall posture, the human hip joint actually has three rotational degrees of freedom (similar to a ball joint structure) during movement. To accurately reproduce the movement of this part, the posture changes of the left hip joint need to be decomposed and assigned to the three single-degree-of-freedom joints of the robot's leg (e.g., the left hip joints L_hip_0, L_hip_1, and L_hip_2) for collaborative representation. Therefore, in Table 1, such joints are classified as "multi-degree-of-freedom joints," meaning that one motion capture joint needs to correspond to multiple single-degree-of-freedom joints of the robot for joint drive to achieve high-precision mapping of human multi-degree-of-freedom movements.

[0059] Specifically, for a single-degree-of-freedom joint, the angle of rotation of the quaternion around the rotation axis is determined based on the quaternion using the following formula:

[0060]

[0061] in, It is a quaternion. For quaternions in Rotational component in the axial direction, For quaternions in Rotational component in the axial direction, For quaternions in Rotational component in the axial direction, Let be the angle of rotation of the quaternion about the rotation axis. The axis of rotation for quaternions. Indicates the size of a quaternion.

[0062] Specifically, for multi-degree-of-freedom joints, firstly, the multiple joints of the robot decomposed from multi-degree-of-freedom joints and the corresponding rotation direction of each joint are determined. Taking the left hip joint as an example, the left hip joints L_hip_0, L_hip_1, and L_hip_2 are the determined multiple joints of the decomposed robot. The multiple joints of the decomposed robot corresponding to other multi-degree-of-freedom joints can be referenced. Figure 3 Then, using the controlled variable method, the operator was instructed to perform movements in a single direction, and the rotation trend of the coordinate system represented by the quaternions was observed to determine the axis of rotation. Through experimental analysis, it was determined that the left hip joints L_hip_0, L_hip_1, and L_hip_2 correspond to roll, yaw, and pitch, respectively, with roll corresponding to the vertical axis (…). The yaw axis is the axis of rotation, and the yaw corresponds to the vertical axis. The left and right turns of the axis, pitch corresponds to the horizontal axis ( The pitching motion of the axis.

[0063] Then, the joint angles of each joint of the robot, obtained from the quaternion decomposition, are calculated, including:

[0064] The internal rotation direction of a multi-degree-of-freedom joint is determined based on the rotation direction corresponding to each joint. Taking the left hip joint as an example, L_hip_0, L_hip_1, and L_hip_2 of the left hip joint correspond to roll, yaw, and pitch, respectively. Therefore, the internal rotation direction is... .

[0065] Based on the quaternions before and after the multi-degree-of-freedom joint rotation, construct the relative rotation matrix in the internal rotation direction; specifically, the following formula can be used:

[0066]

[0067] in, Let be the relative rotation matrix in the inward rotation direction. The rotation matrix is ​​the quaternion transformation before the rotation of a multi-degree-of-freedom joint. The rotation matrix is ​​the quaternion transformation after rotation of a multi-degree-of-freedom joint. for The inverse matrix.

[0068] Construct a relative rotation matrix for the external rotation direction based on the inward rotation direction; due to the inward rotation... Equivalent to external rotation Therefore, the relative rotation matrix can also be expressed as:

[0069]

[0070] in, Let be the relative rotation matrix in the external rotation direction. To bypass Rotation angle of the shaft The corresponding rotation matrix, To bypass Rotation angle of the shaft The corresponding rotation matrix, To bypass Rotation angle of the shaft The corresponding rotation matrix.

[0071]

[0072]

[0073]

[0074] Ultimately, we obtained:

[0075]

[0076] Based on the relative rotation matrices in the inward rotation direction and the relative rotation matrices in the outward rotation direction, the joint angles of each joint of the robot obtained from the decomposition are determined.

[0077] because and Since they are equal, the joint angles of each joint of the robot obtained from the decomposition are:

[0078]

[0079]

[0080]

[0081] in, express The Middle Line number The elements of the column.

[0082] Here, we take the left hip joint as an example. The joint angle corresponding to L_hip_0 of the left hip joint. The joint angle corresponding to the left hip joint L_hip_1, The joint angle corresponding to the left hip joint L_hip_2.

[0083] Step S3: For single-degree-of-freedom joints of the upper body, the difference between the current joint angle and the initial joint angle value is used as the incremental control command for the corresponding joint; for multi-degree-of-freedom joints of the upper body, the current joint angle is used as the incremental control command for the corresponding joint.

[0084] Here, both single-DOF and multi-DOF joints employ an incremental control strategy. The fundamental reason is that the joints of different robots are affected by variations in mechanical structure, assembly tolerances, and zero-point definitions, resulting in inherent angular offsets. If absolute angles obtained through quaternion decomposition are used directly for control, this offset will cause deviations between the robot's reproduced posture and the original human movement.

[0085] Before initiating motion control, system calibration is essential: the operator and robot simultaneously maintain a T-pose, and using this calibration pose as a reference, record the quaternion data for each joint collected by the motion capture equipment. After decomposition, the initial angle reference values ​​for each joint are obtained. In subsequent real-time control, the joint angle calculated based on the new quaternion data at each moment is subtracted from the corresponding initial reference value, and the difference is used as the incremental control command sent to the single-degree-of-freedom joints. For multi-degree-of-freedom joints, the angle increments obtained from the quaternion decomposition before and after rotation are themselves directly applicable.

[0086] Step S4: For the lower body joints, calculate the distance between the robot's center of mass and the support point using forward kinematics based on the joint angles; determine the robot's support state based on the distance and the robot's support state determination rules; the robot's support states include bipedal support state, unipedal support state, and transitional state; apply corresponding constraints to the robot based on its support state, and calculate the corrected joint angles using inverse kinematics; use the corrected joint angles as incremental control commands for the corresponding joints.

[0087] The incremental control results are used as the initial joint states for motion redirection. Upper body joint data can typically be directly mapped to the robot, while lower body data requires further online inverse kinematics correction. This method effectively eliminates systematic errors caused by mechanical offsets, unifies and simplifies the control architecture, and ensures the accuracy of motion mapping and system reliability.

[0088] Here, after adding constraints, the joint angles of the lower body joints are used as the initial solution. Inverse kinematics calculations are used to obtain joint angles that meet the requirements of balance and physical feasibility. The final corrected joint angles are then applied to the robot's leg control, thereby achieving highly realistic motion reproduction under dynamic balance.

[0089] In one embodiment, motion redirection plays a crucial role in controlling humanoid robots using motion capture devices. It enables the robot to directly replicate human movements, thereby reducing the complexity of robot programming and enhancing its ability to learn through observation. However, due to significant differences between humans and robots in height, limb proportions, and joint structures, directly mapping human movements may cause the robot to exceed its physical limitations or lose balance.

[0090] Therefore, this embodiment designs a motion redirection framework that considers real-time changes in contact support and introduces constraints based on inverse kinematics to process the generated motion in stages: applying constraints such as joint limits, speed limits, and self-collision avoidance, and performing balance compensation when necessary, so that the humanoid robot can accurately reproduce human motion captured by wearable motion capture devices while maintaining full-body balance.

[0091] To achieve basic robot motion reproduction based on motion capture data, a constraint control method based on gait temporal decomposition is proposed. This method divides continuous motion into different support states and applies corresponding physical constraints to each state. Taking walking as an example, the gait is divided into a bipedal support state, a ZMP transition state, and a unipedal support state, and corresponding constraint strategies are designed for each: In the bipedal support state, the zero moment point (ZMP) is constrained to move within the support polygon formed by the two feet, while maintaining the same height of the two feet; in the ZMP transition state, the center of gravity is controlled to move smoothly towards the supporting leg, while simultaneously performing a leg lift; in the unipedal support state, the ZMP is constrained to always be located within the support polygon formed by the current supporting foot, ensuring that the foot maintains horizontal contact with the ground.

[0092] First, determine the support surfaces for each state, including the support surface for the left foot, the support surface for the right foot, and the support surface for both feet. Taking the calculation of the support surface for the left foot as an example, the calculation process will be described.

[0093] Based on the joint angles of each joint of the robot obtained in step S3, the pose of the left foot center relative to the robot's center of mass (the center position between the left and right hips) is calculated using forward kinematics. .

[0094] The four corner points of the left foot are fixed points, and the position of each corner point in the local coordinate system can be determined by the position matrix. Unified description:

[0095]

[0096] in, Representing corner point Offset along the length direction, Representing corner point Offset along the width direction.

[0097] Different corner points correspond to different The specific values ​​are shown in Table 2:

[0098] Table 2

[0099]

[0100] in, The length of the robot's foot. This refers to the width of the robot's foot.

[0101] Calculate the transformation matrix of the corner point relative to the centroid:

[0102]

[0103] in, Corner point The transformation matrix relative to the centroid.

[0104] The coordinates formed by the first three elements of the fourth column are the corner points. Once the coordinates of the four corner points are determined, the range of the left foot's support surface can be obtained. The method for determining the right foot's support surface is the same as for the left foot. It should be noted that the range of both feet's support surfaces is equal to the range determined by the four adjacent corner points of the left and right feet, plus the range of the left foot's support surface and the range of the right foot's support surface.

[0105] Robots differ fundamentally from humans in terms of dynamic stability. The human body possesses a high center of mass acceleration and jerk, enabling it to maintain ZMP balance in an inverted pendulum system during the instantaneous leg lift through high acceleration. Robots, limited by hardware performance, cannot achieve such high instantaneous center of mass acceleration, thus requiring gait constraints. The following formula is the inverted pendulum formula for calculating ZMP:

[0106]

[0107] in, For the acceleration of the center of mass, It is the acceleration due to gravity. For the height of the center of mass, The distance between the robot's center of mass and the support point. Let ZMP be the distance between the zero-moment point and the support point (the center point of the support surface), under the condition that equilibrium is satisfied. It should coincide with the support point, that is .

[0108] This embodiment uses the acceleration of the center of mass. With maximum center of mass acceleration By comparing the distance and the robot support state determination rules, the robot support state determination rules are determined. In this embodiment, step S4, determining the robot support state based on the distance and the robot support state determination rules, includes:

[0109] 1) If Then the robot's supporting state is a bipedal supporting state; where, The distance between the robot's center of mass and the support point can be calculated using forward kinematics based on the joint angles of each joint of the robot obtained in S3. For the height of the center of mass, It is the acceleration due to gravity. The distance between the zero-moment point ZMP and the support point is given; at this point, the equilibrium condition is not met.

[0110] 2) If ,and Then the robot's supported state is a transitional state; where, The length of the robot's foot. Let be the width of the robot's foot; at this point, ZMP balance is satisfied.

[0111] 3) If If the robot is in a single-leg support state, then it can achieve COG (center of gravity) balance.

[0112] Figure 4 The diagram shows the transition relationships between different states during the remapping process.

[0113] Specifically, if the robot's support state is a bipedal support state, then a constraint is applied to ensure that both feet are at the same height, using the following formula:

[0114]

[0115]

[0116]

[0117] in, The left foot is the center of gravity. This refers to the angle of the joint in the lower body that belongs to the left leg. The position is centered on the right foot. The angle of the joint in the lower body that belongs to the right leg; The coordinates of the center of the left foot The center of the left foot axis, axis, The coordinates of the axis, Center around the left foot The rotation angle of the shaft, Center around the left foot The rotation angle of the shaft, Center around the left foot The rotation angle of the shaft, The coordinates of the center of the right foot. The center of the right foot axis, axis, The coordinates of the axis, Center around the right foot The rotation angle of the shaft, Center around the right foot The rotation angle of the shaft, Center around the right foot The rotation angle of the shaft, This indicates transpose. Here, and It can be calculated using forward kinematics based on the current joint angle.

[0118] If the robot's supported state is a transitional state, then a posture constraint of the hips being parallel to the ground is applied to the robot using the following formula:

[0119]

[0120]

[0121] in, This is the increment of control volume when the left foot is supporting the weight. The proportional gain coefficient for the left leg. The pitch angle is the data obtained from the waist IMU, representing the current parallel angle between the waist and the ground; To control the cycle; This is the increment of control volume when the right foot is supporting the weight. This is the proportional gain coefficient for the right leg.

[0122] If the robot is in a single-leg support state, then a constraint is applied to keep the supporting leg parallel to the ground, using the following formula:

[0123]

[0124]

[0125] in, This represents the increment of the control quantity for the pitch axis. This is the proportional gain coefficient. To support the roll angle of data acquired by the leg and foot IMU, To control the cycle; This represents the control increment for the roll axis. This is the proportional gain coefficient. The pitch angle obtained by the IMU for supporting the legs and feet. and Used to indicate the distance between the current foot posture and the parallelism of the ground.

[0126] It should be noted that this application modifies the lower body joint angles based on motion capture data. Under the premise of satisfying stability constraints, the robot can still reproduce the operator's movements to the greatest extent. For example, in a single-leg support state, the non-supporting leg can move completely according to the motion capture data. This method prioritizes stability, uses joint angle tracking as a means, and enhances the robustness and motion generalization ability of the system through a ZMP relocation correction strategy. Thus, while reproducing the operator's movements, it strictly ensures the robot's motion feasibility, balance stability, and safety.

[0127] This application also provides a motion redirection device for a humanoid robot based on motion capture equipment, used to implement the motion redirection method for a humanoid robot based on motion capture equipment described in the foregoing embodiments.

[0128] The motion redirection device for humanoid robots based on motion capture equipment in this embodiment has the same inventive concept as the motion redirection method for humanoid robots based on motion capture equipment described above. Therefore, the specific implementation of this device can be found in the embodiment section of the motion redirection method for humanoid robots based on motion capture equipment described above, and its technical effects correspond to the technical effects of the above method, so it will not be repeated here.

[0129] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the above-described motion redirection method for a humanoid robot based on motion capture equipment.

[0130] In summary, this application has the following technical effects:

[0131] 1. This application converts the human posture quaternions collected by the motion capture device into rotation matrices and combines them with the Euler angle decomposition strategy to achieve accurate mapping from Cartesian space to robot joint space. This method is specifically optimized for complex joint structures with multiple degrees of freedom, such as the hip joint. By establishing a multi-rotation center kinematic model and a local coordinate system separation strategy, the decoupling problem of coupled rotational motion is effectively handled.

[0132] 2. This application fully considers real-time changes in contact and support states. Using quaternion-decomposed joint angles as the initial solution, various physical constraints are introduced in real-time based on the robot's actual support state (bipedal / monopalal / transitional state), including: balance constraints based on the zero-moment point (ZMP) to ensure the center of mass always remains within the current support polygon; foot height constraints based on forward kinematics; and horizontal constraints on the torso and supporting foot posture based on IMU data. Furthermore, by constructing and solving an inverse kinematics problem with multiple constraints, the lower limb joint angles are corrected online. This allows the robot to reproduce the operator's actions while strictly meeting requirements for joint limitation, self-collision avoidance, and dynamic balance, thus achieving an effective balance between the accuracy of motion imitation and system stability.

[0133] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for motion redirection of a humanoid robot based on motion capture equipment, characterized in that, The method comprises the following steps: Obtain the rotation posture data of each joint of a human body collected by a motion capture device, wherein the rotation posture data is in the form of a quaternion; the joints of the human body include single degree of freedom joints and multi degree of freedom joints; Determine the joint angles of each joint of a robot according to the rotation posture data of each joint of the human body; for a single degree of freedom joint, determine the angle of rotation of the quaternion around the rotation axis according to the quaternion, and take the angle as the joint angle; For a multi degree of freedom joint, determine the joints of the robot decomposed from the multi degree of freedom joint and the rotation direction corresponding to each joint, and calculate the joint angle of each joint of the robot decomposed according to the quaternion; the joints of the robot include upper body joints and lower body joints; For a single degree of freedom joint of the upper body joints, take the difference between the joint angle at the current moment and the initial value of the joint angle as the incremental control instruction of the corresponding joint; for a multi degree of freedom joint of the upper body joints, take the joint angle at the current moment as the incremental control instruction of the corresponding joint; For the lower body joints, calculate the distance between the robot centroid and the support point by using forward kinematics according to the joint angle; determine the support state of the robot according to the distance and a robot support state determination rule; the support state of the robot includes a double foot support state, a single foot support state and a transition state; According to the support state of the robot, apply the corresponding constraint condition to the robot, and calculate the corrected joint angle by using inverse kinematics; take the corrected joint angle as the incremental control instruction of the corresponding joint.

2. The method of claim 1, wherein, Wherein, For a single degree of freedom joint, determine the angle of rotation of the quaternion around the rotation axis according to the quaternion, using the following formula: wherein, is a quaternion, is a quaternion in is a rotation component of the quaternion in is a quaternion in is a rotation component of the quaternion in is a quaternion in is a rotation component of the quaternion in is an angle of rotation of the quaternion about the rotation axis, is a rotation axis of the quaternion, denotes the magnitude of the quaternion.

3. The method of claim 1, wherein, Wherein, Calculate the joint angle of each joint of the robot decomposed according to the quaternion, including: Determine the internal rotation direction of the multi degree of freedom joint according to the rotation direction corresponding to each joint; Construct the relative rotation matrix of the internal rotation direction according to the quaternions before and after the rotation of the multi degree of freedom joint; Construct the relative rotation matrix of the external rotation direction according to the internal rotation direction; Determine the joint angle of each joint of the robot decomposed based on the relative rotation matrix of the internal rotation direction and the relative rotation matrix of the external rotation direction.

4. The method of claim 1, wherein, Wherein, Determine the support state of the robot according to the distance and a robot support state determination rule, including: If , the support state of the robot is a double support state; wherein, is the distance between the center of mass of the robot and the support point, is the height of the center of mass, is the acceleration of gravity, is the distance between the zero moment point (ZMP) and the support point; If , and , the support state of the robot is a transition state; wherein, is the length of the robot foot, is the width of the robot foot; If , the support state of the robot is a single-foot support state.

5. The method of claim 1, wherein, Wherein, Apply the corresponding constraint condition to the robot according to the support state of the robot, including: If the support state of the robot is the double foot support state, apply the double foot height consistent constraint to the robot, using the following formula: wherein, is a pose of the center of the left foot, is an angle of a joint among the lower body joints that belongs to the left leg, is a pose of the center of the right foot, is an angle of a joint among the lower body joints that belongs to the right leg; is a coordinate of the center of the left foot, are coordinates of the center of the left foot along the x-axis, the y-axis, the z-axis, respectively, is a rotation angle of the center of the left foot around the x-axis, is a rotation angle of the center of the left foot around the y-axis, is a rotation angle of the center of the left foot around the z-axis, is a rotation angle of the center of the left foot around the x-axis, is a rotation angle of the center of the left foot around the y-axis, is a rotation angle of the center of the left foot around the z-axis, is a coordinate of the center of the right foot, are coordinates of the center of the right foot along the x-axis, the y-axis, the z-axis, respectively, is a rotation angle of the center of the right foot around the x-axis, is a rotation angle of the center of the right foot around the y-axis, is a rotation angle of the center of the right foot around the z-axis, is a rotation angle of the center of the right foot around the x-axis, is a rotation angle of the center of the right foot around the y-axis, is a rotation angle of the center of the right foot around the z-axis, denotes the transpose.

6. The method of claim 1, wherein, Wherein, Apply the corresponding constraint condition to the robot according to the support state of the robot, including: If the support state of the robot is the transition state, apply the posture constraint that the hip is parallel to the ground to the robot, using the following formula: wherein, is a control amount increment when the left foot is supported, is a proportional gain coefficient of the left leg, is a pitch angle obtained from data of the waist IMU, is a control period; is a control amount increment when the right foot is supported, is a proportional gain coefficient of the right leg.

7. The method of claim 1, wherein, Wherein, Apply the corresponding constraint condition to the robot according to the support state of the robot, including: If the support state of the robot is the single foot support state, apply the constraint that the support foot is parallel to the ground to the robot, using the following formula: wherein, is a control amount increment of the pitch axis, is a proportional gain coefficient, is a roll angle acquired from the support leg foot IMU, is a control period; is a control amount increment of the roll axis, is a proportional gain coefficient, is a pitch angle acquired from the support leg foot IMU.

8. The method of claim 1, wherein, The single degree of freedom joints include knee joints and elbow joints; the multi-degree of freedom joints include head joints, left shoulder joints, right shoulder joints, waist joints, left hip joints, right hip joints, left ankle joints and right ankle joints.

9. A motion redirection device for a humanoid robot based on motion capture equipment, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the motion re-targeting method of the full humanoid robot based on the motion capture device according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the motion re-targeting method of the full humanoid robot based on the motion capture device according to any one of claims 1-8.

Images