A Humanoid Robot Control Method and System Based on Action Mapping

Abstract

The present disclosure provides a humanoid robot control method and system based on action mapping, which relates to the technical field of humanoid robots, and includes: constructing a humanoid robot animation model according to the servo distribution of the humanoid robot to be controlled and performing bone binding; obtaining the human body visual motion capture data of the action to be reproduced, performing joint-level action mapping on the human body visual motion capture data and the humanoid robot animation model to obtain the action data of the mapped humanoid robot model; grading the task priorities of the humanoid robot torso, arms and legs, and combining the smoothed action data to solve the joint torque through inverse dynamics; using the joint torque as a feedforward quantity to perform reproduction control of the actions of the humanoid robot; through the human body visual motion capture data and the humanoid robot animation model, the present invention significantly improves the efficiency of generating anthropomorphic actions of the humanoid robot, solves the technical problems in the coherent action mapping of the multi-degree-of-freedom humanoid robot, and reduces the design difficulty.

Description

Technical Field

[0001] The present disclosure relates to the technical field of humanoid robots, and particularly to a control method and system for humanoid robots based on motion mapping. Background Art

[0002] Human motion capture originated from the film and animation industries, which can accurately map human motions onto 3D models to achieve realistic effects; humanoid robots adapt to human environmental tasks by imitating the human form and motion patterns; currently, voice technology is relatively mature in humanoid robot interaction, but relying solely on voice communication often appears rigid; by adding non-verbal communication such as limb movements, humanoid robots can interact more humanely and naturally; therefore, the demand for humanoid robots to learn and imitate complex human motions is increasing; directly mapping motion capture data to the corresponding joints of humanoid robots and reproducing complex human motion behaviors therefrom will improve the capabilities of humanoid robots in collaboration, interaction, and performing complex tasks.

[0003] When controlling the motion of humanoid robots in special environments, existing methods face multiple challenges in the process of generating humanoid robot behaviors, such as high professional technical requirements and low motion design efficiency, which limit their promotion and efficient operation. Summary of the Invention

[0004] To solve the above problems, the present disclosure proposes a control method and system for humanoid robots based on motion mapping. Through human visual motion capture data and humanoid robot animation models, visualization mapping and control can be performed without in-depth understanding of humanoid robot technology, significantly improving the efficiency of generating anthropomorphic actions of humanoid robots, solving the technical problems in the coherent motion mapping of multi-degree-of-freedom humanoid robots, and reducing the design difficulty.

[0005] According to some embodiments, the present disclosure adopts the following technical solutions:

[0006] A control method for a humanoid robot based on motion mapping, comprising:

[0007] Construct a humanoid robot animation model according to the servo distribution of the humanoid robot to be controlled, and perform bone binding;

[0008] Obtain human visual motion capture data of the action to be reproduced, perform joint-level motion mapping on the human visual motion capture data and the humanoid robot animation model, and obtain the action data of the mapped humanoid robot animation model;

[0009] Perform task priority grading on the torso, arms, and legs of the humanoid robot, and combine the smoothed action data to solve for joint torques through inverse dynamics;

[0010] Use the joint torques as feedforward quantities to perform reproduction control of the actions of the humanoid robot.

[0011] According to some embodiments, the present disclosure adopts the following technical solutions:

[0012] A humanoid robot control system based on action mapping, comprising a model construction module, an action mapping module, a torque solving module, and an action control module:

[0013] The model construction module is configured to: construct a humanoid robot animation model according to the servo distribution of the humanoid robot to be controlled, and perform bone binding;

[0014] The action mapping module is configured to: obtain the human body visual motion capture data of the action to be reproduced, perform joint-level action mapping on the human body visual motion capture data and the humanoid robot animation model, and obtain the action data of the mapped humanoid robot animation model;

[0015] The torque solving module is configured to: classify the task priorities of the humanoid robot's torso, arms, and legs, and combine the smoothed action data to solve the joint torque through inverse dynamics;

[0016] The action control module is configured to: use the joint torque as a feedforward quantity to perform reproduction control of the actions of the humanoid robot.

[0017] According to some embodiments, the present disclosure adopts the following technical solutions:

[0018] A computer program product, comprising a computer program, where the computer program, when executed by a processor, implements the described humanoid robot control method based on action mapping.

[0019] According to some embodiments, the present disclosure adopts the following technical solutions:

[0020] A non-transitory computer-readable storage medium for storing computer instructions, where the computer instructions, when executed by a processor, implement the described humanoid robot control method based on action mapping.

[0021] According to some embodiments, the present disclosure adopts the following technical solutions:

[0022] An electronic device, comprising: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes and implements the described humanoid robot control method based on action mapping.

[0023] Compared with the prior art, the beneficial effects of the present disclosure are:

[0024] Compared with the traditional humanoid robot artificial motion design, the present invention can directly perform joint-level mapping and optimize the mapped motion using 3D animation software to achieve smooth motion mapping.

[0025] The present invention can obtain the motion trajectory information and sole force of each bone in the humanoid robot animation model through the motion data extraction script, which is convenient for subsequent control of the humanoid robot using the motion data.

[0026] The present invention divides the priorities of multiple tasks for the humanoid robot and uses null space mapping to achieve full-body control, accurately reproducing the captured human motions on the basis of ensuring the stability of the humanoid robot. Brief Description of the Drawings

[0027] The attached drawings forming a part of this disclosure are used to provide a further understanding of this disclosure. The illustrative embodiments and descriptions thereof of this disclosure are used to explain this disclosure and do not constitute an improper limitation of this disclosure.

[0028] Figure 1 Schematic diagram of the human body visual motion capture data skeleton and the humanoid robot servo structure for Embodiment 1, where (a) is the human body visual motion capture data skeleton diagram and (b) is the humanoid robot servo structure diagram.

[0029] Figure 2 Flowchart of the method for the embodiments of this disclosure.

[0030] Figure 3 Diagram of the relationship between the human body visual motion capture data and the bones of the humanoid robot animation model for Embodiment 1, where (a) is the diagram of the relationship between the human body visual motion capture data bones and (b) is the diagram of the relationship between the bones of the finally formed humanoid robot animation model.

[0031] Figure 4 Flowchart of the motion data extraction for Embodiment 1. Detailed Description of the Embodiments

[0032] The following further describes this disclosure in conjunction with the attached drawings and embodiments.

[0033] It should be noted that the following detailed descriptions are all exemplary and are intended to provide a further description of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this disclosure belongs.

[0034] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they specify the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0035] Embodiment 1

[0036] In one embodiment of the present disclosure, a humanoid robot control method based on action mapping is provided, including:

[0037] According to the servo distribution of the humanoid robot to be controlled, construct a humanoid robot animation model and perform bone binding.

[0038] Obtain the human body visual motion capture data of the action to be reproduced, perform joint-level action mapping on the human body visual motion capture data and the humanoid robot animation model, and obtain the action data of the mapped humanoid robot animation model;

[0039] Perform task priority grading on the torso, arms and legs of the humanoid robot, and combine the smoothed action data to solve the joint torque through inverse dynamics;

[0040] Use the joint torque as a feedforward quantity to perform action reproduction control on the humanoid robot.

[0041] The implementation process of a humanoid robot control method based on action mapping in this embodiment will be described in detail below.

[0042] Regarding the problem of obtaining the behavior of a humanoid robot in a complex environment, when studying a humanoid robot control method based on action mapping, multiple issues need to be considered:

[0043] One is the singularity and multiple solution problems of the multi-degree-of-freedom manipulator: These problems will cause the displayed mapped actions to be inconsistent with the motion capture data. Handling the singularity and multiple solution problems is the key to ensuring the compliance and realism of the humanoid robot's actions.

[0044] The second is the acquisition of action data after action mapping: Since different types of humanoid robots use different kinematic models, the extracted action data such as the trajectories of the torso, arms and legs, and the sole forces will be different; in addition, the action state may be affected by previous actions, and continuity needs to be noted when extracting action data, which increases the implementation difficulty.

[0045] To address the above problems, in this embodiment, a humanoid robot control method based on action mapping is disclosed.

[0046] In this embodiment, first, humanoid robot motion mapping is performed in 3D animation software to visualize and simplify the implementation process, lower the entry threshold, and improve efficiency. Then, by writing scripts, key motion parameters of each joint of the humanoid robot animation model after mapping are extracted, such as position, velocity, acceleration, and sole force, that is, motion data. Then, the motion data is smoothed, and joint torques are generated based on the optimized data to control the motion reproduction of the humanoid robot.

[0047] In this embodiment, the skeleton of the human body visual motion capture data is as Figure 1 shown in (a) below. This human body visual motion capture data reflects the distribution positions of human bones, including the skull, spinal bone, upper arm bone, lower arm bone, wrist bone, hand bone, hip bone, thigh bone, calf bone, and foot bone. Each part reflects the motion data of the corresponding part of the human body motion; the servo structure of the humanoid robot is as Figure 1 shown in (b) below. Each robotic arm of this humanoid robot has 7 degrees of freedom and each robotic leg has 6 degrees of freedom; by comparing the skeleton of the human body visual motion capture data and the servo structure of the humanoid robot, it can be seen that there is a high similarity in the degrees of freedom of motion between the two. This similarity enables it to accurately reproduce complex motions in the human body motion capture data, ensuring the naturalness and accuracy of the motions.

[0048] A humanoid robot control method based on motion mapping disclosed in this embodiment, the overall process is as Figure 2 shown below, and the specific steps are as follows:

[0049] Step 1: Build a humanoid robot animation model in 3D animation software (such as Blender), perform bone binding, adjust bone rotation limits, and import human body visual motion capture data.

[0050] Specifically, step 1.1: Export the STL model files of all components of the humanoid robot through the 3D modeling software SolidWorks, and import the STL model files into the Blender animation software. Blender realizes the motion control of the model through the principle of "skinning animation", that is, binding the mesh entity to a specific "bone".

[0051] The skeleton is composed of a series of connected bones. Each bone has a position and a direction, and can form a hierarchical structure to form an overall skeleton; by rotating, translating, and scaling the bones, the shape and motion of the associated model can be controlled; in the humanoid robot animation model, the skeleton is used to represent the two robotic arms, two robotic legs, and the body structure of the humanoid robot, and control their motions.

[0052] Through the "skinning" operation, the mesh entity is bound to the bones one by one, and the parent-child relationship of the bones is set to form a complete skeleton to control the motion of the humanoid robot animation model.Figure 3 Figure (a) is a diagram of the bone relationship of human body visual motion capture data. Figure 3 Figure (b) is a diagram of the bone relationship of the final formed humanoid robot animation model.

[0053] As Figure 3 shown in Figure (b), the bones of the humanoid robot animation model are divided into three parts: the torso control bone, the arm control bone, and the leg control bone:

[0054] (1) Torso control bone:

[0055] Responsible for rotation and movement, achieving overall motion, and can obtain the motion data of the torso.

[0056] (2) Arm control bone:

[0057] It consists of seven control bones, including the shoulder joint pitch bone, the shoulder joint roll bone, the upper arm rotation bone, the elbow joint pitch bone, the forearm rotation bone, the wrist joint roll bone, and the wrist joint pitch bone. The seven control bones provide the rotation of seven degrees of freedom of the robotic arm.

[0058] (3) Leg control bone:

[0059] It consists of six control bones, including the hip joint roll bone, the hip joint pitch bone, the hip joint yaw bone, the knee joint pitch bone, the ankle joint roll bone, and the ankle joint pitch bone, and controls the leg movement through rotation.

[0060] Step 1.2: Precisely map the rotation characteristics (direction and angle range) of the physical humanoid robot servo to the bone rotation attributes of the humanoid robot animation model to ensure action consistency.

[0061] Step 1.3: Import the human body visual motion capture data.

[0062] The human body visual motion capture data is stored in the Biovision Hierarchy (BVH) format. BVH is a text file format used to represent motion capture data, which is used to describe the hierarchical relationship of the human body bone structure and actions. The BVH file organizes data in a hierarchical structure, which contains the bone hierarchical relationship, the rotation angles of the joints, and the time information of the action frames; each joint is defined as a node in the file and is connected through the parent-child relationship of the hierarchical structure; each node contains information such as the name, rotation order, and rotation angle of the joint; the time information of the action frames determines the moment of each action frame. After importing into the 3D animation software, the human body motion data can be visualized.

[0063] Step 2: Perform action mapping in a 3D animation software (such as Blender)

[0064] Specifically, step 2.1: Adjust the skeleton size of the human body visual motion capture data to match the skeleton size of the humanoid robot animation model. Add an inverse kinematics control bone to each end of the limbs of the humanoid robot animation model skeleton and bind it to the ends of the limbs of the skeleton. This inverse kinematics control bone drives the entire arm or leg to move by receiving the action data at the ends of the limbs of the humanoid robot animation model skeleton.

[0065] Step 2.2: Establish a child constraint between the inverse kinematics control bone of the humanoid robot animation model skeleton and the corresponding bones of the human body visual motion capture data skeleton to make the motion trajectories of the ends of the limbs of the two skeletons consistent.

[0066] Step 2.3: Use inverse kinematics to calculate the pose of the ends of the limbs of the current humanoid robot animation model, obtain the rotation or movement data of the joints of the robotic arm and robotic leg, and assign the corresponding rotation and movement control bones to the humanoid robot animation model to achieve action replication under the 3D animation software. The mapped action data is stored in the key frames of the time series.

[0067] In some frames, inverse kinematics calculation problems may occur, such as singular positions and multiple solution problems, resulting in inconsistent actions. For example, the robotic arm of the humanoid robot animation model has seven degrees of freedom. When performing a right hand waving action, a singular position may occur, resulting in action penetration. The detection and processing of the singular point position can be carried out in the following way.

[0068] The kinematic equation of the position of the end of the robotic arm is , where is the velocity vector of the end of the robotic arm in the Cartesian coordinate system, belonging to (m = 3, representing the position in three-dimensional space); is the angular displacement vector of each joint of the robotic arm, with a length of n (n = 7, representing the number of degrees of freedom of the robotic arm); contains the rotation and displacement information of each robotic arm segment relative to other joints, and are in the form of:

[0069]

[0070]

[0071] where (i = 1, …, 7) is the joint angle of the robotic arm, is the position component formed by the rotation matrix and displacement matrix of the i-th robotic arm segment relative to other robotic arm joints. The kinematic equation of the attitude of the end of the robotic arm is:

[0072]

[0073] Among them, is the angular velocity of the end of the robotic arm with respect to the body coordinate system, is the attitude Jacobian matrix, , is the attitude component formed by the transformation matrix and displacement matrix of the i-th robotic arm segment with respect to the seventh segment.

[0074] Define the position and attitude joint variables as , then there is , and the Jacobian matrix is defined as . Since is not a square matrix, the generalized inverse of needs to be calculated when solving the motion laws of each joint, and it is for calculation, which is .

[0075] When the robotic arm moves near the singular position, the modulus value of will approach or reach 0, and the condition number will tend to infinity; these two parameters change continuously when approaching the singular position, so they can be used as warning indicators; the modulus value of the matrix , and the condition number . Take the reciprocal of the condition number, that is

[0076] as the indicator.

[0077]

[0078] Among them, are the control points.

[0079] When interpolating between two key frames, first, determine them as the starting and ending points of the control points, ensure they are at both ends of the curve, and are tangent to the connecting lines of adjacent points to ensure smooth transition; then, select two other control points that have an important impact on the curve shape. Their positions directly affect the bending degree and direction of the curve, thus determining the path and curvature of the curve.

[0080] Step 3: Mapping Action Data Extraction and Optimization

[0081] Specifically, step 3.1: After completing the motion mapping in the 3D animation software, obtain the motion data of the humanoid robot animation model after mapping through the motion data extraction script. The motion data includes the position, linear velocity, and angular velocity of the torso, the position, velocity, and acceleration of each joint, and the sole force, etc. The extraction process is as Figure 4 shown, specifically:

[0082] Import the humanoid robot animation model and the human body visualization motion capture data;

[0083] Create an empty list to store the motion data;

[0084] Traverse each frame of the motion data of the humanoid robot animation model after mapping;

[0085] Obtain the motion data of each frame of the humanoid robot animation model;

[0086] Add the motion data to the list;

[0087] The finally obtained list is the extracted motion data.

[0088] Step 3.2: Smooth the extracted motion data.

[0089] Specifically, perform the following smoothing operations on the torso trajectory of the humanoid robot animation model, the motion trajectory of each joint, and its sole force respectively:

[0090]

[0091] Among them, represents the time interval required for each joint of the humanoid robot model to move from the previous position to the current position is the number of animation frames per second set for the mapping motion process in the 3D animation software Blender, is the average speed value at the current position.

[0092] Step 4: Stable control of the humanoid robot

[0093] The key to the motion control of the humanoid robot is to solve the joint torque through inverse dynamics, specifically:

[0094] First, after prioritizing the tasks of the torso, arms, and legs, combine the torso, arm, and leg trajectories and the sole force, and calculate the position, velocity, and acceleration of each level of tasks through null space mapping;

[0095] Secondly, based on the acceleration and plantar force of each level of tasks, after relaxation optimization through the torso dynamics equation of the humanoid robot, the joint torques are obtained through the whole-body dynamics equation of the humanoid robot;

[0096] Finally, taking the joint torques as the feedforward quantity, combining the positions and velocities of each level of tasks, proportional-derivative control is performed on the joints of the robot to obtain the final joint torques, and the state of the humanoid robot at the current moment is used as a known quantity for feedback to the controller. By looping the above process, the reproduction of the mapping action is completed.

[0097] Specifically, step 4.1: After prioritizing the tasks of the torso, arms, and legs, combining the torso, arm, and leg trajectories and the plantar force, through null space mapping, calculate the positions, velocities, and accelerations of each level of tasks;

[0098] To reproduce the human capture action, the task priority order is set as the support leg, torso pose, arm pose, and swing leg:

[0099] Task 1: Support leg task, which provides the plantar force required for the humanoid robot to perform actions and is the basis for realizing the actions of the humanoid robot.

[0100] Task 2: Torso pose task, which is a prerequisite for the humanoid robot's fuselage to remain stable. Therefore, it is ranked second in priority to ensure the stability of the humanoid robot's fuselage.

[0101] Task 3: Arm pose task. Precise control of the arm pose can enable the humanoid robot to present better action effects.

[0102] Task 4: Swing leg task, which has relatively little impact on the stability and action reproduction effect of the humanoid robot. Therefore, the swing leg foot end task is ranked at the lowest priority.

[0103] Combining the torso, arm, and leg trajectory data, according to the set task priority classification, calculate the positions, velocities, and accelerations of each joint on each task in turn through null space mapping. Specifically:

[0104] Torso trajectory Represents the torso pose task, and the corresponding plantar force Represents the support leg task, the leg trajectory represents the swing leg task, and the arm trajectory represents the arm pose task.

[0105] For the two tasks of the humanoid robot: the Jacobian matrices of Task 1 and 2 、 Respectively complete the inverse kinematics solutions of their respective tasks and the generalized joint state quantities (generalized joint space, representing the angular values of the joints of the robotic arm, the leg joints, and the virtual 6-degree-of-freedom joints of the torso), that is 、 。

[0106] Generally it is not a square matrix. To find the pseudo-inverse matrix of is which can be transformed into , is the null space projection matrix. Its function is to map any to the null space of so that it will not affect the task of Task 1.

[0107] Substitute into the of Task 2: , So to simultaneously achieve the generalized space vectors of Task 1 and Task 2 Since is an idempotent Hermitian matrix, we have = Denote the generalized space state quantity for completing Task 1 as ,and denote the generalized space state quantity for simultaneously achieving Task 1 and Task 2 as Let ,then we have 。

[0108] And so on, map the third task to the null space of the second task, we get where , )。

[0109] When the number of tasks is extended to ,we have the task-level null space mapping , where , , 。

[0110] From (differentiating position to get velocity), (differentiating angle to get angular velocity), we get the iterative formula for the position-level null space mapping:

[0111]

[0112] Differentiate to get the iterative formula for the acceleration-level null space mapping:

[0113]

[0114] Through null space mapping, the priority control of robot multi-tasks can be achieved. The iterative formulas for the position, velocity, and acceleration levels of the nth-level task are integrated as follows:

[0115] Position-level iterative formula:

[0116]

[0117] Differentiating to obtain the velocity-level iterative formula:

[0118]

[0119] Differentiating again to obtain the acceleration-level iterative formula:

[0120]

[0121] Through the above iterative solution, the position 、velocity 、acceleration of the nth-level task are obtained.

[0122] Step 4.2: Based on the acceleration of each level of task and the sole force, after relaxation optimization through the trunk dynamics equation of the humanoid robot, the joint torque is obtained through the whole-body dynamics equation of the humanoid robot.

[0123] Denote the in Step 4.1 as . The trunk dynamics equation of the humanoid robot is:

[0124]

[0125] Among them, is the contact Jacobian matrix of the right foot. C represents contact. Since there is only contact force when the leg and foot end are in contact with the ground, R represents the right foot; represents the total contact Jacobian matrix obtained by combining the contact Jacobian matrices of each supporting leg; represents the total contact force vector combined by the contact forces; represents the 6D selection matrix of the trunk. Without , it is the whole-body dynamics equation.

[0126] The two sides of the equation are not equal because the on the left side is obtained through null space iteration, while the sole force on the right side is obtained when exporting the mapped action data from the humanoid robot animation model. Therefore, it is necessary to perform relaxation optimization on the trunk dynamics equation of the humanoid robot to obtain appropriate to satisfy the trunk dynamics equation with the contact external force.

[0127] The relaxation optimization process is as follows:

[0128]

[0129]

[0130]

[0131]

[0132]

[0133]

[0134] Among them, 、 are the relaxation optimization solution quantities that make the plantar force obtained by trajectory optimization and the joint space acceleration obtained by null space iteration satisfy the trunk dynamics equation, and is the constraint quantity of the plantar force.

[0135] After obtaining and the corresponding joint torques can be obtained through the whole-body dynamics equation of the robot :

[0136]

[0137] Among them, is the mass distribution matrix of the humanoid robot, where, is the generalized Coriolis force matrix, is the generalized Coriolis force matrix, is the generalized gravity matrix, , is obtained through null space iteration, is the plantar force of the robotic leg, and are both relaxation optimization solution quantities that satisfy the trunk dynamics equation, represents the Jacobian matrix, represents the joint torque.

[0138] Step 4.3: Use the joint torque as the feedforward quantity, combine the positions and velocities of each level of tasks, perform proportional-derivative control on the joints of the robot to obtain the final torques of each joint of the humanoid robot, and use the state of the humanoid robot at the current moment as a known quantity to feedback to the controller, and cycle the above step process to complete the reproduction of the mapping action.

[0139] Use the obtained joint torque as the feedforward quantity, and combine the positions and velocities of each level of tasks after null space iteration With , perform joint PD (Proportional-Differential) control on the physical joints of the robot, and control the input torque as follows:

[0140]

[0141] Among them, is the torque of the physical joint motor of the robot, which is the joint torque a vector formed after removing the first 6-dimensional torso virtual joint torques, is the proportional coefficient in the proportional-differential control, is the current joint angle value, is the current angular velocity of the joint.

[0142] Calculate the final torques of each joint of the humanoid robot , and use the state of the humanoid robot at the current moment as a known quantity to feedback to the controller, and loop the above steps to complete the reproduction of the mapping action.

[0143] Embodiment 2

[0144] In an embodiment of the present disclosure, a humanoid robot control system based on action mapping is provided, including a model construction module, an action mapping module, a torque solving module, and an action control module:

[0145] The model construction module is configured to: construct a humanoid robot animation model according to the servo distribution of the humanoid robot to be controlled, and perform bone binding;

[0146] The action mapping module is configured to: obtain the human body visual motion capture data of the action to be reproduced, and perform joint-level action mapping on the human body visual motion capture data and the humanoid robot animation model to obtain the torso, arm, and leg trajectories of the mapped humanoid robot animation model, as well as action data such as sole forces;

[0147] The torque solving module is configured to: perform task priority grading on the torso, arms, and legs of the humanoid robot, and combine the smoothed action data to solve the joint torques through inverse dynamics;

[0148] The action control module is configured to: use the joint torque as a feedforward quantity to perform reproduction control on the humanoid robot.

[0149] Embodiment 3

[0150] In an embodiment of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, it implements the above-mentioned humanoid robot control method based on action mapping.

[0151] Embodiment 4

[0152] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the described humanoid robot control method based on action mapping is implemented.

[0153] Embodiment 5

[0154] In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory, so that the electronic device executes and implements the described humanoid robot control method based on action mapping.

[0155] The present disclosure is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, so that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable devices provide steps for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0157] Although the specific implementation manners of the present disclosure have been described above in conjunction with the accompanying drawings, it is not a limitation to the protection scope of the present disclosure. Those skilled in the art should understand that, based on the technical solutions of the present disclosure, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present disclosure.

Claims

1. A humanoid robot control method based on action mapping, characterized in that, Including: Step 1: Construct a humanoid robot animation model according to the servo distribution of the humanoid robot to be controlled, and perform bone binding; Step 2: Obtain the human body visual motion capture data of the action to be reproduced, and perform joint-level action mapping on the human body visual motion capture data and the humanoid robot animation model to obtain the action data of the mapped humanoid robot animation model. Specifically: (1) Adjust the skeleton size of the human body visual motion capture data to match the skeleton size of the humanoid robot animation model. Add an inverse kinematics control bone to each end of the limbs of the skeleton of the humanoid robot animation model and bind it to the ends of the limbs of the skeleton. This inverse kinematics control bone drives the entire arm or leg to move by receiving the action data at the ends of the limbs of the skeleton of the humanoid robot animation model; (2) Establish a child constraint between the inverse kinematics control bone of the skeleton of the humanoid robot animation model and the corresponding bones of the skeleton of the human body visual motion capture data, so that the movement trajectories of the ends of the limbs of the two skeletons are consistent; (3) Use inverse kinematics to calculate the pose of the ends of the limbs of the current humanoid robot animation model, obtain the rotation or movement data of the joints of the robotic arm and robotic leg, and assign the corresponding rotation and movement control bones to the humanoid robot animation model to achieve action reproduction under the 3D animation software. The mapped action data is stored in the key frames of the time series; Import the human body visual motion capture data; the human body visual motion capture data is stored in the Biovision Hierarchy format, which is used to describe the hierarchical relationship of the human body bone structure and actions. The file organizes data in a hierarchical structure, including the bone hierarchical relationship, the rotation angle of the joints, and the time information of the action frames; each joint is defined as a node in the file and is connected through the parent-child relationship of the hierarchical structure; each node contains the name, rotation order, and rotation angle information of the joint; the time information of the action frames determines the moment of each action frame. After importing into the 3D animation software, it forms the visualization of the human body action data; In the inverse kinematics calculation, the detection and processing of the singular point position are carried out. Specifically: The kinematic equation of the end position of the robotic arm is , where is the velocity vector of the end of the robotic arm in the Cartesian coordinate system, belonging to , and m represents the position in three-dimensional space; is the angular displacement vector of each joint of the robotic arm, with a length of n, and n represents the number of degrees of freedom of the robotic arm; contains the rotation and displacement information of each robotic arm segment relative to other joints, and is in the form of: Among them, (i = 1, …, 7) are the joint angles of the robotic arm, is the position component formed by the rotation matrix and displacement matrix of the i-th robotic arm segment relative to the joints of other robotic arms. The attitude kinematic equation of the robotic arm end is as follows: Among them, is the attitude angular velocity of the end of the robotic arm relative to the body coordinate system, is the attitude Jacobian matrix, , is the attitude component formed by the transformation matrix and displacement matrix of the i-th robotic arm segment with respect to the seventh segment; Define the combined variables of position and orientation as , then we have . The Jacobian matrix is defined as . Since is not a square matrix, the generalized inverse of needs to be used for calculation when solving the motion laws of each joint. It is ; When the robotic arm moves near the singular position, the modulus value will approach or reach 0, and the condition number will tend to infinity; these two parameters change continuously when approaching the singular position, so they can be used as warning indicators; The modulus value of the matrix , the condition number , take the reciprocal of the condition number, that is as an indicator; When any one of the indicators is lower than the set threshold, a singular point position warning will be triggered; during the mapping process, when a singular point position warning appears in a certain frame, it can be solved by adding action key frames; create action key frames before and after the problematic frame respectively to retain the action information of the segments before and after the problematic frame. Manually optimize and adjust the pose of the current humanoid robot animation model at the problematic frame to get rid of the singular position, and overwrite the problematic frame with the optimized action key frame, thereby forming a new action frame segment that avoids singular points; among them, the intermediate action frames are supplemented by Bezier interpolation. Any coordinate on the cubic Bezier curve is: Among them, are control points; the arm control bones include the shoulder joint pitch bone, the shoulder joint side swing bone, the upper arm rotation bone, the elbow joint pitch bone, the forearm rotation bone, the wrist joint side swing bone, and the wrist joint pitch bone; When interpolating between two key frames, first, determine them as the starting point and the ending point of the control points, ensure that they are located at both ends of the curve, and are tangent to the adjacent points to ensure smooth transition; then, select two other control points that have an important impact on the curve shape. Their positions directly affect the bending degree and direction of the curve, thereby determining the path and curvature of the curve; After completing the action mapping in 3D animation software, obtain the action data of the humanoid robot animation model after mapping through the action data extraction script; smooth the extracted action data; Step 3: Perform task priority grading on the torso, arms, and legs of the humanoid robot. Combine the smoothed action data and solve for the joint torques through inverse dynamics; Step 4: Use the joint torques as feedforward quantities to reproduce and control the actions of the humanoid robot.

2. A humanoid robot control method based on action mapping according to claim 1, wherein, The construction of the humanoid robot animation model and bone binding are carried out in 3D animation software. According to the servo distribution of the physical humanoid robot, through skinning operations, one-to-one binding of the physical entity and the bones of the humanoid robot animation model is performed, and the parent-child relationship of the bones is set to form a complete skeleton of the humanoid robot animation model.

3. The humanoid robot control method based on action mapping according to claim 1, characterized in that, The task priority grading of the torso, arms, and legs of the humanoid robot is carried out, and the set task priority order is: supporting leg, torso pose, arm pose, swinging leg.

4. The humanoid robot control method based on action mapping according to claim 1, wherein, The solution of the joint torques through inverse dynamics is specifically as follows: Combined with the torso, arm, and leg trajectories, according to the set task priority grading, calculate the position, velocity, and acceleration of each joint on each task through null space mapping in sequence; Based on the position, velocity, and acceleration of each joint, perform relaxation optimization using the torso dynamics equation, and combine the whole-body dynamics equation to obtain the joint torques.

5. A humanoid robot control method based on action mapping according to claim 1, characterized in that, The reproduction control of the actions of the humanoid robot uses the joint torques as feedforward quantities and performs reproduction control of the actions through the proportional-derivative control algorithm.

6. A humanoid robot control system based on action mapping, characterized in that, It includes a model construction module, an action mapping module, a torque solution module, and an action control module: The model construction module is configured to: construct a humanoid robot animation model according to the servo distribution of the humanoid robot to be controlled and perform bone binding; The action mapping module is configured to: obtain the human body visual motion capture data of the action to be reproduced, perform joint-level action mapping on the human body visual motion capture data and the humanoid robot animation model, and obtain the action data of the humanoid robot animation model after mapping. Specifically: (1) Adjust the skeleton size of the human body visual motion capture data to match the skeleton size of the humanoid robot animation model. Add an inverse kinematics control bone to the end of each limb of the humanoid robot animation model skeleton and bind it to the end of the limb of the skeleton. This inverse kinematics control bone drives the entire arm or leg to move by receiving the action data at the end of the limb of the humanoid robot animation model skeleton; (2) Establish a child constraint between the inverse kinematics control bone of the humanoid robot animation model skeleton and the corresponding bones of the human body visual motion capture data skeleton to make the movement trajectories at the ends of the limbs of the two skeletons consistent; (3) Use inverse kinematics to solve the pose of the end of each limb of the current humanoid robot animation model, obtain the rotation or movement data of the manipulator and mechanical leg joints, and assign corresponding rotation and movement control bones to the humanoid robot animation model to achieve action replication under 3D animation software. The mapped action data is stored in the key frames of the time series; Import human body visualization motion capture data; the human body visualization motion capture data is stored in the Biovision Hierarchy format, which is used to describe the hierarchical relationship of the human body bone structure and actions. The file organizes data in a hierarchical structure, including the bone hierarchical relationship, the rotation angles of joints, and the time information of action frames. Each joint is defined as a node in the file and is connected through the parent-child relationship of the hierarchical structure. Each node contains the name, rotation order, and rotation angle information of the joint. The time information of the action frame determines the moment of each action frame. After importing into the 3D animation software, the visualization of human body action data is formed. In the inverse kinematics solution, the detection and processing of the singular point position are carried out specifically as follows: The kinematic equation of the end position of the robotic arm is , where is the velocity vector of the end of the robotic arm in the Cartesian coordinate system and belongs to , and m represents the position in three-dimensional space; is the angular displacement vector of each joint of the robotic arm, with a length of n, and n represents the number of degrees of freedom of the robotic arm; contains the rotation and displacement information of each robotic arm segment relative to other joints, and is in the form of: Among them, (i = 1, …, 7) are the joint angles of the robotic arm, is the position component formed by the rotation matrix and displacement matrix of the i-th robotic arm segment relative to the joints of other robotic arms. The pose kinematic equation of the end of the robotic arm is: Among them, is the attitude angular velocity of the end of the robotic arm relative to the body coordinate system, is the attitude Jacobian matrix, , is the attitude component formed by the transformation matrix and displacement matrix of the i-th robotic arm section with respect to the seventh section; Define the position and attitude joint variables as , then there is , and the Jacobian matrix is defined as . Since is not a square matrix, the generalized inverse of needs to be used when solving the motion laws of each joint. For , the calculation is ; When the robotic arm moves near the singular position, the modulus value will approach or reach 0, while the condition number will tend to infinity; these two parameters change continuously when approaching the singular position, so they can be used as warning indicators; The modulus value of the matrix , the condition number , take the reciprocal of the condition number, that is as an indicator; When any one of the indicators is lower than the set threshold, a warning for the singular point position will be triggered; during the mapping process, when a warning for the singular point position appears in a certain frame, it can be solved by adding key action frames. Create key action frames before and after the problematic frame respectively to retain the action information of the segments before and after the problematic frame. Manually optimize and adjust the posture of the current humanoid robot animation model at the problematic frame to make it get out of the singular position, and use the optimized key action frames to overwrite the problematic frame, thus forming a new action frame segment that avoids singular points. Among them, the intermediate action frames are supplemented by Bezier interpolation. Any coordinate on the cubic Bezier curve is: Among them, are control points; the arm control bones include the shoulder joint pitch bone, the shoulder joint side swing bone, the upper arm rotation bone, the elbow joint pitch bone, the forearm rotation bone, the wrist joint side swing bone, and the wrist joint pitch bone; When interpolating between two key frames, first, determine them as the starting point and the ending point of the control points, ensure that they are located at both ends of the curve, and are tangent to the connecting lines of adjacent points to ensure smooth transition; then, select two other control points that have an important impact on the curve shape. Their positions directly affect the bending degree and direction of the curve, thus determining the path and curvature of the curve. After completing the action mapping in the 3D animation software, obtain the action data of the humanoid robot animation model after mapping through the action data extraction script; smooth the extracted action data. The torque solving module is configured to: classify the task priorities of the torso, arms, and legs of the humanoid robot, and combine the smoothed action data to solve the joint torque through inverse dynamics. The action control module is configured to: use the joint torque as a feedforward quantity to perform the reproduction control of the actions of the humanoid robot.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements a humanoid robot control method according to any one of claims 1-5.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, it implements a humanoid robot control method according to any one of claims 1-5.

9. An electronic device, characterized in that, Including: A processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes and implements a humanoid robot control method according to any one of claims 1-5.

Images