Humanoid robot action training method based on wearable limb action recognition

Abstract

The invention discloses a humanoid robot action training method based on wearable limb action recognition, and relates to the technical field of artificial intelligence. A whole body dynamic capturing device based on an MEMS is utilized, an action training engineer wears the whole body dynamic capturing device based on the MEMS, wireless connection with a humanoid robot is established, and all action data of the action training engineer are obtained; all the captured action data are transmitted to a joint control system lower computer of the humanoid robot in real time, and the joint rotation action data drive all joints to move; and the joint control system stores the action data to an action database, and performs action data training by using a large language model (LLM) to obtain a motor nerve center system of the humanoid robot. According to the method, the action data acquisition efficiency is improved, the acquisition cost is reduced, a massive action database better conforming to ergonomics is obtained, a humanoid robot movement nerve center system is trained in combination with a large model, and various subdivision application scene databases are formed. And the humanoid robot can serve human beings in various fields earlier and faster.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a humanoid robot motion training method based on wearable limb motion recognition. Background Art

[0002] Current methods for training motion of artificial intelligence humanoid robots include simulation in a virtual environment and engineers manually bending robot joints to complete motions while accumulating a robot motion library to complete robot motion training.

[0003] For example, simulation in a virtual environment (sim2real) is a cost-effective training method that conducts a lot of trials and error corrections in a simulated virtual environment and then migrates these results to robots in the real world. The advantage of this method is that it can greatly reduce the risks and costs in actual operations. However, its disadvantages are also obvious, that is, there is a deviation between the simulation data and the real environment, which may lead to challenges in the migration process, and the training process requires a huge amount of computing power resources to complete the simulation calculations.

[0004] The method in which engineers manually bend robot joints to complete movements while accumulating a robot motion library is a traditional and long-standing method. This method is extremely inefficient in obtaining motion data, and the time and labor costs of obtaining motion data are extremely high. The data obtained is also far from the movement habits of human kinematics and ergonomics, which can also cause the robot to move sluggishly and be unable to complete complex tasks and stress responses.

[0005] Therefore, it is an urgent problem for technical personnel in this field to propose a humanoid robot motion training method based on wearable limb motion recognition to overcome the difficulties existing in the prior art. Summary of the invention

[0006] In view of this, the present invention provides a humanoid robot motion training method based on wearable limb motion recognition, which improves the efficiency of motion data acquisition, reduces the acquisition cost, obtains a more ergonomic massive motion database, trains the humanoid robot motor nerve center system in combination with a large model, and forms a database of various subdivided application scenarios, so that humanoid robots can serve humans in various fields earlier and faster.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A humanoid robot motion training method based on wearable limb motion recognition, using a full-body motion capture device based on MEMS, includes the following steps:

[0009] S1. Motion training engineers wear MEMS-based full-body motion capture equipment;

[0010] S2, establish wireless connection between the MEMS-based full-body motion capture device and the humanoid robot;

[0011] S3, the motion training engineer performs exercise, and the MEMS-based full-body dynamic capture device obtains all the motion data of the motion training engineer;

[0012] S4, the whole body dynamic capture device based on MEMS transmits all the captured motion data to the joint control system lower computer of the humanoid robot in real time;

[0013] S5, the lower computer drives each joint to move according to the corresponding joint rotation action data;

[0014] S6. The joint control system stores the motion data in the motion database, and uses the large language model LLM to train the motion data to obtain the motor nerve center system of the humanoid robot.

[0015] In the above method, optionally, the wireless connection mode in S2 includes: WiFi, Bluetooth or other wireless connection modes with its own protocol stack.

[0016] In the above method, optionally, the action training engineer in S3 performs random actions or specific actions as required.

[0017] In the above method, optionally, the MEMS-based full-body motion capture device in S1 includes: a gesture recognition data glove and a wearable full-body motion capture device.

[0018] In the above method, optionally, the action data in S6 is stored in the action database of the robot operating system ROS in the form of Action message communication.

[0019] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a humanoid robot motion training method based on wearable limb motion recognition, which has the following beneficial effects:

[0020] 1) Use full-body dynamic capture equipment based on MEMS gesture sensors to obtain real-time motion data of real people. This data is completely in line with human kinematics and ergonomics, with extremely high data accuracy, extremely high data generation efficiency, and extremely low data acquisition cost. A huge motion database can be formed in a short period of time, and human stress response data can also be obtained very directly;

[0021] 2) The motion data of all joints in the whole body can be obtained by wearing a wearable limb motion recognition device based on MEMS posture sensing, which can complete the full range of motion data in any location and any scene, indoors and outdoors. The data obtained is real and fully in line with the ergonomics of humanoid robots, in line with the real humanoid robot training requirements, and has significant advantages in obtaining the whole body motion coordination data of humans under stress;

[0022] 3) The motion database is easy to accumulate, and the efficiency of motion training can be greatly improved by classifying and labeling the accumulated massive motion data and using a large model for training. In this way, the motor nerve center system of the humanoid robot can be obtained. This method can achieve data barriers in all application fields and various subdivided scenarios, and operate efficiently in a standardized process. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0024] Figure 1 A flow chart of a humanoid robot motion training method based on wearable limb motion recognition disclosed in the present invention;

[0025] Figure 2 A logic diagram of a humanoid robot motion training method based on wearable limb motion recognition disclosed by the present invention;

[0026] Figure 3 This is a schematic diagram of connecting the full-body dynamic capture disclosed in the present invention with a humanoid robot and realizing real-time control. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0029] In the existing technology, robot training includes simulation in a virtual environment and engineers manually bending robot joints to complete movements while accumulating a robot motion library to complete robot motion training. The core goal of the first two methods of training is to generate motion data through virtual simulation software and to use engineers to manually bend robot joints to complete specific movements. The human and time costs of generating movements in this process are very high and the efficiency is extremely low. The generated motion data is far from the real data of ergonomics and kinematics, which is one of the main reasons why humanoid robots at this stage seem to be standardized, stuck, and sluggish.

[0030] The present invention adopts a set of motion data acquisition and training methods that are completely different from the existing methods: its core innovation is to use a new full-body dynamic capture device based on MEMS posture sensors to obtain real-time motion data of real people. These data are completely in line with human kinematics and ergonomics. Because the motion data collection is completed by real wearable devices, the data accuracy is extremely high, the data generation efficiency is extremely high, and the data acquisition cost is extremely low. A huge motion database can be formed in a short period of time, and human stress response data can also be directly obtained, such as: the reaction data of various joints of the body when a person falls, the stress response data of a person in the process of fighting, or the whole body stress action data when encountering a series of dangerous situations. The above-mentioned data cannot be obtained by the above two methods.

[0031] The specific technical solutions are as follows:

[0032] See also Figure 1 and Figure 2 As shown, the present invention discloses a humanoid robot motion training method based on wearable limb motion recognition, using a full-body dynamic capture device based on MEMS, comprising the following steps:

[0033] S1. Motion training engineers wear MEMS-based full-body motion capture equipment;

[0034] S2, establish wireless connection between the MEMS-based full-body motion capture device and the humanoid robot;

[0035] S3, the motion training engineer performs exercise, and the MEMS-based full-body dynamic capture device obtains all the motion data of the motion training engineer;

[0036] S4, the whole body dynamic capture device based on MEMS transmits all the captured motion data to the joint control system lower computer of the humanoid robot in real time;

[0037] S5, the lower computer drives each joint to move according to the corresponding joint rotation action data;

[0038] S6. The joint control system stores the motion data in the motion database, and uses the large language model LLM (Large Language Model) to train the motion data to obtain the motor nerve center system of the humanoid robot.

[0039] Furthermore, the wireless connection method in S2 includes: WiFi, Bluetooth or other wireless connection methods with its own protocol stack.

[0040] Furthermore, the action training engineers in S3 perform random actions or make specific actions as required.

[0041] Furthermore, the MEMS-based full-body motion capture devices in S1 include: gesture recognition data gloves and wearable full-body motion capture devices. The above devices are easy to wear, can be operated by one person, are resistant to electromagnetic interference and metal interference, are not affected by the strength of the light field, and can be used both indoors and outdoors. Among them, the gesture recognition data gloves are used to capture the motion data of all knuckles of the hand, and the wearable full-body motion capture device is used to capture the data of all joints of the body.

[0042] In a specific embodiment, the gesture recognition data gloves and the wearable full-body motion capture device both use the data gloves and full-body motion capture system of Wuhan Haocun Technology Co., Ltd.

[0043] Furthermore, the action data in S6 is stored in the action database of the robot operating system ROS (Robot Operating System) in the form of Action message communication.

[0044] Some robot virtual simulation systems store joint motion data in fbx format.

[0045] Large Language Model (LLM) is a general term for all large models that can perform deep learning, understanding, processing and reasoning on data, including pictures, text, number series, videos, action data, etc.

[0046] In a specific embodiment, see Figure 3 As shown, after the wearable full-body motion capture device is wirelessly connected to the humanoid robot, the motion training engineer makes any action or specified action (for example: when performing a Tai Chi, the humanoid robot will also follow the motion training engineer to perform a Tai Chi in real time), and the humanoid robot will also follow the trainer to make the action in real time. And this action will be stored in the action database while controlling the humanoid robot in real time. After the action data in the action database is calibrated, the large model can be used for action training to achieve the purpose of training the real humanoid robot's motor nerve center (cerebellum). Ultimately, this system will also become the motion control system of the humanoid robot.

[0047] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0048] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A humanoid robot motion training method based on wearable limb motion recognition, characterized in that: Using a MEMS-based full-body dynamic capture device, the following steps are included: S1. Motion training engineers wear MEMS-based full-body motion capture equipment; S2, establish wireless connection between the MEMS-based full-body motion capture device and the humanoid robot; S3, the motion training engineer performs exercise, and the MEMS-based full-body dynamic capture device obtains all the motion data of the motion training engineer; S4, the whole body dynamic capture device based on MEMS transmits all the captured motion data to the joint control system lower computer of the humanoid robot in real time; S5, the lower computer drives each joint to move according to the corresponding joint rotation action data; S6. The joint control system stores the motion data in the motion database, and uses the large language model LLM to train the motion data to obtain the motor nerve center system of the humanoid robot.

2. A humanoid robot motion training method based on wearable limb motion recognition according to claim 1, characterized in that: The wireless connection methods in S2 include: WiFi, Bluetooth or other wireless connection methods with their own protocol stack.

3. A humanoid robot motion training method based on wearable limb motion recognition according to claim 1, characterized in that: The actions in S3 train engineers to perform random actions or specific actions on demand.

4. The humanoid robot motion training method based on wearable limb motion recognition according to claim 1 is characterized in that: The MEMS-based full-body motion capture devices in S1 include: gesture recognition data gloves and wearable full-body motion capture devices.

5. The humanoid robot motion training method based on wearable limb motion recognition according to claim 1 is characterized in that: The action data in S6 is stored in the action database of the robot operating system ROS in the form of Action message communication.

Images