Electric control system, method and device for humanoid robot

Abstract

The present invention provides an electric control system for a humanoid robot, wherein the humanoid robot includes a robot body and a drive mechanism for driving the motion of the robot, and the electric control system includes a positioning fusion module, an understanding and decision-making module, a motion control module, and a joint drive module. Through a heterogeneous multi-core processing method and a private 5G network, efficient multi-task data processing between modules is realized, high-performance computing and analysis capabilities with low power consumption are realized, reliable execution of non-real-time scene understanding and real-time motion control tasks is realized, and it is ensured that the robot body can work efficiently in multiple scenes with low computing power and low power consumption, reducing costs and system complexity. The present invention further provides an electric control system method and device for a humanoid robot. The device provided by the present invention is used to achieve high-reliability and high-real-time motion control in multiple scenes of a humanoid robot.

Description

Technical Field

[0001] The present invention belongs to the field of robot control technology, and particularly relates to an electric control system, method, and apparatus for a humanoid robot.

Background Art

[0002] Currently, industrial robots play a major role in various fields. With the continuous development of computer technology and the continuous progress of artificial intelligence, robots are gradually penetrating from the industrial field into fields such as service, entertainment, and education. Due to the high intelligent characteristics of humanoid robots, there is a large potential market in fields such as service and entertainment. Currently, the research and development of humanoid robots are actively carried out in various countries, and many research institutions and enterprises at home and abroad have launched related research products. In a few years, humanoid robots will be applied in various fields of human life.

[0003] The working environment of humanoid robots has characteristics such as multi-scene, diversification, dynamics, uncertainty, and complexity. Higher requirements are imposed on the performance of robots in terms of scene understanding, positioning accuracy, interaction function, and stability. To meet these requirements, humanoid robots need to be equipped with various sensing sensors such as positioning, vision, voice, and touch, and also need high real-time motion control performance to complete more complex tasks. Therefore, to achieve stable and highly reliable control of robots, it is necessary to develop a robot electric control system with high intelligence, powerful computing power, excellent stability, high-speed response, powerful real-time performance, and high data bandwidth. Currently, most humanoid robots use a general-purpose CPU architecture to create the entire robot's electric system, but they cannot handle large-scale computing tasks related to sensing sensors such as vision, voice, positioning, and navigation. Furthermore, most electric control systems combine the non-real-time tasks and real-time tasks of robots into one system, so they cannot effectively meet the high real-time control requirements of robots.

[0004] Patent Document CN110666820A discloses a high-performance industrial robot controller including a processor board, a machine vision unit, a teaching pendant, an external IO unit, a gripper unit, an external sensor, and a motor driver. The processor board includes an ARM processor unit, an FPGA unit, a power module, an Ethernet interface A, an Ethernet interface B, an Ethernet interface C, an Ethernet interface D, an IO interface, and a CAN interface. The system uses the FPGA unit to process all sensor data, but to complete the data processing problems of sensing sensors such as vision, voice, positioning, and navigation, it is necessary to create a large-scale and complex system with an independent FPGA unit.

Summary of the Invention

Problems to be Solved by the Invention

[0005] Patent Document CN115599024A discloses a high-precision turntable control system based on the EtherCAT bus, including a high-performance motion controller, a high-precision servo driver, a touch screen, a filter, and a control power supply. The high-performance motion controller is communicatively connected to a plurality of high-precision servo drivers via the EtherCAT bus. The high-precision servo driver is connected to a filter. The touch screen is connected to the high-performance motion controller. The control power supply is connected to the high-performance motion controller and a plurality of high-precision servo drivers respectively. The device connects a plurality of servo drivers via the EtherCAT bus, but this specification does not propose specific implementation means on how to introduce this method into the control system of a humanoid robot.

Means for Solving the Problems

[0006] The object of the present invention is to provide an electric control system, method and device for a humanoid robot for realizing multi-scene, high-reliability and high-real-time motion control of the humanoid robot.

[0007] To achieve the first object, the present invention provides an electric control system for a humanoid robot, wherein the humanoid robot includes a robot body and a drive mechanism for driving the motion of the robot, and the electric control system includes a positioning fusion module, an understanding and decision-making module, a motion control module, and a joint drive module. The positioning fusion module is used to obtain the direction and coordinate information of the robot body and generate positioning information.

[0008] The understanding and decision-making module obtains the environmental data of the scene where the robot body is located, uploads it to the cloud for decision-making, generates an interaction command and an operation command, and in the robot body, based on the interaction command, completes the interactive operation of the robot body, and is used to generate a motion command based on the operation command and the positioning information.

[0009] The motion control module includes a control and calculation unit and an EtherCAT master station. Based on the generated motion command and the real-time status data of each joint, the control and calculation unit performs kinematic and dynamic analysis of the robot body through the control and calculation unit to obtain the joint commands of each joint. The EtherCAT master station obtains the real-time status data of each joint and is used to transmit the joint commands to the corresponding joint drive module.

[0010] The joint drive module includes a drive unit and an EtherCAT slave station equipped with a plurality of status sensors. The drive unit receives joint commands from the EtherCAT master station, adjusts the output of the drive mechanism, and at the same time the EtherCAT slave station transmits the real-time status data collected by the plurality of status sensors to the EtherCAT master station.

[0011] Preferably, through data fusion and cross-data comparison, the direction and coordinate information of multiple sensors are processed to generate the positioning information of the robot, and through data fusion and cross-data comparison, the errors between sensors are eliminated, and more accurate position information can be obtained.

[0012] Specifically, the understanding and decision-making module includes a vision unit and an audio unit for acquiring environmental data, a cloud decision-making unit for outputting commands, and a perception planning unit. The vision unit is used to acquire the image information of the external environment and obstacles in the scene where the robot body is located. The audio unit is used to acquire the audio information of the external environment. The cloud decision-making unit generates corresponding interaction commands and operation commands based on the issued task commands and the received environmental data. The perception planning unit is used for data uploading and data analysis. The data uploading includes compressing the image information and audio information as environmental data and transmitting it to the cloud decision-making unit. The data analysis includes generating motion commands based on the operation commands and positioning information, and generating interactive actions of the humanoid robot based on the interaction commands.

[0013] Specifically, the cloud decision-making unit includes a debugging terminal and an understanding and decision-making terminal. The debugging terminal is used to obtain the real-time status data of each joint of the EtherCAT master station in order to obtain the operation status of the robot body. The understanding and decision-making terminal is used to analyze the environmental data and operation status together according to the issued task instructions, and obtain the corresponding interaction instructions and operation instructions. The task instructions are issued by the operator or automatically assigned according to the work plan.

[0014] Specifically, the operation instructions include a target location, a moving speed, and an operation posture.

[0015] Specifically, the interactive operation includes voice output control and video display of the humanoid robot.

[0016] Preferably, the electric control system uses a private 5G network to perform data transmission between the understanding and decision-making module and the functional units of the motion control module, ensuring that the heterogeneous multi-core processing unit and the cloud decision-making unit can cooperate to process the multi-tasks of the robot, realizing high-performance computing and analysis capabilities with low power consumption and low cost of the robot body. Thereby, the robot can operate efficiently even in a complex environment and enable collaborative work between multiple robots.

[0017] To achieve the second object, the present invention provides an electric control method for a humanoid robot realized by the above-mentioned electric control system, including the following steps.

[0018] Step 1: Turn on the power of the robot. After successfully turning on the power, the status sensor collects information and uploads it to the debugging terminal of the cloud decision-making unit to feedback the operation status of the entire humanoid robot.

[0019] Step 2: Obtain the environmental data of the scene where the humanoid robot is located through the voice unit and the visual unit, and compress and upload the environmental data through the perception planning unit.

[0020] Step 3: The understanding and decision-making terminal of the cloud decision-making unit performs task analysis on the received task instructions, and at the same time performs scene understanding algorithm processing on the environmental data. According to the task analysis results, scene understanding results, and the overall operation status of the humanoid robot, corresponding interaction instructions and operation instructions are generated.

[0021] Step 4: Obtain the positioning information of the humanoid robot through the positioning fusion module and transmit it to the perception planning unit.

[0022] Step 5: Based on the interaction instructions, the perception planning unit controls the voice output and video display of the humanoid robot.

[0023] At the same time, based on the positioning information and operation instructions, the perception planning unit performs path planning to generate corresponding motion instructions.

[0024] Step 6: The control and calculation unit performs dynamics and kinematics calculations in combination with the motion instructions and the real-time status data of each joint, obtains the corresponding joint instructions, and transmits them to the joint drive module through the EtherCAT master station.

[0025] Step 7: The drive module adjusts the output of the drive mechanism according to the received joint instructions to enable the humanoid robot to execute the task. At the same time, the EtherCAT slave station transmits the real-time status data collected by multiple status sensors to the EtherCAT master station.

[0026] Specifically, the electric control method includes the self-detection process of the humanoid robot. When the debug terminal evaluates based on the operation status of the humanoid robot and determines that it cannot operate normally, it transmits a maintenance prompt. When it determines that it can operate normally, it transmits a prompt indicating that the task can be executed.

[0027] To achieve the third objective, the present invention provides an electric control device for a humanoid robot, which includes a computer memory, a sensor, a computer processor, and a program stored in the computer memory and executable on the computer processor. The computer processor adopts the above-mentioned electric control system for the humanoid robot.

[0028] When the computer processor executes the computer program, the following steps are realized. According to the input task instruction, through the electric control system, corresponding interaction instructions and operation instructions are generated in real time to realize the task execution and human-computer interaction of the humanoid robot.

Advantages of the Invention

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows.

[0030] The electric control system is divided into four parts: a positioning fusion module, an understanding and decision-making module, a motion control module, and a joint drive module. Through the heterogeneous multi-core processing method and the private 5G network, efficient cooperation between each module is realized, and high-reliability and real-time motion control of the humanoid robot in multiple scenarios are realized.

Brief Description of the Drawings

[0031]

Figure 1

Figure 2

Figure 3

Figure 4

DETAILED DESCRIPTION OF THE INVENTION

[0032] Hereinafter, the present invention will be described in detail based on the accompanying drawings and preferred embodiments, so that the objectives and effects of the present invention will be further clarified. It should be understood that the specific embodiments described here are only used to explain the present invention and are not intended to limit the present invention.

[0033] An electric control system for a humanoid robot, the humanoid robot comprising a robot body and a drive mechanism for driving the motion of the robot.

[0034] As shown in FIG. 1, the electric control system includes a positioning fusion module, an understanding and decision-making module, a motion control module, and a joint drive module.

[0035] The positioning fusion module is used to obtain the position information of the robot, and obtains the direction and position information of the robot through various positioning sensors such as UWB, lidar, IMU, odometer, and GPS. Data fusion is performed using an FPGA processor, and errors are eliminated by comparing cross-data information to obtain accurate positioning information.

[0036] The understanding and decision-making module includes a vision unit, an audio unit, a cloud decision-making unit, and a perception planning unit.

[0037] The visual unit is used to acquire the image information of the external environment and objects of the robot, and includes a depth camera and a display screen. The depth camera is used to identify objects, people, etc. within the environment, and the display screen can output interaction videos.

[0038] The audio unit is used to acquire audio information from the external environment, and includes a microphone array and a speaker device. The microphone array is used to collect and process voice commands, and the speaker is used for voice output to realize the human-computer interaction effect.

[0039] The cloud decision-making unit generates corresponding interaction commands and operation commands based on the issued task commands and the received environmental data.

[0040] The perception planning unit is used for data uploading and data analysis. Data uploading includes compressing image information and audio information as environmental data and transmitting it to the cloud decision-making device. Data analysis includes generating operation commands based on operation commands and positioning information, and generating interactive actions of the humanoid robot based on interaction commands.

[0041] Furthermore, the cloud decision-making unit includes a debug terminal and an understanding and decision-making terminal. The debug terminal is used to obtain the real-time status data of each joint of the EtherCAT master station to acquire the operation status of the robot body. The understanding and decision-making terminal is used to analyze the received task commands together with the environmental data and operation status, and obtain the corresponding interaction commands and operation commands. The task commands are issued by the operator or automatically assigned according to the work plan.

[0042] When the cloud decision-making unit includes powerful computing capabilities, provides computing support to multiple robots simultaneously, promotes the cooperation of multiple robots, and transfers the analysis work of complex task instructions to the cloud decision-making unit, the computing stress of the humanoid robot itself can be reduced.

[0043] The motion control module includes a control and calculation unit and an EtherCAT master station. Based on the generated motion instructions and the real-time status data of each joint, it performs kinematic and dynamic analysis of the robot body through the control and calculation unit to obtain joint instructions for each joint. The EtherCAT master station is used to obtain the real-time status data of each joint and transmit the joint instructions to the corresponding joint drive module.

[0044] The joint drive module includes a drive unit and an EtherCAT slave station equipped with multiple status sensors. The drive unit receives joint instructions from the EtherCAT master station, adjusts the output of the drive mechanism, and at the same time the EtherCAT slave station transmits the real-time status data collected by the multiple status sensors to the EtherCAT master station.

[0045] Furthermore, the understanding and decision-making module and the motion control module transmit data through a high-bandwidth and high-security private 5G network to achieve low-latency interconnection, ensuring that the heterogeneous multi-core processing unit and the cloud decision-making unit can cooperate to process the multi-tasks of the robot, realizing high-performance computing and analysis capabilities with low power consumption and low cost of the robot body, enabling the robot to operate efficiently in complex environments, and enabling collaborative work among multiple robots.

[0046] As shown in Figure 2, it is an electric control method for a humanoid robot provided by this embodiment, which is realized by the electric control system described in the above embodiment and includes the following steps.

[0047] Step 1: Turn on the power of the robot. After successfully turning on the power, the status sensor collects information and uploads it to the debug terminal of the cloud decision-making unit to feedback the operation status of the whole humanoid robot. At the same time, the debug terminal evaluates based on the operation status of the whole humanoid robot. If it is determined that it cannot operate normally, it transmits a maintenance prompt. If it is determined that it can operate normally, it transmits a prompt indicating that the task can be executed.

[0048] Step 2: Obtain the environmental data of the scene where the humanoid robot is located through the voice unit and the vision unit, and compress and upload the environmental data through the perception planning unit.

[0049] Step 3: The understanding and decision-making terminal of the cloud decision-making unit performs task analysis on the received task instruction, and at the same time performs scene understanding algorithm processing on the environmental data. According to the task analysis result, the scene understanding result and the operation status of the whole humanoid robot, corresponding interaction instructions and operation instructions are generated.

[0050] Step 4: Obtain the positioning information of the humanoid robot through the positioning fusion module and transmit it to the perception planning unit.

[0051] Step 5: The perception planning unit performs voice output control and video display of the humanoid robot based on the interaction instruction.

[0052] At the same time, the perception planning unit generates corresponding motion instructions based on the positioning information and the operation instructions.

[0053] Step 6: The control and calculation unit performs dynamics and kinematics calculations in combination with the motion instructions and the real-time status data of each joint, obtains the corresponding joint instructions, and transmits them to the joint drive module through the EtherCAT master station.

[0054] Step 7: The drive unit adjusts the output of the drive mechanism according to the received joint commands to enable the humanoid robot to execute tasks. At the same time, the EtherCAT slave station transmits the real-time status data collected by multiple status sensors to the EtherCAT master station.

[0055] As shown in Figure 3, an electric control device for a humanoid robot is provided according to this embodiment, which includes a communication device, a data calculation unit equipped with an FPGA, a sensing and planning unit equipped with an ARM, a control and calculation unit equipped with an X86, sensors, and a drive device.

[0056] The sensors include a GPS module, an IMU sensor, a UWB sensor, an odometer, and a lidar used in combination with the data calculation unit equipped with an FPGA, thereby obtaining the status information and position information of the robot. a microphone array, a speaker, an audio capture card, an audio amplifier, a depth vision camera, and a face display used in combination with the sensing and planning unit equipped with an ARM. It further includes status sensors used in combination with the control and calculation unit equipped with an X86.

[0057] The communication device includes a private 5G network and an EtherCAT network.

[0058] The drive device includes a battery for power supply and a BMS system for managing the power supply of the battery.

[0059] As shown in Figure 4, it is the EtherCAT conversion circuit logic, and the specific process is as follows.

[0060] Obtain the original data of each sensor according to the specified sampling order.

[0061] Intercept the original data and convert it into data with a unified structure.

[0062] After transmitting data with a unified structure to an EtherCAT network via an EtherCAT slave station, it is received and processed by an EtherCAT master station.

[0063] According to the above embodiments, the present invention performs multi-task data processing through the cooperation of different functional units, realizes high-performance computing and analysis capabilities under low power consumption, realizes non-real-time scene understanding and reliable execution of real-time motion control tasks, enables the robot body to operate efficiently in various environments under low computing power requirements, and reduces costs and system complexity. A data processing unit with low latency and high concurrency is used to realize the collection and fusion of position and orientation data, and a low-power consumption sensing and planning unit is used to realize the compression of audio and visual data, the execution of human-computer interaction, the realization of path and obstacle avoidance planning algorithms, and the output of motion commands. The powerful computing resources of the cloud decision-making unit are used to process audio and visual information, realize the semantic understanding of complex and diverse scene environments, and at the same time analyze the task commands issued by the debug terminal and output interaction commands. High-performance and highly compatible control and computing units are utilized to calculate motion control algorithms and realize an EtherCAT master-master station network, thus realizing a stable real-time network.

[0064] The above are only preferred embodiments of the present invention and do not limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can understand that it is possible to modify the technical solutions recorded in the foregoing examples or equivalently replace some of the technical features. All modifications, equivalent replacements, etc. made within the spirit and principle scope of the present invention shall be included in the protection scope of the present invention.

Claims

1. An electric control system for a humanoid robot, wherein the humanoid robot comprises a robot body and a drive mechanism for driving the motion of the robot, a positioning fusion module for acquiring the direction and coordinate information of the robot body and generating positioning information, an understanding and decision-making module for acquiring the environmental data of the scene where the robot body is located, uploading it to the cloud for decision-making, generating interaction instructions and motion instructions, and completing the interactive motion of the robot body based on the interaction instructions at the robot body, and generating motion instructions based on the motion instructions and positioning information, including a control and calculation unit and an EtherCAT master station, performing kinematic and dynamic analysis of the robot body through the control and calculation unit based on the motion instructions and the real-time status data of each joint to obtain joint instructions for each joint, and the EtherCAT master station is a motion control module for acquiring the real-time status data of each joint and transmitting the joint instructions to the corresponding joint drive module, including a drive unit and an EtherCAT slave station equipped with a plurality of status sensors, the drive unit receives joint instructions from the EtherCAT master station and adjusts the output of the drive mechanism, and at the same time the EtherCAT slave station is a joint drive module for transmitting the real-time status data collected by the plurality of status sensors to the EtherCAT master station. An electric control system for a humanoid robot, characterized in that.

2. The electric control system for a humanoid robot according to claim 1, characterized in that the direction and coordinate information of a plurality of sensors are processed through data fusion and cross-data comparison to generate the positioning information of the robot.

3. The understanding and decision-making module includes a visual unit and an audio unit for acquiring environmental data, a cloud decision-making unit for outputting instructions, and a perception planning unit, the visual unit is used to acquire the image information of the external environment and obstacles in the scene where the robot body is located, the audio unit is used to acquire the audio information of the external environment, The cloud decision-making unit generates corresponding interaction instructions and operation instructions based on the issued task instructions and the received environmental data. The sensing and planning unit is used for data uploading and data analysis. The data uploading includes compressing image information and voice information as environmental data and transmitting them to the cloud decision-making unit. The data analysis generates motion instructions based on the operation instructions and positioning information, and generates interactive actions of the humanoid robot based on the interaction instructions. The electric control system for a humanoid robot according to claim 1, characterized in that.

4. The cloud decision-making unit includes a debug terminal and an understanding and decision-making terminal. The debug terminal is used to obtain the real-time status data of each joint of the EtherCAT master station to obtain the operation status of the robot body. The understanding and decision-making terminal analyzes in combination with the environmental data and operation status according to the issued task instructions, and obtains corresponding interaction instructions and operation instructions. The electric control system for a humanoid robot according to claim 1 or 3, characterized in that.

5. The operation instructions include a target location, a moving speed, and an operation posture. The electric control system for a humanoid robot according to claim 1 or 3, characterized in that.

6. The interactive actions include voice output control and video display of the humanoid robot. The electric control system for a humanoid robot according to claim 1 or 3, characterized in that.

7. The electric control system uses a private 5G network to perform data transmission between the functional units of the understanding and decision-making module and the motion control module. The electric control system for a humanoid robot according to claim 1, characterized in that.

8. An electric control method for a humanoid robot, which is realized by the electric control system for a humanoid robot according to any one of claims 1 to 7. Step 1: Turn on the power of the robot. After successfully turning on the power, the status sensor collects information and uploads it to the debug terminal of the cloud decision-making unit to feedback the operation status of the entire humanoid robot. Step 2: Obtain the environmental data of the scene where the humanoid robot is located through the voice unit and the visual unit, and compress and upload the environmental data through the perception planning unit; Step 3: The understanding and decision-making terminal of the cloud decision-making unit performs task analysis on the received task command, and at the same time performs scene understanding algorithm processing on the environmental data. According to the task analysis result, the scene understanding result and the overall operation status of the humanoid robot, generate corresponding interaction commands and operation commands; Step 4: Obtain the positioning information of the humanoid robot through the positioning fusion module and transmit it to the perception planning unit; Step 5: The perception planning unit performs voice output control and video display of the humanoid robot based on the interaction command; At the same time, the perception planning unit performs path planning based on the positioning information and the operation command to generate corresponding motion commands; Step 6: The control and calculation unit performs dynamics and kinematics calculations together with the motion command and the real-time status data of each joint, obtains the joint commands of the corresponding joints, and transmits them to the joint drive module through the EtherCAT master station; Step 7: The drive unit adjusts the output of the drive mechanism according to the received joint command to enable the humanoid robot to execute the task. At the same time, the EtherCAT slave station transmits the real-time status data collected by a plurality of status sensors to the EtherCAT master station. An electric control method for a humanoid robot, characterized in that it includes the above steps.

9. The electric control method includes the self-detection process of the humanoid robot. The debug terminal evaluates based on the operation status of the humanoid robot. If it is determined that it cannot operate normally, it transmits a maintenance prompt. If it is determined that it can operate normally, it transmits a prompt indicating that the task can be executed. The electric control method for a humanoid robot according to claim 8, characterized in that it is as described above.

10. An electric control device for a humanoid robot, including a computer memory, a sensor, a computer processor, and a program stored in the computer memory and executable on the computer processor, wherein the computer processor adopts the electric control system for a humanoid robot according to any one of claims 1 to 7, When the computer processor executes the computer program, according to the input task instruction, through the electric control system, corresponding interaction instructions and operation instructions are generated in real time, and the steps of realizing the task execution of the humanoid robot and the human-computer interaction are performed. An electric control device for a humanoid robot, characterized by this.

Images