Robot control method, system and electronic device based on general command

Abstract

The application discloses a robot control method based on a general command, and has the characteristics that the method comprises the following steps: S1: obtaining a target general command; S2: performing an executability check on the target general command based on the state of the robot; if the executability check is passed, executing S3; if the executability check is not passed, converting the target general command into a SPEAK command and playing a message that cannot be executed; S3: obtaining a target code to be filled based on the action code of the target general command; S4: filling the semantic information of the target general command into the target code to be filled to obtain a filled target code; S5: performing format conversion on the filled target code to obtain a target code suitable for the robot; and S6: calling a robot interface, sending the target code suitable for the robot to the robot to drive the robot to execute the action corresponding to the target general command. The application can realize unified control of general actions of robots of different manufacturers.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, and more particularly to a robot control method and system based on a general command and an electronic device. BACKGROUND

[0002] At present, artificial intelligence technology has realized ChatGPT-style "brain leap", and embodied intelligent robots are promoting robots from "digital intelligence" to "physical intelligence" by means of deep coupling of AI algorithms and mechanical ontology. At present, large models endow robots with the ability to understand instructions, and multi-modal perception systems help them realize environmental interaction. With the decline of hardware costs and the evolution of algorithms, general robots are gradually moving from the laboratory to the factory and to the family.

[0003] However, different types of robots have unique hardware and communication protocols, and there are differences in operation control commands and interfaces. Although each robot comes with basic actions such as handshaking, waving, and walking, these actions can only be operated through the self-provided remote controller or the APP developed by the manufacturer.

[0004] Under the existing mode of relying on the remote controller or APP provided by the manufacturer for each robot, the commands supported by different robots are different, for example, some support waving, and some only support lifting the hand, which leads to the difficulty in forming a unified standard and docking method when docking with robots from the application level, and only custom development docking can be used for different robots.

[0005] Therefore, how to provide a robot control method, system and electronic device which can realize unified control of general actions of different manufacturer robots is a problem that needs to be solved by those skilled in the art. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a robot control method and system based on a general command and an electronic device.

[0007] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0008] In a first aspect, a robot control method based on a general command is provided, comprising the following steps:

[0009] S1: obtaining a target general command;

[0010] The target general command belongs to a preset general command library; the general command library includes a plurality of general commands, each general command includes action code and semantic information; the semantic information includes one or more of the direction of the executed action, the numerical value of the executed action, the unit of the numerical value, and the device ID of the executed action;

[0011] S2: performing executability verification on the target general command based on the state of the robot;

[0012] If the executability verification is passed, S3 is performed;

[0013] If the executability verification is not passed, the target general command is converted into a SPEAK command and a message that the target general command cannot be executed is played;

[0014] S3: obtaining a target code to be filled based on the action code of the target general command;

[0015] The target code to be filled belongs to a preset robot command library; the robot command library includes a plurality of target codes to be filled; one action code corresponds to one target code to be filled;

[0016] S4: filling semantic information of the target general command into the target code to be filled to obtain a filled target code;

[0017] S5: performing format conversion on the filled target code to obtain a target code suitable for the robot;

[0018] S6: calling a robot interface to send the target code suitable for the robot to the robot to drive the robot to perform an action corresponding to the target general command.

[0019] Preferably, S1 specifically includes the following steps:

[0020] S11: obtaining voice information and converting the voice information into text information;

[0021] S12: inputting the text information into a human-computer interaction large model to obtain an action code corresponding to the text information; wherein the human-computer interaction large model is obtained based on the general command library;

[0022] S13: determining a regular expression corresponding to the text information based on the action code corresponding to the text information;

[0023] S14: extracting semantic information in the text information based on the regular expression; wherein the semantic information in the text information and the action code corresponding to the text information constitute the target general command.

[0024] Preferably, S2 specifically includes the following steps:

[0025] S21: determining whether the robot is in a power-on state or a power-off state;

[0026] If the robot is in a power-off state, the executability verification is not passed;

[0027] If the robot is in a power-on state, S22 is performed;

[0028] S22: determining whether the robot is in a normal state or an abnormal state;

[0029] if the robot is in an abnormal state, the executable verification is failed;

[0030] if the robot is in a normal state, S23 is performed;

[0031] S23: determining whether the robot is in an idle state or a working state;

[0032] if the robot is in an idle state, the executable verification is passed;

[0033] if the robot is in a working state, it is determined whether the command currently executed by the robot and the target general command can be executed simultaneously;

[0034] if the command currently executed by the robot and the target general command can be executed simultaneously, the executable verification is passed;

[0035] if the command currently executed by the robot and the target general command cannot be executed simultaneously, the executable verification is failed.

[0036] Preferably, S12 further comprises the following steps:

[0037] if the human-computer interaction large model cannot output the action code corresponding to the text information, the text information is packaged as a SPEAK command and the text to be played is obtained; wherein the obtained SPEAK command and the text to be played constitute a delivery instruction.

[0038] Preferably, S2 specifically comprises the following steps:

[0039] the delivery instruction is subjected to an executable verification, a conclusion that the executable verification is failed is obtained, and the text to be played is played; wherein the text to be played is used to play a message of an unrecognized command.

[0040] Preferably, if the robot has a built-in microphone module and a built-in speech-to-text module, the built-in microphone module and the built-in speech-to-text module are used to convert the speech information into the text information.

[0041] if the robot has a built-in microphone module but does not have a built-in speech-to-text module, the speech information obtained by the built-in microphone module is obtained through a microphone connection device, and the text information is converted through an external speech-to-text module.

[0042] Preferably, S12 further comprises the following steps:

[0043] the text information is packaged into a speech event and sent to the human-computer interaction large model through a microphone event queue.

[0044] Preferably, the target general command is obtained by calling a general command library through APP or directly calling the general command library.

[0045] In a second aspect, a general command-based robot control system is provided for implementing the general command-based robot control method as described in the first aspect, comprising a target general command acquisition module, an executability verification module and a robot component connection plug-in.

[0046] The target general command acquisition module is configured to acquire a target general command.

[0047] The target general command belongs to a preset general command library; the general command library comprises a plurality of general commands, each general command comprising an action code and semantic information; the semantic information comprises one or more of a direction of an executed action, a numerical value of the executed action, a unit of the numerical value and a device ID of the executed action.

[0048] The executability verification module is configured to perform executability verification on the target general command based on a state of the robot; if the executability verification is passed, the target general command is sent to the robot component connection plug-in; if the executability verification is not passed, the target general command is converted into a SPEAK command and a message indicating that the target general command cannot be executed is played.

[0049] The robot component connection plug-in is configured to obtain a target code to be filled based on the action code of the target general command.

[0050] The target code to be filled belongs to a preset robot command library; the robot command library comprises a plurality of target codes to be filled; one action code corresponds to one target code to be filled.

[0051] The robot component connection plug-in is further configured to fill the semantic information of the target general command into the target code to be filled to obtain a filled target code.

[0052] The robot component connection plug-in is further configured to perform format conversion on the filled target code to obtain a target code suitable for the robot.

[0053] The robot component connection plug-in is further configured to call a robot interface and send the target code suitable for the robot to the robot to drive the robot to execute an action corresponding to the target general command.

[0054] In a third aspect, an electronic device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the general command-based robot control method as described in the first aspect when executing the computer program.

[0055] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a robot control method, system and electronic device based on general commands, which can realize unified control of the general actions of robots from different manufacturers. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0057] Figure 1 This is a flowchart of a robot control method based on general commands provided in an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Firstly, such as Figure 1 As shown, this embodiment of the invention provides a robot control method based on general commands, including the following steps:

[0061] S1: Obtain the target's general command;

[0062] The target general command belongs to a preset general command library; the general command library includes several general commands, each of which includes action code and semantic information; the semantic information includes one or more of the following: the direction of the action, the magnitude of the action, the unit of the value, and the device ID of the action.

[0063] In one or more embodiments, the action codes included in the general command library are shown in Table 1:

[0064] Table 1 shows the action codes included in the general command library.

[0065]

[0066] The direction of the execution action includes FRONT, BACK, LEFT, RIGHT, UP, DOWN, EAST, SOUTH, WEST, NORTH, LEFT_FRONT, LEFT_BACK, RIGHT_FRONT, RIGHT_BACK, a total of 14 directions.

[0067] The numerical size of the execution action is a floating point value, for example, for the action code BODY_MOVE, a command of one step, two steps, etc. can be issued, and here one and two are the size of the numerical value;

[0068] The unit of the numerical value represents the unit of measurement of the numerical value of the execution action, including meters, steps and degrees;

[0069] The device ID of the execution action is used to represent the code of the device performing the action, and the device ID included in the general command library of the present application is shown in Table 2:

[0070] Table 2 Device ID included in the general command library

[0071]

[0072] It can be understood that:

[0073] The action code BODY_MOVE, the direction of the execution action, the numerical size of the execution action, the unit of the numerical value and the device ID of the execution action constitute a general command of walking;

[0074] The action code BODY_TURN, the direction of the execution action, the numerical size of the execution action, the unit of the numerical value and the device ID of the execution action constitute a general command of turning less than 90 degrees;

[0075] The action code BODY_ROTATE, the direction of the execution action, the numerical size of the execution action, the unit of the numerical value and the device ID of the execution action constitute a general command of turning 180 degrees;

[0076] The action code ARM_LIFT, the direction of the execution action and the device ID of the execution action constitute a general command of lifting the hand;

[0077] The remaining action codes are respectively combined with the corresponding device ID of the execution action to form a general command.

[0078] The chassis in Table 2 is used for turning or walking of a wheeled robot;

[0079] The body in Table 2 is used for turning of a humanoid robot.

[0080] In one or more embodiments, S1 specifically comprises the following steps:

[0081] S11: Acquire voice information and convert the voice information into text information;

[0082] In one or more embodiments, if the robot has a built-in microphone module and a built-in voice-to-text module, the built-in microphone module and the built-in voice-to-text module are used to convert the voice information into the text information.

[0083] If the robot has a built-in microphone module but does not have a built-in voice-to-text module, the voice information acquired by the built-in microphone module is acquired through a microphone connection device, and the text information is converted through an external voice-to-text module.

[0084] S12: Input the text information into a human-computer interaction large model to obtain an action code corresponding to the text information; wherein the human-computer interaction large model is trained based on the general command library;

[0085] In one or more embodiments, S12 further comprises the following steps:

[0086] If the human-computer interaction large model cannot output the action code corresponding to the text information, the text information is packaged into a SPEAK command and a text to be played is acquired; wherein the acquired SPEAK command and the text to be played constitute a delivery instruction.

[0087] In one or more embodiments, S12 further comprises the following steps:

[0088] The text information is packaged into a voice event and sent to the human-computer interaction large model through a microphone event queue.

[0089] S13: Determine a regular expression corresponding to the text information based on the action code corresponding to the text information;

[0090] It can be understood that: the present application formulates a regular expression for each action code to identify the direction, value, unit and device to be executed by the command.

[0091] The present application solves the problem that the robot command is difficult to be recognized by a simple regular expression through the way of combining a large model with a regular expression. On the other hand, a corresponding regular expression is formulated for each action code, thereby solving the problem that a general command cannot design an accurate regular expression to identify the direction, value, unit and device to be executed by the command.

[0092] S14: extracting the semantic information in the text information based on the regular expression; wherein the semantic information in the text information and the action code corresponding to the text information constitute the target universal command.

[0093] In one or more embodiments, the target universal command can also be obtained by calling the universal command library through an APP or directly calling the universal command library.

[0094] It can be understood that: the universal command library is exposed to the outside through the API in the manner of the http protocol, so that the third party and the extension development application can realize the direct control of the robot action by calling the universal command library, thereby realizing the control and operation of the heterogeneous robot without binding the interface related to the robot hardware.

[0095] S2: performing the executability check on the target universal command based on the state of the robot:

[0096] If the executability check is passed, S3 is performed;

[0097] If the executability check is not passed, the target universal command is converted into a SPEAK command and a message that cannot be executed is played;

[0098] In one or more embodiments, S2 specifically comprises the following steps:

[0099] S21: judging whether the robot is in a power-on state or a power-off state;

[0100] If the robot is in the power-off state, the executability check is not passed;

[0101] If the robot is in the power-on state, S22 is performed;

[0102] S22: judging whether the robot is in a normal state or an abnormal state;

[0103] If the robot is in the abnormal state, the executability check is not passed;

[0104] If the robot is in the normal state, S23 is performed;

[0105] S23: judging whether the robot is in an idle state or a working state;

[0106] If the robot is in the idle state, the executability check is passed;

[0107] If the robot is in the working state, it is judged whether the current command executed by the robot and the target universal command can be executed simultaneously:

[0108] If the commands can be executed simultaneously, the executability check is passed;

[0109] If not, the target general command fails the executability check.

[0110] In one or more embodiments, after passing the executability check, the target general command is sent to a to-be-executed command queue, waiting to be executed. By continuously monitoring the commands from the to-be-executed command queue, the commands that need to be executed can be obtained in real time, and the state of the robot can be automatically updated when the commands are executed, so that the state of the robot can be updated in real time.

[0111] It can be understood that the state of the robot is used to maintain the global real-time state of the robot, and the state can be used to coordinate the work and instructions from the microphone, arm, body and other parts of the robot, to avoid conflicts between the devices of the robot and ensure smooth execution of the basic commands of the robot.

[0112] In one or more embodiments, S2 specifically comprises the following steps:

[0113] The executability of the issued instruction is checked, and a conclusion that the instruction fails the executability check is obtained and the to-be-played text is played; wherein the to-be-played text is used to play a message that the command cannot be recognized.

[0114] S3: obtaining a to-be-filled target code based on the action code of the target general command;

[0115] The to-be-filled target code belongs to a preset robot command library; the robot command library comprises a plurality of to-be-filled codes; one action code corresponds to one to-be-filled code;

[0116] S4: filling the semantic information of the target general command into the to-be-filled target code to obtain a filled target code;

[0117] It can be understood that:

[0118] In the robot, in order to enable the robot arms to perform various actions, essentially the angles or angular velocities of the motors of the arms are controlled, and in order to enable the lower body of the robot to perform various actions, essentially the angles or angular velocities of the motors of the legs are controlled for humanoid robots, and for wheeled robots, the target coordinate points of the chassis movement are controlled. Therefore, the to-be-filled target code is essentially a python code of the to-be-filled motor angular velocity or angle, and the target point of the chassis movement;

[0119] The following is an example of a filled target code obtained after filling semantic information into the action code WAVE:

[0120] WAVE: / / indicates the action code

[0121] cmd.params[dct.TARGET_POSE] = [-40, 75, 0, 0, 180, 0] / / represents the target pose of the robot arm, i.e. the target angle of the arm joint motor

[0122] cmd.params[dct.ARM_SPEED] = 30 / / represents the speed of the robot arm joint motor rotating to the target angle

[0123] cmd.params[dct.MOVE_METHOD] = 'X_YZ' / / represents the rotation method of the robot joint, which is to rotate the X direction first, then the Y, and then the Z

[0124] cmd.params[dct.GRAP_METHOD] = 'OPEN_CLOSE' / / represents that the robot end effector is in the closed state

[0125] Wherein, cmd.params represents the command parameter structure defined in the robot plug-in, and dct is a constant file.

[0126] S5: formatting the filled target code to obtain a target code suitable for the robot;

[0127] S6: calling the robot interface to send the target code suitable for the robot to the robot to drive the robot to perform the action corresponding to the target general command.

[0128] In a second aspect, the embodiments of the present application provide a robot control system based on a general command, which is used to implement the robot control method based on a general command as described in the first aspect, and includes a target general command acquisition module, an executability verification module and a robot component connection plug-in.

[0129] The target general command acquisition module is used to acquire a target general command.

[0130] The target general command belongs to a preset general command library; the general command library includes a plurality of general commands, each general command includes action encoding and semantic information; the semantic information includes one or more of the direction of performing an action, the numerical value of performing an action, the unit of the numerical value and the device ID of performing an action.

[0131] The executability verification module is used to perform executability verification on the target general command based on the state of the robot: if the executability verification is passed, the target general command is sent to the robot component connection plug-in; if the executability verification is not passed, the target general command is converted into a SPEAK command and a message that cannot be executed is played;

[0132] The robot component connection plug is used to obtain the target code to be filled based on the action encoding of the target general command;

[0133] The target code to be filled belongs to a preset robot command library; the robot command library includes several codes to be filled; one action code corresponds to one code to be filled.

[0134] The robot component connection plug is also used to fill the semantic information of the target general command into the target code to be filled, so as to obtain the filled target code;

[0135] The robot component connection plug is also used to convert the format of the filled target code to obtain target code suitable for the robot.

[0136] The robot component connection plug is also used to call the robot interface and send the target code applicable to the robot to drive the robot to perform the action corresponding to the target general command.

[0137] Thirdly, embodiments of the present invention also provide an electronic device, such as... Figure 2 As shown, the electronic device may include a processor 201, a communications interface 202, a memory 203, and a communication bus 204. The processor 201, communications interface 202, and memory 203 communicate with each other via the communication bus 204. The processor 201 can call logical instructions from the memory 203 to execute a robot control method based on general commands.

[0138] Furthermore, the logical instructions in the aforementioned memory 203 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] The various embodiments described in this specification are implemented in a progressive manner, each embodiment focusing on the differences from other embodiments, and the same or similar parts between embodiments can be mutually referred to. For the apparatus disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0140] The above description of disclosed embodiments enables one of ordinary skill in the art to make or use the application. Various modifications to these embodiments will be readily apparent to those of ordinary skill in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A robot control method based on general commands, characterized in that, Includes the following steps: S1: Obtain the target's general command; The target general command belongs to a pre-defined general command library; the general command library includes several general commands, each of which includes an action code and semantic information; the semantic information includes one or more of the following: the direction of the action, the magnitude of the action, the unit of the value, and the device ID of the action; specifically including: S11: Acquire voice information and convert the voice information into text information; S12: Input the text information into the human-computer interaction model to obtain the action code corresponding to the text information; wherein, the human-computer interaction model is trained based on the general command library; S13: Determine the corresponding regular expression based on the action code corresponding to the text information; S14: Extract the semantic information from the text information based on the regular expression; wherein, the semantic information in the text information and the action code corresponding to the text information constitute the target general command; S2: Perform executability verification on the target general command based on the robot's state: If the executability check is passed, then execute S3; If the executability check fails, the target general command is converted into a SPEAK command and a message indicating that it cannot be executed is played. S3: Obtain the target code to be filled based on the action encoding of the target general command; The target code to be filled belongs to a preset robot command library; the robot command library includes several codes to be filled; one action code corresponds to one code to be filled. S4: Fill the semantic information of the target general command into the target code to be filled, and obtain the filled target code; S5: Convert the format of the filled target code to obtain target code suitable for the robot; S6: Call the robot interface to send the target code applicable to the robot to drive the robot to execute the action corresponding to the target general command.

2. The robot control method based on general commands according to claim 1, characterized in that, S2 specifically includes the following steps: S21: Determine whether the robot is powered on or powered off; If the device is powered off, the executableness check will fail. If the device is powered on, execute S22; S22: Determine whether the robot is in a normal or abnormal state; If the status is abnormal, the executability check will not be passed; If the state is normal, then execute S23; S23: Determine whether the robot is in an idle state or a working state; If the state is idle, then the executableness check is passed; If the robot is in a working state, determine whether the command it is currently executing and the target general command can be executed simultaneously: If they can be executed simultaneously, then the executability check is passed. If they cannot be executed simultaneously, the executability check fails.

3. The robot control method based on general commands according to claim 1, characterized in that, S12 also includes the following steps: If the large human-computer interaction model cannot output the action code corresponding to the text information, then the text information is encapsulated into a SPEAK command and the text to be played is obtained; wherein, the encapsulated SPEAK command and the text to be played constitute the instruction to be issued.

4. The robot control method based on general commands according to claim 3, characterized in that, S2 specifically includes the following steps: The issued command is subjected to an executability check. If the executability check fails, the text to be played is played. The text to be played is used to play a message about an unrecognizable command.

5. A robot control method based on general commands according to claim 1, characterized in that: If the robot has a built-in microphone module and a built-in speech-to-text module, then the built-in microphone module and the built-in speech-to-text module are used to convert the speech information into the text information. If the robot has a built-in microphone module but no built-in speech-to-text module, then a microphone connection device is used to acquire the voice information acquired by the built-in microphone module, and then an external speech-to-text module is used to convert it into the text information.

6. The robot control method based on general commands according to claim 1, characterized in that, S12 also includes the following steps: The text information is encapsulated into a voice event and sent to the human-computer interaction model via a microphone event queue.

7. A robot control method based on general commands according to claim 1, characterized in that, The target general command can be obtained by calling the general command library through the APP or by directly calling the general command library.

8. A robot control system based on general commands, characterized in that, The robot control method based on general commands as described in any one of claims 1 to 7 includes a target general command acquisition module, an executability verification module, and a robot component connection plug-in. The target general command acquisition module is used to acquire the target general command; The target general command belongs to a preset general command library; the general command library includes several general commands, each of which includes action code and semantic information; the semantic information includes one or more of the following: the direction of the action, the magnitude of the action, the unit of the value, and the device ID of the action. The executability verification module is used to perform executability verification on the target general command based on the robot's state: if the executability verification passes, the target general command is sent to the robot component connection plug-in; if the executability verification fails, the target general command is converted into a SPEAK command and a message indicating that it cannot be executed is played. The robot component connection plug is used to obtain the target code to be filled based on the action encoding of the target general command; The target code to be filled belongs to a preset robot command library; the robot command library includes several codes to be filled; one action code corresponds to one code to be filled. The robot component connection plug is also used to fill the semantic information of the target general command into the target code to be filled, so as to obtain the filled target code; The robot component connection plug is also used to convert the format of the filled target code to obtain target code suitable for the robot. The robot component connection plug is also used to call the robot interface and send the target code applicable to the robot to drive the robot to perform the action corresponding to the target general command.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the robot control method based on general commands as described in any one of claims 1 to 7.

Images