Robot-based task execution method, device, computer equipment, readable storage medium, and program product

Abstract

The present application relates to a robot-based task execution method, apparatus, computer device, computer-readable storage medium, and computer program product. The method comprises: when a task type is determined to be a complex task type based on a received voice signal, decomposing the complex task using a large model to obtain subtasks and the corresponding contextual relationships of the subtasks; decomposing each subtask to obtain the actions and / or skills corresponding to each subtask; generating a task execution graph based on the contextual relationships of the subtasks, the decomposed actions and skills, the contextual relationships between the actions and skills, and the robot's state and posture; generating a task execution target path based on the task execution graph, and executing the task based on the actions and skills on the target path to obtain a task execution result. This method can improve task processing efficiency and ensure accurate task execution.

Description

Technical Field

[0001] The present application relates to a robot-based task execution method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of modern technology, the application scope of intelligent robots is becoming more and more extensive. Intelligent robots can be seen in homes, shopping malls, banks and other public places.

[0003] However, in the process of executing tasks, the robot in the wear technology can smoothly execute the defined tasks, but cannot smoothly execute complex tasks due to the lack of definition, which leads to the reduction of the robot's processing performance. Summary of the Invention

[0004] Based on this, it is necessary to provide a robot-based task execution method, device, computer equipment, computer-readable storage medium and computer program product that can improve task processing efficiency and ensure accurate execution of tasks in response to the above technical problems.

[0005] In a first aspect, the present application provides a robot-based task execution method, the method comprising:

[0006] When the task type is determined to be a complex task based on the received speech signal, the complex task is decomposed into subtasks and the corresponding context relationships of the subtasks using the large model;

[0007] Decompose each subtask to obtain the corresponding actions and / or skills for each subtask;

[0008] Generate a task execution graph based on the contextual relationships of subtasks, the disassembled actions and skills, the contextual relationships between actions and skills, and the robot's state and posture;

[0009] A task execution target path is generated based on the task execution graph, and the task is executed based on the actions and skills on the target path to obtain a task execution result.

[0010] In one embodiment, generating a task execution graph based on the contextual relationship of subtasks, the decomposed actions and skills, the contextual relationship between actions and skills, and the state and posture of the robot includes:

[0011] Use the disassembled actions and skills as graph nodes;

[0012] Determining a first connection line for each of the graph nodes based on the contextual relationship of the subtasks and the decomposed actions and skills;

[0013] Based on the contextual relationship between actions and skills and the state and position of the robot, a second connection line of each of the graph nodes is determined.

[0014] In one embodiment, determining the second connection line of each graph node based on the contextual relationship between the action and the skill and the state and position of the robot includes:

[0015] Determine the virtual connections between each graph node based on the context of actions and skills;

[0016] When the states and positions of the robots of the two graph nodes connected by the virtual line are the same, the virtual line is used as the second line;

[0017] When the states and positions of the robots of two graph nodes connected by a virtual line are different, the virtual line is deleted.

[0018] In one embodiment, generating a task execution target path based on the task execution graph includes:

[0019] Determine the key graph nodes corresponding to each subtask, and determine the start nodes based on the context of the subtasks.

[0020] Taking each starting node as the starting point and traversing all key nodes as the goal, the execution path of each task is obtained;

[0021] The task execution path that meets the requirements is selected as the task execution target path.

[0022] In one embodiment, before determining that the task type is a complex task type based on the received voice signal, the method includes:

[0023] Receive user voice signals, convert them into text information, input the text information into the big model, and obtain user intentions;

[0024] Identify the task type based on user intent and obtain the task type;

[0025] When the task type is an action in an action library corresponding to a task or a skill in a skill library corresponding to a task, obtaining an instruction corresponding to the task;

[0026] Control the robot to execute the instructions.

[0027] In one embodiment, identifying the task type based on the user's intention to obtain the task type includes:

[0028] Based on the pre-trained classification model, the user intention is identified to obtain the task category;

[0029] The pre-trained classification model is trained based on a small data sample set.

[0030] In a second aspect, the present application further provides a robot-based task execution device, the device comprising:

[0031] A first disassembly module is configured to, when determining based on the received speech signal that the task type is a complex task type, disassemble the complex task into subtasks and the corresponding context relationships of the subtasks using a large model;

[0032] The second disassembly module is used to disassemble each subtask to obtain the corresponding action and / or skill of each subtask;

[0033] The task execution graph generation module is used to generate a task execution graph based on the contextual relationship of subtasks, the decomposed actions and skills, the contextual relationship between actions and skills, and the robot's state and posture;

[0034] The task execution module is used to generate a task execution target path based on the task execution graph, and execute the task based on the actions and skills on the target path to obtain a task execution result.

[0035] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in any one of the above embodiments when executing the computer program.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method in any one of the above-mentioned embodiments when the computer program is executed by a processor.

[0037] In a fifth aspect, the present application also provides a computer program product, comprising a computer program, which implements the steps of the method in any one of the above embodiments when executed by a processor.

[0038] The above-mentioned robot-based task execution method, device, computer equipment, computer-readable storage medium and computer program product, when determining that the task type is a complex task type, can decompose the complex task to obtain sub-tasks and the context of sub-tasks, decompose the sub-tasks to obtain the actions and / or skills corresponding to each sub-task, and finally generate a task execution graph based on the context of the sub-tasks, the decomposed actions and skills, the context of the actions and skills, and the state and position of the robot, and then generate the task execution target path based on the task execution graph, so that some nodes can be filtered out, the task processing efficiency can be improved, and the accurate execution of the task can be ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 1 is a flowchart of a robot-based task execution method according to an embodiment;

[0041] Figure 2 A flowchart of the steps of generating a task execution graph in one embodiment;

[0042] Figure 3 A flowchart of a task execution target path generation step in one embodiment;

[0043] Figure 4 is a structural block diagram of a robot-based task execution device in one embodiment;

[0044] Figure 5 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0046] In one embodiment, Figure 1 As shown, a robot-based task execution method is provided. This embodiment uses the method applied to a robot terminal as an example for illustration. It is understandable that the method can also be applied to a server corresponding to the robot terminal, and can also be applied to a system including a robot terminal and a server, and is implemented through the interaction between the robot terminal and the server. In this embodiment, the method includes the following steps:

[0047] S102: When the task type is determined to be a complex task type based on the received voice signal, the complex task is decomposed into subtasks and corresponding context relationships of the subtasks using a large model.

[0048] After receiving the voice signal, the robot can determine the task type based on the voice signal. Optionally, the voice signal can be processed based on the large model to obtain the task type. In this application, the task type can include the action corresponding to the task type in the action library, the skill corresponding to the task type in the skill library, and the complex task type. Among them, the task type that is not the action corresponding to the task type in the action library or the skill corresponding to the task type in the skill library is a complex task type.

[0049] When it is determined that the task type is a complex type, the complex task is disassembled by using a large model, so that each subtask and the corresponding front-and-back relationship of the subtasks can be obtained. The subtasks are abstract tasks. For example, if the complex task is "exchanging water cups on two tables", it can be disassembled into the subtasks of "putting water cup A1 on table 1 on table 2", "putting water cup A2 on table 1 on table 2", "putting water cup B1 on table 2 on table 1", and "putting water cup B2 on table 2 on table 1". In addition, it should be noted that the splitting here is based on the grammar of natural language.

[0050] The contextual relationship of subtasks refers to the relationship between subtasks, where a certain subtask must be executed before another subtask, or a certain subtask must be executed after another subtask, or there is no contextual relationship between two subtasks, etc. In one optional embodiment, the contextual relationship of subtasks can be represented by a tree structure.

[0051] S104: Decompose each subtask to obtain the action and / or skill corresponding to each subtask.

[0052] To execute subtasks, they need to be broken down into actions in an action library and / or skills in a skill library. The action library includes pre-configured basic robot actions, such as "pick up" and "put down." The skill library includes encapsulated skills, each of which can include at least two actions in the action library.

[0053] In this application, because subtasks are expressed in abstract natural language, which the robot cannot recognize, each subtask is broken down into actions in the action library and / or skills in the skill library, thereby obtaining a corresponding execution sequence for the robot. It should also be noted that this breakdown is based on the actions in the action library and the skills in the skill library.

[0054] For ease of understanding, the subtask of "putting cup A1 on table 1 onto table 2" can be divided into the actions of "picking up cup A1 on table 1", "moving it above table 2" and "putting down cup A1".

[0055] S106: Generate a task execution graph based on the contextual relationship of the subtasks, the decomposed actions and skills, the contextual relationship between the actions and skills, and the state and posture of the robot.

[0056] The contextual relationship between actions and skills can be: a certain action or skill must be performed before another action or skill, or a certain action or skill must be performed after another action or skill, or there is no contextual relationship between the two actions or skills. In one optional embodiment, the contextual relationship between actions or skills can be configured as an attribute of the action or skill when generating the action library and skill library.

[0057] The robot's status refers to its working state, including both active and inactive. The working state can be further categorized as whether it is holding other objects, etc. The specific configuration needs to be tailored to the specific application scenario.

[0058] Posture refers to the posture of the robot, including the position and posture of the robot.

[0059] A task execution graph is an execution flow chart of a complex task. In the task execution graph, graph nodes represent actions or skills, and the lines between graph nodes represent execution relationships. The lines can have directions, meaning that only after a graph node is executed can the next graph node after the line is executed.

[0060] Based on the contextual relationships of subtasks, actions and skills, and the robot's state and posture, the connections between the various graph nodes can be determined, thereby generating a task execution graph.

[0061] S108: Generate a task execution target path based on the task execution graph, and execute the task based on the actions and skills on the target path to obtain a task execution result.

[0062] The task execution target path is the optimal path for task execution. The optimal path may be the shortest path, or the optimal path may be the path with the least number of turns, etc., which is not specifically limited here.

[0063] In this application, the task execution graph can be traversed using an A* algorithm to obtain the optimal path. The robot can then call the instructions corresponding to the actions in the action library and the instructions corresponding to the skills in the skill library based on each graph node on the optimal path to execute the corresponding task and obtain the task execution result.

[0064] The above-mentioned robot-based task execution method, when determining that the task type is a complex task type, can decompose the complex task to obtain the sub-tasks and the context of sub-tasks, decompose the sub-tasks to obtain the actions and / or skills corresponding to each sub-task, and finally generate a task execution graph based on the context of sub-tasks, the decomposed actions and skills, the context of actions and skills, and the state and position of the robot, and then generate the task execution target path based on the task execution graph, so that some nodes can be filtered out, the task processing efficiency can be improved, and the accurate execution of the task can be ensured.

[0065] In one of the optional embodiments, see Figure 2 As shown, Figure 2 This is a flowchart of the task execution graph generation step in one embodiment, which includes generating a task execution graph based on the contextual relationship of subtasks, the disassembled actions and skills, the contextual relationship between actions and skills, and the state and posture of the robot. The flowchart includes: using the disassembled actions and skills as graph nodes; determining the first connection line of each graph node based on the contextual relationship of subtasks and the disassembled actions and skills; and determining the second connection line of each graph node based on the contextual relationship between actions and skills and the state and position of the robot.

[0066] This embodiment involves a process of generating a task execution graph, which includes graph nodes and connections between graph nodes.

[0067] The decomposed actions and skills are used as graph nodes. It should be noted that different subtasks may include the same actions and skills. Therefore, different graph nodes are generated for the same actions and skills of different subtasks. The index of each graph node can be the identifier of the subtask, action, and skill.

[0068] Since there is a contextual relationship among subtasks, the first connection line of each action and skill can be determined based on the contextual relationship among subtasks, thus completing the initialization of the task execution graph.

[0069] Subsequently, since there is a contextual relationship between actions and skills, and each graph node corresponds to the robot's state and position, the second connection line of each graph node can be determined based on the robot's state and position and the contextual relationship between actions and skills, thereby enriching the task execution graph.

[0070] For ease of understanding, the above example is still used for illustration. The subtask "Put cup A1 on table 1 onto table 2" can be broken down into the actions of "picking up cup A1 on table 1", "moving it above table 2" and "putting down cup A1". Similarly, the subtask of "Put cup A2 on table 1 onto table 2", the subtask of "Put cup B1 on table 2 onto table 1" and the subtask of "Put cup B2 on table 2 onto table 1" can also be broken down into corresponding actions or skills.

[0071] Therefore, various graph nodes can be generated: the action of "picking up cup A1 on table 1", the action of "moving to the top of table 2", and "putting down cup A1", etc., and the first connection of the graph nodes can be established. Since there is no front-and-back relationship between the subtasks here, the starting point of each subtask can be connected to the end point of any subtask.

[0072] A second connection can then be determined based on the robot's state and position, as well as the contextual relationship between actions and skills. For example, the action of "putting down cup A1" must occur after the action of "picking up cup A1 on table 1," and the action of "picking up cup B1 on table 2" must occur after the action of "moving to the top of table 2," provided the robot is not currently holding any other objects. Therefore, the second connection can be determined based on these contextual relationships.

[0073] In the above embodiment, the connection between each graph node in the task execution graph is determined not only by the contextual relationship of subtasks but also by the contextual relationship of actions and skills, thereby providing multiple paths for task execution.

[0074] In one embodiment, the second connection line of each graph node is determined based on the contextual relationship between actions and skills and the state and position of the robot, including: determining the virtual connection line of each graph node based on the contextual relationship between actions and skills; when the state and position of the robots of the two graph nodes connected by the virtual connection line are the same, using the virtual connection line as the second connection line; when the state and position of the robots of the two graph nodes connected by the virtual connection line are different, deleting the virtual connection line.

[0075] The virtual connections between graph nodes are determined based solely on the context of actions and skills. These virtual connections do not take into account the robot's state and position, making some virtual connections inappropriate. Therefore, we filter the virtual connections based on the robot's state and position, retaining only those with the same state and position as the second connection.

[0076] Specifically, in this application, the virtual lines can be filtered first based on the robot's position to delete some virtual lines, and then a second filtering can be performed based on the robot's state. In conjunction with the above example, the first filtering can be performed based on whether the robot is located above Table 2, and then a second filtering can be performed based on whether the robot is holding other objects, thereby ensuring the accuracy of the second line.

[0077] In the above embodiment, the connections are filtered according to the position and state of the robot, thereby reducing the number of connections in the task execution graph and ensuring the accuracy of the task execution graph.

[0078] In one of the optional embodiments, combined with Figure 3 As shown, Figure 3 This is a flowchart of the task execution target path generation step in an embodiment. The task execution target path generation step, that is, generating the task execution target path based on the task execution graph, includes: determining the key graph node corresponding to each subtask, and determining each starting node based on the corresponding front-end relationship of the subtask; taking each starting node as the starting point and traversing each key node as the goal, to obtain each task execution path; selecting the task execution path that meets the requirements as the task execution target path.

[0079] The key graph nodes may include the end graph node of each subtask and the start node determined based on the subtask. The start node is the first node of each subtask determined after considering the context of the subtasks. If a subtask does not need to be executed before any other subtasks, the first node of the subtask can be the start node, and the obtained start node is stored in the start node set. The key graph node is the end graph node of each subtask, and the end graph nodes of these subtasks can be stored as key graph nodes in the key graph node set.

[0080] Then, each task execution path is generated, starting from the start node and aiming to traverse all the end graph nodes in the key graph nodes. In this application, only traversing all the end graph nodes in the key graph nodes is the goal because the end graph nodes must be completed before the subtasks are completed. For example, the action of "putting down cup A1" must precede the action of "picking up cup A1 on table 1".

[0081] The requirements to be met may include the shortest path, the least number of turns, etc. In this application, corresponding requirements can be set based on the scenario needs, and then the task execution path can be screened based on the requirements to obtain the task execution target path.

[0082] Still taking the above example of picking up cups as an example, if path optimization is not performed, the final task execution plan is the subtask of "putting cup A1 on table 1 onto table 2", the subtask of "putting cup A2 on table 1 onto table 2", the subtask of "putting cup B1 on table 2 onto table 1", and the subtask of "putting cup B2 on table 2 onto table 1"; in this application, optimization is performed, and the task execution becomes the subtask of "putting cup A1 on table 1 onto table 2", the subtask of "putting cup B2 on table 2 onto table 1", the subtask of "putting cup A2 on table 1 onto table 2", and the subtask of "putting cup B1 on table 2 onto table 1", thereby achieving path optimization.

[0083] In the above embodiment, each starting point and key graph node is first determined, and then each task execution path is generated with the start node as the starting point and with the goal of traversing each end graph node in the key graph node. The paths corresponding to the task execution can be obtained, laying the foundation for subsequent path screening.

[0084] In one of the optional embodiments, before determining that the task type is a complex task type based on the received voice signal, it includes: receiving a user voice signal, converting the user voice signal into text information, inputting the text information into a large model, and obtaining the user intention; identifying the task type based on the user intention to obtain the task type; when the task type is an action in the action library corresponding to a task or a skill in the skill library corresponding to a task, obtaining instructions corresponding to the task; and controlling the robot to execute the instructions.

[0085] The voice signal is input by the user, and after the robot collects the user's voice signal, it can convert the voice signal into text information. Specifically, the text conversion can be performed through voice recognition.

[0086] The large model is pre-trained to obtain user intent. The large model can process text information to obtain user intent, where the user intent can be expressed in natural language.

[0087] Task types include action-corresponding task types in the action library, skill-corresponding task types in the skill library, and complex task types. Task types that are not action-corresponding task types in the action library or skill-corresponding task types in the skill library are all complex task types.

[0088] Among them, the actions in the action library corresponding to the tasks or the skills in the skill library corresponding to the tasks can be executed directly, for example, directly obtaining the robot executable instructions corresponding to the actions in the action library or the robot executable instructions corresponding to the skills in the skill library, and then controlling the robot to execute these executable instructions.

[0089] In one of the optional embodiments, task type identification is performed based on user intention to obtain the task type, including: identifying user intention based on a pre-trained classification model to obtain the task category; the pre-trained classification model is trained based on a small data sample set.

[0090] The task type can also be identified using a large model, which can be trained based on a small sample data set, thereby improving training accuracy while ensuring training speed. Subsequently, the user's intent is input into the large model to obtain the corresponding task category.

[0091] In each of the above-mentioned embodiments, user intent is first identified based on a large model, and the task type (actions in the action library corresponding to tasks, skills in the skill library corresponding to tasks, and complex tasks) is determined. For the first two types of tasks, the action library and skill library can be directly invoked for execution. For complex tasks, task decomposition is performed. Next, executing the complex task involves first decomposing the complex task into subtasks using the large model, and then decomposing the subtasks into actions and / or skills corresponding to each subtask. A task execution graph is generated based on the tree structure of the subtasks, the decomposed actions and skills, the action attributes of the action library and the skill attributes of the skill library, and the robot's state and position. The key nodes (end nodes) corresponding to each subtask are identified, and each initial node is determined based on the tree structure of the subtasks. Starting from the initial node and aiming to traverse all key nodes, an optimal task execution path is generated based on the task execution graph, and the task is executed based on this optimal path. This approach enables the execution of complex tasks and ensures accurate task execution.

[0092] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0093] Based on the same inventive concept, embodiments of the present application also provide a robot-based task execution device for implementing the aforementioned robot-based task execution method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more robot-based task execution device embodiments provided below can be found in the above-described limitations of the robot-based task execution method and will not be further elaborated here.

[0094] In an exemplary embodiment, Figure 4 As shown, a robot-based task execution device is provided, comprising: a first disassembly module 401, a second disassembly module 402, a task execution graph generation module 403 and a task execution module 404, wherein:

[0095] A first decomposition module 401 is configured to, when the task type is determined to be a complex task type based on the received speech signal, decompose the complex task into subtasks and the corresponding context relationships of the subtasks using a large model;

[0096] The second decomposition module 402 is used to decompose each subtask to obtain the corresponding action and / or skill of each subtask;

[0097] A task execution graph generation module 403 is configured to generate a task execution graph based on the contextual relationships of the subtasks, the decomposed actions and skills, the contextual relationships between actions and skills, and the robot's state and posture;

[0098] The task execution module 404 is used to generate a task execution target path based on the task execution graph, and execute the task based on the actions and skills on the target path to obtain a task execution result.

[0099] In one of the optional embodiments, the above-mentioned task execution graph generation module 403 is specifically used to use the disassembled actions and skills as graph nodes; determine the first connection line of each graph node based on the contextual relationship of the subtasks and the disassembled actions and skills; and determine the second connection line of each graph node based on the contextual relationship of the actions and skills and the state and position of the robot.

[0100] In one of the optional embodiments, the above-mentioned task execution graph generation module 403 is specifically used to determine the virtual connection of each graph node based on the contextual relationship between actions and skills; when the status and position of the robots of the two graph nodes connected by the virtual connection are the same, the virtual connection is used as the second connection; when the status and position of the robots of the two graph nodes connected by the virtual connection are different, the virtual connection is deleted.

[0101] In one of the optional embodiments, the above-mentioned task execution module 404 is specifically used to determine the key graph node corresponding to each subtask, and determine each starting node based on the previous and next relationships corresponding to the subtasks; taking each starting node as the starting point and traversing each key node as the goal, each task execution path is obtained; and the task execution path that meets the requirements is selected as the task execution target path.

[0102] In one of the optional embodiments, the above-mentioned device also includes: an instruction acquisition module, used to receive user voice signals, convert the user voice signals into text information, input the text information into the large model, and obtain the user intention; identify the task type based on the user intention to obtain the task type; when the task type is an action in the action library corresponding to the task or a skill in the skill library corresponds to the task, obtain the instruction corresponding to the task; control the robot to execute the instruction.

[0103] In one of the optional embodiments, the above-mentioned instruction acquisition module is also used to identify the user intention based on a pre-trained classification model to obtain the task category; the pre-trained classification model is trained based on a small data sample set.

[0104] Each module in the aforementioned robot-based task execution device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in the form of hardware, or may be stored in a computer device memory in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0105] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 5As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be implemented via Wi-Fi, a mobile cellular network, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a robot-based task execution method. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0106] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0107] In an exemplary embodiment, a computer device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: when the task type is determined to be a complex task type based on a received voice signal, the complex task is decomposed into subtasks and the corresponding context relationships of the subtasks through a large model; each subtask is decomposed into actions and / or skills corresponding to each subtask; a task execution graph is generated according to the context relationships of the subtasks, the decomposed actions and skills, the context relationships between the actions and skills, and the state and posture of the robot; a task execution target path is generated based on the task execution graph, and the task is executed based on the actions and skills on the target path to obtain a task execution result.

[0108] In one embodiment, when a processor executes a computer program, a task execution graph is generated based on the context of subtasks, the disassembled actions and skills, the context of actions and skills, and the state and posture of the robot, including: using the disassembled actions and skills as graph nodes; determining the first connection line of each graph node based on the context of subtasks and the disassembled actions and skills; and determining the second connection line of each graph node based on the context of actions and skills and the state and position of the robot.

[0109] In one embodiment, when a processor executes a computer program, the second connection line of each graph node is determined based on the contextual relationship between actions and skills and the state and position of the robot, including: determining the virtual connection line of each graph node based on the contextual relationship between actions and skills; when the state and position of the robots of the two graph nodes connected by the virtual connection line are the same, the virtual connection line is used as the second connection line; when the state and position of the robots of the two graph nodes connected by the virtual connection line are different, the virtual connection line is deleted.

[0110] In one embodiment, the generation of a task execution target path based on a task execution graph implemented by a processor when executing a computer program includes: determining a key graph node corresponding to each subtask, and determining each start node based on the contextual relationship corresponding to the subtask; taking each start node as a starting point and traversing each key node as a goal to obtain each task execution path; and selecting a task execution path that meets the requirements as the task execution target path.

[0111] In one embodiment, the processor executes a computer program, and before determining that the task type is a complex task type based on the received voice signal, the steps include: receiving a user voice signal, converting the user voice signal into text information, inputting the text information into a large model, and obtaining the user intention; identifying the task type based on the user intention to obtain the task type; when the task type is an action in the action library corresponding to a task or a skill in the skill library corresponding to a task, obtaining instructions corresponding to the task; and controlling the robot to execute the instructions.

[0112] In one embodiment, the task type identification based on user intention implemented by the processor when executing the computer program to obtain the task type includes: identifying the user intention based on a pre-trained classification model to obtain the task category; the pre-trained classification model is obtained by training based on a small data sample set.

[0113] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: when the task type is determined to be a complex task type based on the received voice signal, the complex task is decomposed into each subtask and the corresponding context relationship of the subtasks through a large model; each subtask is decomposed into the action and / or skill corresponding to each subtask; a task execution graph is generated according to the context relationship of the subtasks, the decomposed actions and skills, the context relationship between the actions and skills, and the state and posture of the robot; a task execution target path is generated based on the task execution graph, and the task is executed based on the actions and skills on the target path to obtain a task execution result.

[0114] In one embodiment, when a computer program is executed by a processor, it generates a task execution graph based on the context of subtasks, the decomposed actions and skills, the context of actions and skills, and the state and posture of the robot, including: using the decomposed actions and skills as graph nodes; determining the first connection line of each graph node based on the context of subtasks and the decomposed actions and skills; determining the second connection line of each graph node based on the context of actions and skills and the state and position of the robot.

[0115] In one embodiment, the computer program implemented when executed by the processor determines the second connection line of each graph node based on the contextual relationship of actions and skills and the state and position of the robot, including: determining the virtual connection line of each graph node based on the contextual relationship of actions and skills; when the state and position of the robots of the two graph nodes connected by the virtual connection line are the same, using the virtual connection line as the second connection line; when the state and position of the robots of the two graph nodes connected by the virtual connection line are different, deleting the virtual connection line.

[0116] In one embodiment, the computer program generated based on the task execution graph when executed by the processor includes: determining the key graph node corresponding to each subtask, and determining each start node based on the contextual relationship corresponding to the subtask; taking each start node as the starting point and traversing each key node as the goal to obtain each task execution path; and selecting the task execution path that meets the requirements as the task execution target path.

[0117] In one embodiment, the computer program implemented when executed by the processor includes: receiving a user voice signal, converting the user voice signal into text information, inputting the text information into a large model, and obtaining the user intention; identifying the task type based on the user intention to obtain the task type; obtaining the instructions corresponding to the task when the task type is an action in the action library corresponding to the task or a skill in the skill library corresponding to the task; and controlling the robot to execute the instructions.

[0118] In one embodiment, when a computer program is executed by a processor, the task type is identified based on the user intention to obtain the task type, including: identifying the user intention based on a pre-trained classification model to obtain the task category; the pre-trained classification model is obtained by training based on a small data sample set.

[0119] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps: when the task type is determined to be a complex task type based on a received voice signal, decomposing the complex task through a large model to obtain subtasks and the corresponding context relationships of the subtasks; decomposing each subtask to obtain the actions and / or skills corresponding to each subtask; generating a task execution graph based on the context relationships of the subtasks, the decomposed actions and skills, the context relationships between the actions and skills, and the state and posture of the robot; generating a task execution target path based on the task execution graph, and executing the task based on the actions and skills on the target path to obtain a task execution result.

[0120] In one embodiment, when a computer program is executed by a processor, it generates a task execution graph based on the context of subtasks, the decomposed actions and skills, the context of actions and skills, and the state and posture of the robot, including: using the decomposed actions and skills as graph nodes; determining the first connection line of each graph node based on the context of subtasks and the decomposed actions and skills; determining the second connection line of each graph node based on the context of actions and skills and the state and position of the robot.

[0121] In one embodiment, the computer program implemented when executed by the processor determines the second connection line of each graph node based on the contextual relationship of actions and skills and the state and position of the robot, including: determining the virtual connection line of each graph node based on the contextual relationship of actions and skills; when the state and position of the robots of the two graph nodes connected by the virtual connection line are the same, using the virtual connection line as the second connection line; when the state and position of the robots of the two graph nodes connected by the virtual connection line are different, deleting the virtual connection line.

[0122] In one embodiment, the computer program generated based on the task execution graph when executed by the processor includes: determining the key graph node corresponding to each subtask, and determining each start node based on the contextual relationship corresponding to the subtask; taking each start node as the starting point and traversing each key node as the goal to obtain each task execution path; and selecting the task execution path that meets the requirements as the task execution target path.

[0123] In one embodiment, the computer program implemented when executed by the processor includes: receiving a user voice signal, converting the user voice signal into text information, inputting the text information into a large model, and obtaining the user intention; identifying the task type based on the user intention to obtain the task type; obtaining the instructions corresponding to the task when the task type is an action in the action library corresponding to the task or a skill in the skill library corresponding to the task; and controlling the robot to execute the instructions.

[0124] In one embodiment, when a computer program is executed by a processor, the task type is identified based on the user intention to obtain the task type, including: identifying the user intention based on a pre-trained classification model to obtain the task category; the pre-trained classification model is obtained by training based on a small data sample set.

[0125] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0126] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0127] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0128] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A robot-based task execution method, characterized in that: The method comprises: When the task type is determined to be a complex task based on the received speech signal, the complex task is decomposed into subtasks and the corresponding context relationships of the subtasks using the large model; Decompose each subtask to obtain the corresponding actions and skills for each subtask; A task execution graph is generated based on the contextual relationships of subtasks, the decomposed actions and skills, the contextual relationships between actions and skills, and the robot's state and posture; the task execution graph includes graph nodes and lines between the graph nodes; the graph nodes are actions or skills, and the lines between the graph nodes are execution relationships. The lines have directions, and the directions are used to indicate that a graph node can only be executed after the execution of the previous graph node. Generating a task execution target path based on the task execution graph, and executing the task based on the actions and skills on the target path to obtain a task execution result; The task execution graph is generated based on the contextual relationship of the subtasks, the decomposed actions and skills, the contextual relationship between the actions and skills, and the state and posture of the robot, including: Use the disassembled actions and skills as graph nodes; Determining a first connection line for each of the graph nodes based on the contextual relationship of the subtasks and the decomposed actions and skills; Based on the contextual relationship between actions and skills and the state and position of the robot, a second connection line of each of the graph nodes is determined.

2. The method according to claim 1, characterized in that The determining of the second connection line of each of the graph nodes based on the contextual relationship between the action and the skill and the state and position of the robot includes: Determine the virtual connections between each graph node based on the context of actions and skills; When the states and positions of the robots of the two graph nodes connected by the virtual line are the same, the virtual line is used as the second line; When the states and positions of the robots of two graph nodes connected by a virtual line are different, the virtual line is deleted.

3. The method according to claim 1, characterized in that Generating a task execution target path based on the task execution graph includes: Determine the key graph nodes corresponding to each subtask, and determine the start nodes based on the context of the subtasks. Taking each starting node as the starting point and traversing all key nodes as the goal, the execution path of each task is obtained; The task execution path that meets the requirements is selected as the task execution target path.

4. The method according to any one of claims 1 to 3, characterized in that Before determining that the task type is a complex task type based on the received voice signal, the method includes: Receive user voice signals, convert them into text information, input the text information into the big model, and obtain user intentions; Identify the task type based on user intent and obtain the task type; When the task type is an action in an action library corresponding to a task or a skill in a skill library corresponding to a task, obtaining an instruction corresponding to the task; Control the robot to execute the instructions.

5. The method according to claim 4, characterized in that The task type identification based on the user intention is performed to obtain the task type, including: Based on the pre-trained classification model, the user intention is identified to obtain the task type; The pre-trained classification model is trained based on a small data sample set.

6. A robot-based task execution device, characterized in that: The device comprises: A first disassembly module is configured to, when determining based on the received speech signal that the task type is a complex task type, disassemble the complex task into subtasks and the corresponding context relationships of the subtasks using a large model; The second disassembly module is used to disassemble each subtask to obtain the corresponding actions and skills of each subtask; A task execution graph generation module is configured to generate a task execution graph based on the contextual relationships of subtasks, the decomposed actions and skills, the contextual relationships between actions and skills, and the robot's state and posture. The task execution graph includes graph nodes and lines between the graph nodes. The graph nodes are actions or skills, and the lines between the graph nodes are execution relationships. The lines have directions, indicating that a subsequent graph node can only be executed after the execution of the previous graph node. A task execution module is used to generate a task execution target path based on the task execution graph, and execute the task based on the actions and skills on the target path to obtain a task execution result; The task execution graph generation module is specifically used to use the disassembled actions and skills as graph nodes; determine the first connection line of each graph node based on the contextual relationship of the subtasks and the disassembled actions and skills; and determine the second connection line of each graph node based on the contextual relationship of the actions and skills and the state and position of the robot.

7. The device according to claim 6, characterized in that The task execution graph generation module is specifically used to determine the key graph nodes corresponding to each subtask, and determine each starting node based on the contextual relationship of the subtasks; taking each starting node as the starting point and traversing all key nodes as the goal, the execution path of each task is obtained; The task execution path that meets the requirements is selected as the task execution target path.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images