Task processing method and robot

Abstract

The application provides a task processing method and a robot, the robot comprising a classification machine, a task machine and a process state machine, the method comprising: the classification machine first determining whether a target task is a process task, if yes, sending task information corresponding to the target task to the task machine, if not, determining a task type corresponding to the target task, and sending the task information corresponding to the target task to the process state machine according to the task type. The task machine pre-processes the task information to make it meet the task execution requirements corresponding to the process task, and then sends it to the process state machine. After receiving the task information, the process state machine executes a task process corresponding to the target task. The target task is an agent task sent by a large language model corresponding to the robot, and the process task is a task comprising multiple different types of tasks. In this way, the robot can better process the agent task, improve the performance of the robot, and reduce the software development and maintenance cost of the robot.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a task processing method and a robot. BACKGROUND

[0002] In the field of artificial intelligence, large language models and agents are two important concepts. Large language models generally refer to large-scale pre-trained language models that can implement natural language understanding, generation and other processing. Agents are autonomous software entities that can perceive environmental states, make decisions and take actions in a specific environment.

[0003] In the software development process of robots such as full-size humanoid robots, a robot task processing scheme based on large language models and agents can be implemented through a robot operating system (ROS) platform. The robot can execute an agent task sent by the large language model to implement the corresponding function.

[0004] As the functions of robots become increasingly complex, how to better implement the processing of agent tasks by robots to improve the performance of robots and reduce the software development and maintenance costs of robots is a problem that is being explored in the current field. SUMMARY

[0005] The present application provides a task processing method and a robot, which can solve the problem of how to better implement the processing of agent tasks by robots to improve the performance of robots and reduce the software development and maintenance costs of robots.

[0006] To solve the above technical problems, in a first aspect, an embodiment of the present application provides a task processing method applied to a robot, the robot comprising a classifier, a task machine and a process state machine. The method comprises: determining, by the classifier, whether a target task is a process task, sending, by the classifier, task information corresponding to the target task to the task machine in a case where it is determined that the target task is a process task, determining, by the classifier, a task type corresponding to the target task in a case where it is determined that the target task is not a process task, sending, by the classifier, the task information corresponding to the target task to the process state machine according to the task type, the target task being an agent task sent by a large language model corresponding to the robot, the process task being a task comprising a plurality of different types of tasks; pre-processing, by the task machine, the task information corresponding to the target task in a case where the task information corresponding to the target task is received, so that the task information corresponding to the target task meets a task execution requirement corresponding to the process task, and sending, by the task machine, the pre-processed task information corresponding to the target task to the process state machine; and executing, by the process state machine, a task process corresponding to the target task in a case where the task information corresponding to the target task is received.

[0007] By using the above task processing method, three independent modules of a classification machine, a task machine and a process state machine are constructed in the robot, wherein the classification machine first determines whether the target task is a process task; if it is determined to be a process task, the task information is transmitted to the task machine; if it is not, the task type is determined, and the task information is transmitted to the process state machine. After receiving the information, the task machine performs preprocessing to make the task information meet the task execution requirements corresponding to the process task, and then transmits it to the process state machine. After receiving the task information, the process state machine executes the corresponding task process. In this way, the robot is based on the three independent modules of the classification machine, the task machine and the process state machine, and each module has clear division of labor and mutual cooperation. According to whether the task is a process task and the task type, the task processing is performed, which can avoid mutual interference, realize accurate and rapid processing of the task, and improve the task execution efficiency, thereby improving the performance of the robot. Moreover, each module is independent, which is also convenient for software development and maintenance. If subsequent optimization of a certain link is required, such as upgrading the preprocessing logic of the task machine, only the task machine needs to be adjusted, without changing the classification machine and / or the process state machine. That is, each model can be modified and maintained independently, which reduces the cost of software development and maintenance of the robot.

[0008] In a possible implementation of the first aspect, the target task includes task configuration information, and the task configuration information includes task type identification information. The classification machine determines whether the target task is a process task, including: the classification machine performs task matching processing through the task type identification information in the task configuration information included in the target task and a preset task type detection function, to determine whether the target task is a process task. If multiple different types of tasks are matched in the target task, it is determined that the target task is a process task. If only one task is matched in the target task, it is determined that the target task is not a process task. The task type detection function is a type detection function corresponding to a corresponding task type.

[0009] By using the above task processing method, the task configuration information included in the target task includes task type identification. The classification machine uses this identification to match with a preset task type detection function. If multiple different types of tasks are matched in the task, it is a process task. If there is only one task, it is not a process task. This way makes the task classification more accurate, reduces the debugging time, and improves the efficiency of task execution to improve the performance of the robot. Further, the task type detection function can be understood as a special detection tool corresponding to each task type, or can be referred to as a type marking function.

[0010] In a possible implementation of the first aspect, the task processing method further includes: if the classification machine does not match the task type included in the target task, sending corresponding notification information to the task machine; and in response to the notification information, the task machine determines whether the target task is error information, and in a case where the target task is determined to be error information, does not execute the task processing corresponding to the target task, and causes the robot to present task error prompt information, and in a case where the target task is determined to be non-error information, determines the task type corresponding to the target task, and according to the task type, sends the task information corresponding to the target task to the flow state machine.

[0011] With the task processing method, the secondary determination of the task machine is used to determine whether the target task is error information, forming a double-check mechanism of the classification machine and the task machine, so that system stagnation caused by a classification blind area can be avoided. If the target task is determined to be error information, the robot directly presents the task error prompt information by playing the task error prompt information, and does not need to continue to execute invalid tasks (such as empty tasks or format error tasks), thereby saving computing resources. If the target task is determined to be non-error information (such as only missing a state identifier but the task itself is valid), the task machine can continue to determine the type and send the task information to the flow state machine, so that normal tasks are not discarded due to small defects (such as missing identifiers), the integrity of task processing is improved, and the performance of the robot is improved.

[0012] In a possible implementation of the first aspect, the preprocessing is task information processing based on a robot operating system, and the task information processing includes task execution logic switching processing, task execution sequence adjustment processing, encapsulation and packaging processing.

[0013] With the task processing method, the preprocessing is task information processing based on a robot operating system, so that execution failure caused by incompatibility between a task information format, logic and the system can be avoided. Furthermore, through the logic switching processing, the task execution sequence adjustment processing, the encapsulation and the packaging processing, it can be ensured that the task flow matches the hardware capability and the running scene of the robot, and the smoothness of task execution is improved. In addition, after the preprocessing, the information format is uniform and the logic is adapted, and the flow state machine does not need to process the basic adaptation problem again, and can further promote the task flow, thereby improving the performance of the robot.

[0014] In a possible implementation of the first aspect, the flow state machine includes a plurality of task modules, and the task machine sends the task information corresponding to the target task after preprocessing to the flow state machine, including: the task machine sequentially sends the task information corresponding to each task to the corresponding task module in the flow state machine according to the plurality of different types of tasks included in the target task.

[0015] In a possible implementation of the first aspect, the task machine sends the task information corresponding to the target task to the flow state machine after preprocessing, and the task machine can also pack the task information corresponding to a plurality of different types of tasks included in the target task together and send the task information to the flow state machine, or pack the task information of several tasks together and send the task information to the flow state machine, and the corresponding task module in the flow state machine processes the task information, and the sending manner can be set as needed.

[0016] According to the task processing method, the plurality of tasks of the flow task are corresponded to the plurality of modules of the flow state machine, each module processes only the task in the corresponding state, and the logic confusion or high load caused by processing all states by a single module can be avoided, so that the smooth processing of the task is ensured and the performance of the robot is improved. The task machine distributes the tasks in sequence, so that the task flow advances in the order of states, and the current task stage is easy to monitor. Meanwhile, the independence of the modules facilitates the subsequent optimization of the processing logic of a state without changing other modules, and the software development and maintenance cost is reduced.

[0017] In a possible implementation of the first aspect, the classification machine sends the task information corresponding to the target task to the flow state machine according to the task type, including: the classification machine invokes the corresponding service or topic according to the task type to send the task information corresponding to the target task to the corresponding task module in the flow state machine.

[0018] According to the task processing method, the classification machine directly locates the target module of the flow state machine by invoking the corresponding service or topic, which is the communication manner of the robot system, so that the task is prevented from being transferred in disorder among a plurality of modules, the efficiency of task execution is improved, and the performance of the robot is improved.

[0019] In a possible implementation of the first aspect, in the case where the target task is a flow task, the flow state machine executes the task flow corresponding to the target task after receiving the task information corresponding to the target task, including: the flow state machine executes the tasks in the target task in sequence according to the execution order of the tasks until the execution of the tasks in the target task is completed.

[0020] In a possible implementation of the first aspect, in the case where the target task is a flow task, if the flow state machine receives the task information corresponding to a plurality of target tasks, the method further includes: the flow state machine executes the task flows corresponding to the target tasks in sequence according to the execution order of the target tasks.

[0021] The process task refers to a task sequence composed of multiple different types of tasks, which need to be executed in sequence according to the order of their types. In the actual task processing process, there are usually multiple such process tasks in the task list, and each single task in the process task can be regarded as a separate execution unit or as a whole. For example, process task A includes a stage task and a stage task, and process task B also includes a stage task and a stage task, but the specific operation content of the same stage task in different process tasks is not the same. Then, if the process state machine receives task information corresponding to multiple target tasks, the task list to be executed is [A, B...]. The a-stage task is, for example, a hand task, and the b-stage task is, for example, a navigation task.

[0022] By using the above task processing method, the tasks are executed in sequence according to the task execution order, avoiding skipping of key tasks or reversal of the task order, ensuring that the tasks ultimately achieve the goal, and reducing task failures caused by incorrect execution order. After the execution of each task is completed, the execution result can be recorded, and if the subsequent task fails, the task that caused the problem can be quickly located, reducing the difficulty of troubleshooting and further improving the overall task execution efficiency to improve the performance of the robot.

[0023] In a possible implementation of the above first aspect, the process state machine executes the tasks in sequence according to the task execution order, including: after executing a preset task in the tasks, the process state machine determines whether there is an event of interacting with the large language model, and in the case that there is an event of interacting with the large language model, the interaction with the large language model is realized using a free question and answer mode, and in the case that there is no event of interacting with the large language model, the next task is executed.

[0024] The preset task can be set as needed, for example, the preset task is a navigation task.

[0025] By using the above task processing method, the process state machine determines whether there is an event of interacting with the large language model after executing a preset task in the tasks, that is, allows interaction with the large language model at a key node, and improves the flexibility and intelligence of task processing through the free question and answer mode. By judging whether to interact, unnecessary communication can be avoided, and process lag caused by frequent communication with the large language model can be prevented to ensure execution efficiency. Only when necessary, the number of calls to the large language model is reduced, saving computing power and communication resources, and the adaptability of the free question and answer mode can be compatible with the interaction format of different large language models, reducing the adaptation cost of model replacement, thereby reducing the software development and maintenance cost.

[0026] In a possible implementation of the first aspect, each task includes a plurality of task implementation stages, the plurality of task implementation stages include an idle stage, a triggered task stage, a task running waiting stage, and a task completion stage, and the flow state machine executes each task according to the plurality of task implementation stages.

[0027] By adopting the task processing method, the whole cycle process of each task from waiting for starting to ending execution is clear through the stage division of the idle stage, the triggered task stage, the task running waiting stage, and the task completion stage, and execution loopholes caused by ambiguous stages are avoided. Moreover, the state of each stage can be fed back in real time, and a maintainer can quickly locate problems. In addition, the division of the four stages is universal, and stage logic does not need to be designed for each task separately, thereby reducing software development and maintenance costs.

[0028] In a second aspect, an embodiment of the present application provides a robot, including a memory for storing a computer program, the computer program including program instructions, and a processor for executing the program instructions to enable the robot to execute the task processing method.

[0029] In a third aspect, an embodiment of the present application provides a task processing system, including a server and a robot, wherein the robot includes a classifier, a task machine, and a flow state machine, and the server integrates a large language model.

[0030] The server is configured to generate a target task through the large language model and send the target task to the robot.

[0031] The robot is configured to receive the target task through the classifier, determine whether the target task is a flow task, send task information corresponding to the target task to the task machine in a case where the target task is determined to be the flow task, determine a task type corresponding to the target task in a case where the target task is determined to be a non-flow task, and send the task information corresponding to the target task to the flow state machine according to the task type; the task machine is configured to, in a case where the task information corresponding to the target task is received, pre-process the task information corresponding to the target task, and send the pre-processed task information corresponding to the target task to the flow state machine; and the flow state machine is configured to, in a case where the task information corresponding to the target task is received, execute a task flow corresponding to the target task.

[0032] In a fourth aspect, an embodiment of the present application provides a computer program product including instructions, which, when executed by a robot, enable the robot to execute the task processing method.

[0033] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium including computer program instructions, which, when executed by a robot, enable the robot to execute the task processing method.

[0034] The related beneficial effects of the above-mentioned second aspect to the fifth aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the present application, the drawings used in the description of the embodiments of the present application will be briefly introduced.

[0036] Figure 1 is a state diagram of the interaction between the robot and the server provided by the embodiments of the present application;

[0037] Figure 2 is a structural diagram of the robot provided by the embodiments of the present application;

[0038] Figure 3 is a flow diagram of the task processing method provided by the embodiments of the present application;

[0039] Figure 4 is another flow diagram of the task processing method provided by the embodiments of the present application;

[0040] Figure 5 is a principle diagram of the full-size humanoid robot implementing the task processing provided by the embodiments of the present application;

[0041] Figure 6 is a flow diagram of the rockai_mission_idf implementing the corresponding task processing provided by the embodiments of the present application;

[0042] Figure 7 is another flow diagram of the rockai_mission_idf implementing the corresponding task processing provided by the embodiments of the present application;

[0043] Figure 8 is a principle diagram of the rockai_mission_transfer implementing the corresponding task processing provided by the embodiments of the present application;

[0044] Figure 9 is a flow diagram of the rockai_mission_transfer implementing the corresponding task processing provided by the embodiments of the present application;

[0045] Figure 10 is another flow diagram of the rockai_mission_transfer implementing the corresponding task processing method provided by the embodiments of the present application;

[0046] Figure 11 is a flow diagram of the rockai_mission_sm implementing the corresponding task processing provided by the embodiments of the present application;

[0047] Figure 12 is a schematic diagram of a task processing process corresponding to the WalkMusic state provided by an embodiment of the present application;

[0048] Figure 13 is a schematic diagram of a task processing process corresponding to the Navigation state provided by an embodiment of the present application;

[0049] Figure 14 is a schematic diagram of a task processing process corresponding to the ArmAction state provided by an embodiment of the present application;

[0050] Figure 15 is a schematic diagram of a task processing process corresponding to the Introduction state provided by an embodiment of the present application;

[0051] Figure 16 is a schematic diagram of a task processing process corresponding to the Modeling state provided by an embodiment of the present application;

[0052] Figure 17 is another structural schematic diagram of a robot provided by an embodiment of the present application. DETAILED DESCRIPTION

[0053] As described above, with the complication of the functions of the robot, how to better implement the processing of the Agent task by the robot, improve the performance of the robot, and reduce the software development and maintenance cost is a problem that is being explored in the current field.

[0054] Taking a human-shaped robot as an example, in the current prior art, the mainstream robot Agent task processing system based on the Robot Operating System (ROS) platform includes a robot and a server such as a cloud server, as shown in Figure 1 . Among them, a large language model (which can be referred to as a large model, such as a Yan large model, etc.) is integrated in the server, which is used to generate an Agent task instruction (which can also be referred to as an Agent task).

[0055] And based on the mainstream development process of the ROS platform, the processing logic of the robot for the Agent task instructions returned by the cloud server large language model has formed a relatively fixed pattern. First, the Agent task instructions output by the large language model are taken as the core callback function, and the callback function is taken as the execution entry to schedule and control each task module contained in the task flow. Specifically, for asynchronous tasks that do not need to be executed synchronously, the system will directly run them in sequence according to the preset order of the task modules; and for tasks that have synchronization requirements (i.e., tasks that require multiple modules to cooperate and advance simultaneously), the related two or more task modules are scheduled to be executed in parallel through multi-threading technology, ensuring that the synchronization requirements between tasks are met.

[0056] When all the task modules in a single task flow are executed, the robot system will immediately close the current task module, release the resources occupied by the task, and enter a standby state, waiting for the next Agent task instruction sent by the large language model. The system runs in a loop of receiving instructions, scheduling execution, closing modules, and waiting, completing the task interaction between the humanoid robot and the large language model.

[0057] Further, the above scheme has simplicity in the acquisition of Agent task instructions and the design of basic flow structure, for example, the sequential execution of task scheduling can reduce the difficulty of code understanding, and developers can quickly master the code logic and corresponding business logic.

[0058] However, this scheme has some defects, which restrict the extensibility and maintainability of the humanoid robot task processing system. On the one hand, the core logic and execution flow of task processing are fixed in the code architecture, and there is a lack of interaction interface and collaboration mechanism between task modules. Each module can only execute a single function according to the preset path, and cannot flexibly adjust the collaboration relationship between modules according to business needs, resulting in poor compatibility when the system faces new function expansion and needs to make large-scale modifications to the original code. On the other hand, when dealing with complex functions involving multiple module collaboration (such as the linkage task of simultaneously completing navigation, mechanical arm action, and voice explanation for the robot), the interaction process between modules becomes cumbersome due to the lack of modularization and framework support, and the code coupling degree is high. Once a fault occurs during task execution and needs to be debugged and repaired, or a new task logic needs to be added to the business scenario, the developer needs to find the associated part in the lengthy and coupled code, which not only increases the difficulty of debugging, but also easily causes new code problems, resulting in a significant increase in time cost for development and maintenance.

[0059] More importantly, the scheme also has the problem of poor adaptability to project progress. When facing unexpected temporary project tasks, the modification and adaptation of the original code often occupy a large number of development resources, and then the development progress of other normally advancing projects is forced to delay. For a system that has been developed for a long time and accumulated multiple historical versions, as the number of code modifications increases, historical baggage (such as redundant code, obsolete logic, and implicit dependencies between modules) accumulates, and any subsequent modification of the code needs to consider complex historical compatibility issues, the difficulty and cost of code modification are large, which affects the efficiency and stability of the human robot and large model interaction task processing.

[0060] In summary, the current task processing method and robot mainly have the problems of poor system expansibility and maintainability, and poor adaptability to project progress. That is, the existing task processing logic and process are fixed and have poor expansibility, and the existing technology has no modular and framework design support, resulting in high system maintenance cost and low problem troubleshooting efficiency; the complex task interaction process is cumbersome, and the multi-module collaboration capability is weak; historical version iteration leads to historical baggage accumulation and increased modification cost; the response capability to unexpected tasks is insufficient, and the project progress is easily affected. Thus, the reliability and efficiency of the robot for task processing are affected, the performance of the robot is affected, and the software development and maintenance cost is affected.

[0061] To solve the above technical problems, the embodiments of the present application disclose a robot, as shown in Figure 2 The robot includes a classification machine, a task machine, and a process state machine. The classification machine is used to determine whether the target task is a process task. If it is determined to be a process task, the task information is transmitted to the task machine. If it is not, the task type is determined, and the task information is transmitted to the process state machine. After receiving the task information, the task machine performs preprocessing to make the task information meet the task execution requirements of the process task, and then transmits it to the process state machine. After receiving the task information, the process state machine executes the corresponding task process.

[0062] Further, the embodiments of the present application provide a task processing method applied to Figure 2 The robot as shown in Figure 3 The task processing method includes the following steps:

[0063] Step S101: The classification machine determines whether the target task is a process task, and in the case of determining that the target task is a process task, sends the task information corresponding to the target task to the task machine, and in the case of determining that the target task is not a process task, determines the task type corresponding to the target task, and according to the task type, sends the task information corresponding to the target task to the process state machine. The target task is an agent task sent by a large language model corresponding to a robot, and the process task is a task including multiple different types of tasks.

[0064] Step S102: The task machine, in the case of receiving the task information corresponding to the target task, pre-processes the task information corresponding to the target task to make the task information corresponding to the target task meet the task execution requirements corresponding to the process task, and sends the pre-processed task information corresponding to the target task to the process state machine.

[0065] Step S103: The process state machine, in the case of receiving the task information corresponding to the target task, executes the task process corresponding to the target task.

[0066] The target task is an agent task, or also referred to as an agent task instruction. The large language model can be a Yan large model, or other large models.

[0067] The above task processing method is adopted, and three independent modules of a classification machine, a task machine and a process state machine are constructed in a robot. The classification machine first determines whether the target task is a process task, and if it is determined to be a process task, the task information is transmitted to the task machine, and if it is not, the task type is determined and the task information is transmitted to the process state machine. After receiving the task information, the task machine will pre-process the task information to meet the task execution requirements of the process task, and then transmit it to the process state machine. After receiving the task information, the process state machine executes the corresponding task process. In this way, the robot is based on the three independent modules of the classification machine, the task machine and the process state machine, and each module has clear division of labor and mutual cooperation. According to whether the task is a process task and the task type, the task is processed, mutual interference can be avoided, precise and rapid processing of the task is realized, and the task execution efficiency is improved, thereby improving the performance of the robot. Moreover, each module is independent, which is convenient for software development and maintenance. If subsequent optimization of a certain link is required, such as upgrading the pre-processing logic of the task machine, only the task machine needs to be adjusted, without the need to change the classification machine and / or the process state machine. That is, each model can be modified and maintained independently, reducing the software development and maintenance cost of the robot.

[0068] Further, the target task includes task configuration information, and the task configuration information includes task type identification information. The classification machine determines whether the target task is a process task by using the task type identification information in the task configuration information included in the target task and a preset task type detection function. If the target task includes multiple different types of tasks, the target task is determined to be a process task. If the target task includes only one task, the target task is determined to be a non-process task. The task type detection function is a type detection function corresponding to a corresponding task type.

[0069] In the above task processing method, the task configuration information included in the target task includes task type identification information. The classification machine uses the identification information to match the preset task type detection function. If the target task includes multiple different types of tasks, the target task is a process task. If the target task includes only one task, the target task is not a process task. This method can accurately classify tasks, reduce debugging time, improve task execution efficiency, and improve robot performance. Further, the task type detection function can be understood as a special detection tool corresponding to each task type, or can be referred to as a type marking function.

[0070] Further, the task configuration information can include other information in addition to the task type identification information, which can be set as needed.

[0071] Further, the preprocessing is a task information processing based on a robot operating system. The task information processing includes task execution logic switching processing, task execution order adjustment processing, encapsulation and packaging processing. Of course, the task information processing can also include other types of processing, which can be set as needed.

[0072] In the above task processing method, the preprocessing is a task information processing based on a robot operating system. This avoids execution failure caused by incompatibility between task information format and logic and the system. Through the logic switching processing, the task execution order adjustment processing, the encapsulation and the packaging processing, the task flow can be ensured to match the hardware capability and the running scene of the robot, and the smoothness of task execution can be improved. In addition, after the preprocessing is completed, the information format is uniform and the logic is adapted. The flow state machine does not need to process the basic adaptation problem again, and can further promote the task flow, thereby improving the robot performance.

[0073] Further, the flow state machine includes multiple task modules. The task machine sends the task information corresponding to the preprocessed target task to the flow state machine, including: the task machine sends the task information corresponding to each task to the corresponding task module in the flow state machine in sequence according to the multiple different types of tasks included in the target task.

[0074] Of course, the flow state machine can also send the task information of all tasks included in the target task to the flow state machine together, or send the task information of several tasks to the flow state machine together, and the corresponding task module can process it according to the needs.

[0075] The task module can be understood as a software and / or function module for executing a task, which is used to realize the processing of the corresponding task.

[0076] By using the above task processing method, the multiple tasks of the flow task are corresponded to the multiple modules of the flow state machine, and each module only processes the tasks of the corresponding state, which can avoid the logical confusion or high load caused by processing all states by a single module, thereby ensuring the smooth processing of the task and improving the performance of the robot. The task machine distributes in sequence, which makes the task flow proceed in the order of state, facilitating the monitoring of the current task stage. At the same time, the independence of the module facilitates the subsequent optimization of the processing logic of a certain state without the need to change other modules, thereby reducing the software development and maintenance cost.

[0077] Further, the classification machine sends the task information corresponding to the target task to the flow state machine according to the task type, including: the classification machine calls the corresponding service or topic according to the task type to send the task information corresponding to the target task to the corresponding task module in the flow state machine.

[0078] By using the above task processing method, the classification machine can directly locate the target module of the flow state machine by calling the corresponding service or topic, which is the corresponding communication mode of the robot system, thereby avoiding the disorderly circulation of tasks among multiple modules and improving the efficiency of task execution, and thus improving the performance of the robot.

[0079] Further, in the case that the target task is a flow task, the flow state machine executes the task flow corresponding to the target task after receiving the task information corresponding to the target task, including: the flow state machine executes each task in turn according to the task execution order according to the multiple different types of tasks included in the target task, until the execution of each task included in the target task is completed.

[0080] Further, in the case that the target task is a flow task, if the flow state machine receives the task information corresponding to multiple target tasks, it executes the task flow corresponding to each target task in turn according to the task execution order of each target task.

[0081] The process task refers to a task sequence composed of multiple different types of tasks, which need to be executed in sequence according to the order of their types. In the actual task processing process, there are usually multiple such process tasks in the task list, and each single task in the process task can be regarded as a separate execution unit or as a whole. For example, process task A includes a stage a task and a stage b task, and process task B also includes a stage a task and a stage b task, but the specific operation content of the same stage task in different process tasks is not the same. Then, if the process state machine receives task information corresponding to multiple target tasks, the task list to be executed is [A, B...]. The stage a task is, for example, a hand task, and the stage b task is, for example, a navigation task.

[0082] By using the above task processing method, the tasks are executed in sequence according to the execution order, avoiding skipping of key tasks or reversing the order, ensuring that the tasks ultimately achieve the goal, and reducing task failures caused by incorrect execution order. After the execution of each task is completed, the execution result can be recorded, and if the subsequent task fails, the task that caused the problem can be quickly located, reducing the difficulty of troubleshooting and further improving the overall task execution efficiency to improve the performance of the robot.

[0083] Further, the process state machine executes the tasks in sequence according to the execution order, including: after executing a preset task in the tasks, the process state machine determines whether there is an event of interacting with the large language model, and in the case that there is an event of interacting with the large language model, the interaction with the large language model is realized using a free question and answer mode, and in the case that there is no event of interacting with the large language model, the next task is executed.

[0084] The preset task can be set as needed, for example, the preset task is a navigation task.

[0085] Of course, the process state machine can also determine whether there is an event of interacting with the large language model during the execution of the preset task, and the confirmation timing can be set as needed.

[0086] By using the above task processing method, the process state machine determines whether there is an event of interacting with the large language model after executing a preset task in the tasks, that is, allows interaction with the large language model at a key node, and improves the flexibility and intelligence of task processing through the free question and answer mode. By judging whether to interact, unnecessary communication can be avoided, and execution efficiency can be ensured by preventing process lag caused by frequent communication with the large language model. Only when necessary, the number of calls to the large language model is reduced, saving computing power and communication resources, and the adaptability of the free question and answer mode can be compatible with the interaction format of different large language models, reducing the adaptation cost of model replacement, thereby reducing the software development and maintenance cost.

[0087] Further, each task includes a plurality of task implementation stages, and the plurality of task implementation stages include an idle stage, a triggering task stage, a task running waiting stage, and a task completion stage. The flow state machine executes each task according to the plurality of task implementation stages.

[0088] Among them, Idling can represent the idle stage, do nothing, Triggering represents the triggering task stage, issues corresponding topics, instructions, interactions, etc., Running issues the running waiting stage after the instruction, and Done represents the task completion stage.

[0089] By adopting the above task processing method, the whole cycle process of each task from waiting to start to execution end is clear through the stage division of the idle stage, the triggering task stage, the task running waiting stage, and the task completion stage, and the execution loophole caused by ambiguous stages is avoided. In addition, the state of each stage can be fed back in real time, and the maintainer can quickly locate the problem. In addition, the division of the four stages has universality, and there is no need to design stage logic for each task separately, which reduces the software development and maintenance cost.

[0090] In an embodiment of the present application, as shown in Figure 4 The task processing method further includes the following steps:

[0091] Step S201: If the classification machine does not match the task type included in the target task, the classification machine sends corresponding notification information to the task machine.

[0092] Step S202: The task machine determines whether the target task is error information in response to the notification information. If the target task is determined to be error information, the task machine does not execute the corresponding processing of the target task, and makes the robot play a task error prompt information. If the target task is determined to be non-error information, the task machine determines the task type corresponding to the target task, and sends the task information corresponding to the target task to the flow state machine according to the task type.

[0093] By adopting the above task processing method, the secondary judgment of the task machine is used to determine whether the target task is error information, forming a double-checking mechanism of the classification machine and the task machine, which can avoid system stagnation caused by classification blind area. If it is determined to be error information, the robot directly plays a prompt without continuing to execute invalid tasks (such as empty tasks and format error tasks), saving the computing resource. If it is determined to be non-error information (such as only missing state identifier but the task itself is valid), the task machine can continue to determine the type and send it to the flow state machine, avoiding discarding normal tasks due to small defects (such as missing identifier), improving the integrity of task processing, and improving the performance of the robot.

[0094] The task processing method and the robot provided by the embodiment of the present application will be further described in combination with a specific implementation scenario.

[0095] In the first embodiment of the present application, a full-size humanoid robot is taken as an example. As shown in the figure, the full-size humanoid robot includes a classification machine, a task machine, and a process state machine. Among them, the classification machine may be, for example, rockai_mission_idf, the task machine may be, for example, rockai_mission_transfer, and the process state machine may be, for example, rockai_mission_sm. In addition, the robot also includes a voice interaction machine, or can be called a voice interaction module, and the voice interaction machine may be, for example, rockai_audio. Figure 5

[0096] Among them, rockai_audio runs normally, can trigger rockai_mission_idf, rockai_mission_idf passes the task to rockai_mission_transfer, and rockai_mission_transfer triggers the core process state machine node rockai_mission_sm; rockai_mission_sm feeds back completion to rockai_mission_transfer after completing the task, and also triggers rockai_audio, thereby forming an interaction cycle between nodes to guarantee the linkage operation of the task and the voice interaction function.

[0097] ​The Agent tasks performed by the robot are, for example, tasks such as playing audio while walking in a museum tour scenario, navigation, pointing to play explanation audio, performing arm actions, and the like. For example, the aforementioned target tasks are Agent tasks (i.e., Agent task instructions), which include tasks of types such as navigation (nav), arm action (arm), explanation, check, and none. Correspondingly, different stages (also referred to as different states) in the execution of the tasks can be referred to as task stages (also referred to as task states), which include, for example, a WalkMusic / WalkingMusicCommon stage (state), a Navigation stage (state), an ArmAction stage (state), an Introduction stage (state), a Modeling stage (state), and the like. Each task has a corresponding task module for processing the corresponding task. In addition, each task stage (state) includes multiple task implementation stages, which include, for example, an Idling stage, a Triggering stage, a Running stage, a Done stage, and the like, and can also be referred to as an Idling state, a Triggering state, a Running state, a Done state, and the like. Further, the preset task type detection function can also be referred to as a type marking function. In addition, the large language model can be referred to as a large model for short.

[0098] Further, a flow task refers to a task that is defined in advance in a configuration file and has a fixed sequence and logical relationship, such as a task of a task chain 1→2→3 or a task of setting whether to perform an arm action and an explanation module at point A. It can be understood as the aforementioned task including multiple different types of tasks. A non-flow task refers to a task that is triggered randomly at any time and any place and is not embedded in a fixed task chain in advance. For example, at point A, the user randomly issues a hand waving or explanation instruction, and the two instructions have no certain order.

[0099] For example, the configuration file defines task configuration related information, for example, the starting point of the museum tour guide task is task 1, task ID = 1. The starting point is the welcome area point, at which the robot will execute two sets of arm actions in turn and the fixed explanation audio synchronization subtask. As the starting point, the task will be started directly without playing the walking music. After the task is completed, it is automatically jumped to the next task point, for example, task ID = 2, and the LED light and free question and answer function are not turned on in the process. When the robot leaves this point to go to the next point, the specified walking audio will be played. Among them, the index name of the task in the configuration file can be any, but the index must be a number.

[0100] The specific implementation process of rockai_mission_idf, rockai_mission_transfer and rockai_mission_sm in the task processing method provided in the application embodiment will be introduced in turn.

[0101] Among them, rockai_mission_idf is a classifier node used for sorting tasks, which is suitable for tasks with relatively strong randomness. It is used to filter whether the human robot task is a flow design in the task chain (for example: whether to set whether to go to point A to execute arm action and explanation two modules of the task), and the non-flow task is not in the task chain, which can be triggered at any time and anywhere (for example: going to point A can execute arm action, also can execute explanation task, is random). rockai_mission_idf needs to return what type of Agent task instruction is, classifies the task, and calls the corresponding service or topic according to the different task types after classification. For example: arm action task will be sent to arm, navigation task will be sent to nav, explanation task will be sent to explanation, etc.

[0102] Further, regarding the Agent task instruction, since the task instruction needs to be filtered and classified, the format convention is to put the type in front and the field behind. It has a certain generalization ability, because the result returned by the large model will have multiple different contents for the same type of instruction, which is roughly as follows:

[0103] head / move_head / run_head + rotate_around: task instruction of rotate_around executed by the head module.

[0104] wave / arm_action / hand / arm + good_bye: task instruction of good_bye executed by the arm action module.

[0105] move / run / go / walk / nav + lobby_center: the navigation module executes the task of lobby_center.

[0106] explain / speak / introduce / illustrate + laboratory_room: the explanation module executes the task instruction of laboratory.

[0107] Further, the method for saving the task type of rockai_mission_idf is to first store all types of tasks in the container for detecting the task type, for marking the type and detecting the match. New task types are added in rockai_mission_idf, one type one function, the naming specification is "mark_" + type name, such as "mark_nav". If there are new tasks later, new containers can be created for the task types that need to be detected to quickly add.

[0108] As shown in Figure 6 and Figure 7 , the implementation of task classification of rockai_mission_idf includes the following steps:

[0109] Step S301: parse the Agent task from the rockai_audio large model module and input it to the rockai_mission_idf module to start the subsequent task processing flow.

[0110] Step S302: rockai_mission_idf performs task recognition operation on the input Agent task to extract key information in the task name.

[0111] The key information can be the target point in the navigation task or the action instruction in the arm action task.

[0112] Step S303: rockai_mission_idf calls the type marking function to match and determine the type of the task according to the key information identified in step S302, and generates and returns the task type result.

[0113] For example, as shown in Figure 7 , it is judged in turn whether it matches the arm action class, the head class, the explanation class, the navigation class, etc. The task type matching result is, for example, matching the navigation class and the arm action class.

[0114] Step S304: rockai_mission_idf performs type labeling on the instruction content of the task according to the type result returned in step S303, determines the task category to which the instruction belongs, and forms the type-labeled instruction content.

[0115] Step S305: The type-labeled instruction content is input into rockai_mission_transfer, which is sent to the corresponding rockai_mission_sm for processing according to the task type.

[0116] For example, if it is a navigation task (such as the original task nav_lobby_center), the processed navigation task containing the lobby_center navigation point is generated, marked as "mark_nav", and sent to the flow state machine for processing. If it is an arm action task (such as the original task arm_good_bye), the processed arm action task containing the good_bye action is generated, marked as "mark_arm", and sent to the flow state machine responsible for processing the robot arm communication for processing.

[0117] Further, if no corresponding type is detected or matched in the type detection container, the none type is returned, and in rockai_mission_transfer, it is also determined whether it is an error information. If it is an error information, an error audio is directly played as a task error prompt information and this task is skipped; if not, the next step is detected to determine whether it is an independent task flow or a task logic relationship (next / last / check + area navigation to the next / previous task, check the current task, etc.). If it is detected as the above corresponding type in rockai_mission_transfer, the correct task is determined according to the task logic, and then the task is sent to rockai_mission_sm for task execution by the flow state machine.

[0118] Using the above task processing method, a dedicated type labeling function is configured for each type of task. Thanks to the advantages of modular design, the expansion of type labeling functions becomes convenient and efficient. If new task types need to be added in the future, only the corresponding type labeling function needs to be added. At the same time, the modular characteristics also make problem troubleshooting more convenient. If there is a defect in the screening of a certain type of task, but other types are not affected, only the corresponding type labeling function of this type needs to be focused on for troubleshooting. In addition, the operation of the type labeling function will not interfere with the main running logic, thereby reducing the coupling degree of each module and the main logic.

[0119] About rockai-mission_transfer (hereinafter referred to as mission_transfer), it is used to ensure the sending of tasks, and the management and execution of tasks are separated, so that mission_transfer is used to handle task transmission and logic processing.

[0120] Further, mission_transfer is mainly used to perform the function of issuing tasks. The basic idea is to obtain tasks from different places, process them and send them out. As shown in Figure 8 , the main sources of mission_transfer tasks are three, one is to send the signal of task completion from rockai-mission-sm after the completion of the current task through the ROS topic, and mission-transfer receives the next task topic; one is the initial task sending in the initialization stage (program startup, default startup configuration list task); and the model result Agent string returned by the model received from rockai-audio. Abstractly, the normal tasks obtained from the three parts are then sent through the fixed function interface to implement the task issuing.

[0121] As shown in Figure 9 and Figure 10 , mission_transfer and rockai-mission-sm implement task processing including the following steps:

[0122] Step 401: mission_transfer imports the configuration file, loops through each task and fills it into the corresponding queue, and determines the initial task according to the specified field in the configuration file, and sets it to the corresponding state of rockai-mission-sm after sending.

[0123] Step 402: After the initial task is successfully issued, rockai-mission-sm waits for the next task request; the request source is the callback function of the next task or the Agent task instruction, which are both used to retrieve tasks through the task identifier (such as agent_id) read in the configuration file.

[0124] Among them, agent_id has been set in the configuration file in advance and read by the program.

[0125] Step 403: After retrieving the corresponding task in the configuration file, update the current task first; if the task is repeated, do not operate, if not repeated, mark its attribute as normal.

[0126] If the task is not repeated, the attribute of the task is marked as normal, indicating that the task is a normal task rather than other types of tasks.

[0127] Step 404: The completed task is marked and pushed to the task queue, and the task in the queue will be sent to rockai-mission-sm through a fixed interface; the current task is updated again, and the task is sent to the state machine through the task sending interface, and the task is stored in the sent task queue after being sent.

[0128] Step 405: After the task is sent, the current state of rockai_mission_sm is updated through the rockai_mission_sm network. Whether to play walking audio is determined when the next task is sent, and the walking audio state is sent.

[0129] The processing logic of the walking audio is distinguished according to the following conditions: whether it belongs to the audio of the last logical point, whether it is a direct guide point, and the relationship with the audio of the next logical point. In order to avoid inconsistency with the actual situation, the system divides the walking audio into two states, and distinguishes according to the attribute when the task is sent: the task in the normal task chain enters the WalkingMusic state, and the last task or the direct guide point task enters the WalkingMusicCommon state.

[0130] Further, as Figure 10As shown, the mission_transfer implements the task processing, which further includes that the rockai_audio processes the playing parameter (P_Play) and ends the talking (Endtalking), triggers the end of the talking (Endtalking) and plays the audio (play_wave), which interacts with the cloud server large model for example. The system receives the next task instruction or the agent task instruction, and issues the task to generate the agent task ID (Agent_id). Then, the task retrieval link is entered, if the task is found, the current task is updated; if the task is not found, logical judgment is performed, which involves options such as judging visiting the last area, visiting the next area, visiting the just visited area, visiting the random area, and querying the just visited area, querying the current area, and the like, and then the task position index is updated, the task queue in the configuration file is searched, the current task is updated and the task attribute is marked, and then the task queue is issued. In addition, if it is judged that the current area is queried, the current task is updated. For the issued task queue, the agent task (Agent task) is set as the current task, after the task is issued and enters the sent task queue, it is judged whether the task is the last task, if not, the rockai_mission_sm plays the walking music (WalkingMusic), if yes, the common walking music (WalkingMusicCommon) is played. The issued task queue will make the historical task queue reach a constant number of tasks. Moreover, after the current task is updated, the non-repeated task is popped into the issued task queue, and after the task is sent, the issued task queue is popped out. In order to prevent too many tasks in history, a constant number of historical tasks can be maintained (that is, the capacity of the historical task queue is set, and the specific capacity can be set as needed, for example, 10 or the like), if the number of historical tasks in the historical task queue exceeds the capacity of the historical task queue, the task at the head of the historical task list is deleted, if the number of tasks in the historical task queue does not exceed the capacity of the historical task queue, the new historical task is added. Further, after the task is found and the current task is updated, if it is a non-repeated task, the task attribute is marked and the corresponding subsequent processing is performed, if it is a repeated task, nothing is done.

[0131] Further, if no task recorded in the configuration file is found in the task retrieval process, the system will enter the logical judgment process, which can be divided into two categories: navigation type and search type, which need to be discussed separately.

[0132] Among them, the navigation type task, for example, the logical instruction visit_next_area, the task attribute is marked as normal, the task position index is updated, for example, the position state corresponding to the last task, the position state of the current task, and the like are updated; the corresponding task in the configuration queue is re-searched, and the subsequent process is consistent with the search task.

[0133] Further, the task logic instruction of the navigation class in the logical judgment block in the Figure 10 The task logic instruction of the navigation class in the logical judgment block in the

[0134] The task logic instruction of the navigation class in the logical judgment block in the

[0135] Further, the task logic instruction of the navigation class in the logical judgment block in the Figure 10 The task logic instruction of the navigation class in the logical judgment block in the

[0136] Further, the mission_transfer queue management and fault tolerance mechanism includes: constant task quantity limitation and repetition detection.

[0137] The constant task quantity limitation is configured by the size of the historical task queue (default 10), and the earliest task is automatically deleted when it exceeds, so as to avoid historical task accumulation.

[0138] Need to explain, the historical task queue records all the tasks executed, and the number is limited, the tasks that need to be executed will not be limited by the mechanism. That is, only the historical task queue will be limited, and the historical task queue and the tasks that need to be executed are two different task queues.

[0139] Further, the constant task quantity limit means that the historical task queue mentioned above will store the sent tasks for easy finding and repairing; if there are many tasks, it will affect the number of this queue, and even may cause fullness and system lag. In order to prevent this situation from happening, the number of the task queue will be limited, and the number can be adjusted, and the default is 10. If there are more than 10 tasks, the task queue will automatically add the following tasks and delete the first task.

[0140] Repetitive detection, filtering and deduplication of the sent task queue, for example [1,2,3,3] → [1,2,3]; maintain historical task records such as [1,5,3,4], support query current task, last task and other context logic.

[0141] Further, the repetitive detection is to filter the task queue that has been sent. If there is a repeated task, it will not be added, for example, [1, 2, 3, 3] will become [1, 2, 3], where the task name 123 refers to the task index in the configuration list, that is, the first, second and third tasks in the task list; if there is no repeated task, add a new task, such as [1,2,3] adding [5] to [1,2,3,5]. The significance of the already sent task queue is to detect historical tasks, such as [1, 5, 3, 4], the current task is 4, and the last task is 3, so it will not affect other tasks. If you need the last task or the current task, whether it is navigation or detection, you can use the help of this queue to realize the corresponding logic.

[0142] About rockai_mission_sm, it is used to focus on the implementation of the task flow. That is, the Agent task returned by the large model interface is packaged and packaged according to the ROS format and sent to rockai_mission_sm for execution to execute the logical task sent by the task machine.

[0143] For example Figure 11As shown, the overall task of rockai_mission_sm starts after the robot receives the task. At the starting point, it performs a task to play a fixed audio narration while simultaneously executing arm movements. Both arm movements and the fixed audio narration end simultaneously. After this, it enters the next stage, where navigation and walking audio playback are performed concurrently. Once the target location is reached, arm movements and fixed audio narration are executed again simultaneously. By default, tasks are performed in ascending order of task IDs in the configuration file, starting from 0, with -1 indicating a task is stopped. After each execution of tasks such as arm movements and fixed audio narration, the robot is asked whether there is any audio for interaction with the large model and whether a question-and-answer mode is set. If so, interaction is required, and a free-response question-and-answer mode is used. If not set, the robot defaults to proceeding to the next task in the configuration. Outside the starting point, the navigation task must be executed first, followed by the arm movements and audio narration. This is because at the starting point, the walking audio narration is not played by default.

[0144] Among them, such as Figure 12 As shown, regarding the WalkMusic phase, the task issued by mission_transfer directly enters the Triggering phase of WalkMusic. This phase performs two operations: first, it calls the play_wave module of rockai_audio to play the fixed audio carried in the task field; second, without waiting for the audio playback to complete, it directly switches to the Triggering phase of Navigation, realizing the synchronization of walking audio playback and navigation task issuance.

[0145] Furthermore, the difference between WalkingMusic and WalkingMusicCommon lies in their audio formats. WalkingMusic plays the walking audio read from the configuration file, which is dynamically loaded and used for the next sequential task (because the walking audio in the configuration file has a certain logical order, not a free one). WalkingMusicCommon, on the other hand, plays the guided tour task from the previous point or the direct route task to a specific point. The audio here is fixed and universal, without any logical positional relationship. The reason for specifically creating a separate stage is to avoid future additions disrupting the existing logic. All direct routes with no logical relationship are placed in this new stage, separated from the normal task logic, ensuring that the two stages are independent and do not affect each other. If adjustments to the direct route stage or the normal sequential logic are needed later, these can be directly performed within the stage, achieving a certain degree of scalability.

[0146] like Figure 13As shown, about Navigation stage, after the system starts, rockai_mission_sm enters Idling stage (empty operation, only to ensure program integrity and subsequent task preparation) by default until it is triggered by the Triggering stage of WalkMusic / WalkMusicCommon. It directly jumps from the Triggering stage of WalkingMusic / WalkingMusicCommon to the Triggering stage of Navigation, and then judges in the Triggering stage of Navigation whether it is a starting point task. If it is a starting point task, it does not need to send a navigation task, in order to reflect the natural gait of the robot walking like a person, so it navigates first and then explains; it directly enters the Triggering stage of ArmAction because the robot needs to ensure that the explanation and arm action are executed at the same time. Therefore, the arm action instruction is sent first, and then the explanation audio is played. The sending of the arm action instruction is instantaneous, meaning that the robot side service receives it immediately and responds within a second, which is non-blocking. If it is not a starting point task, it is sent as a navigation instruction client to the navigation instruction service end of the robot end (completed by the robot hardware algorithm), and at the same time the stage is switched to the Running stage, indicating that it will not return to the Triggering stage, avoiding pollution of the Triggering stage and affecting the scalability. After the navigation action is completed, the corresponding message is sent to the navigation action topic, and then the topic receiver in rockai_mission_sm will switch the stage (ROS topic sending and receiving mechanism) to Done according to the received navigation action topic message, and at the same time the task flow stage in rockai_mission_sm is switched to ArmAction as Triggering.

[0147] As Figure 14As shown, regarding the ArmAction phase, it needs to be combined with the Introduction phase to achieve the simultaneous execution of arm action and explanation audio. The flow of the task is to start playing the explanation audio task in the Introduction after the ArmAction performs the arm action task, and return to the ArmAction to perform the arm action task. Regardless of whether it comes from the Triggering of the Navigation or the Done phase of the ArmAction (after the completion of the previous arm action task), it will first judge whether the arm action task queue is empty. If it is empty, nothing will be done, and it will switch to the Done phase and wait for the completion of the explanation audio task by default. If the arm action task queue is not empty, the arm action instruction in the queue will be sent to the arm action service of the robot, and then it will switch to the Triggering phase of the Introduction. It will again judge whether the explanation audio queue is empty, but this is a matter of the Introduction phase. After the completion of the action of the arm action service of the robot, the arm action topic subscriber receives the message and directly switches to the Done phase of the ArmAction. Because the arm action of the robot is received through the service end, but the return after the completion of the task is realized through the topic. Repeat this flow, and again judge whether the arm action task queue is empty until the arm action task queue is empty.

[0148] As shown in Figure 15 regarding the Introduction phase, from the Triggering of the ArmAction, the arm action task queue first judges whether it is empty. If it is not empty, it will switch to the Triggering phase of the Introduction after the completion of the sending of the arm action instruction. In this phase, it will first judge whether the explanation audio task queue is empty. If it is empty, it means that there is no explanation at the current task point. If it is not empty, it will send the audio name that needs to be explained to the play_wave module to play the fixed explanation audio, and switch the phase to the Running phase of the Introduction. After the completion of the playing of the explanation audio in the play_wave module, it will go to the Done phase of the Introduction, and then go to the Triggering phase of the Modeling to wait for the interaction with the large model.

[0149] As shown in Figure 16As shown, regarding the Modeling phase, from the Done phase of the Introduction, switch to the Triggering phase of the Modeling, first determine whether to use a large model access in the configuration file, if yes, send the audio to be processed to the cloud model processing, at the same time switch to the Speaking phase in the free conversation (F_Talking) phase of the rockai_audio (voice interaction module), communicate with the large model through the interface of the play_wave module; if not, directly switch to the NoTalking phase in the Idle of the rockai_audio, and publish the next task (the information containing the next task in the current task point in the configuration file) through the ROS topic, receive and publish the corresponding next task point in the task machine to form a task chain.

[0150] Further, the task chain mechanism, if the current task point, for example, 1 point, is associated with the next task point, for example, 2 point, then after completing the current task, publish the next point information; the task machine node mission_transfer receives, obtains the next point task from the configuration file and issues, forms a task chain, for example, 1→2→3 or 1→3→5→2→4. If the next task in the configuration file is -1, the current task is completed, and the task is stopped.

[0151] Using the above task processing method, three modules are created under ROS according to self-development, which are classification machine, task machine and process state machine. According to the different Agent instructions returned from the model end of the voice interaction module, the classification machine and the task machine are entered in turn for preprocessing and parsing into the ROS message type in the framework, and then the parsed task process is sent to the process state machine. Here, the processing and execution of the task are decoupled by using the pipeline form, and the general technology is implemented together in this part. Among them, the role of the classification machine is to determine whether the current Agent task is a process task. For part of the non-process task, the task needs to be classified in the classification machine first, and then the task type is determined, and then transmitted to the process state machine for execution; and for the process task, it needs to be preprocessed in the task machine, which includes task logic switching, task order adjustment and other functions. But the task in the process state machine is the processed task, which will not process the logic of the task again, and only executes the process of the task.

[0152] The modular design facilitates the execution of logic or sequence of tasks with different requirements in the later stage, keeps the interface unchanged, and if the requirement changes less, the modification can be made in each module without affecting other parts of the process; if the requirement changes more, the whole process can be replaced directly, or if any module is wrong, only the debugging needs to be done in the separate module. Each module only focuses on its own function and is only responsible for the part it needs to be responsible for, such as the task machine only needs to handle the task process, the classification machine only needs to focus on the classification result and classification algorithm of the task, and the process state machine only needs to focus on the execution logic of the task itself, without needing to focus on the logical processing part of the task.

[0153] In addition, the task machine, the classification machine, and the process state machine are considered in the setting of their scalability; the task machine and the classification machine can increase the adaptability to different tasks by quickly increasing or decreasing the types of tasks (when adding or reducing tasks, only the corresponding module function needs to be ensured to have no effect on other code parts); the process state machine also retains a certain scalability under the clear task execution process (equivalent to the process that needs to be adjusted is implemented by modifying the state / phase); in addition, if there is a new requirement, the scalability can be met by directly creating a new state / phase. In the process state machine, the stage of interacting with the large model can determine whether to start the function of interacting with the large model according to the configuration of the corresponding task. This function can directly switch the state of the task process state machine node and the voice interaction process node, transmit the audio through the model interface for subsequent processing, and return the result of the model interface to the task process state machine for the next task start. At the same time, the process state machine will switch the state to the communication end state, which means that there is not only a sequence relationship between some nodes, but also a chain relationship.

[0154] In summary, in the ROS environment, the task processing scheme is constructed based on the three modules of the classification machine, the task machine and the process state machine. After receiving the model end Agent instruction transmitted by the voice interaction module, the task processing and task execution are decoupled through the pipeline form. First, the classification machine determines whether the task is a process task. Non-process tasks are directly transmitted into the process state machine after the type is determined by classification. Process tasks are preprocessed by the task machine, such as logical switching and sequence adjustment. Finally, only the processed task process is executed by the process state machine. The design has the advantages of modularization and easy maintenance. The module interface is fixed. When the demand changes, the single module can be modified or the whole process can be replaced. The single module error only needs to be debugged alone, and each module focuses on its own function. At the same time, it has high scalability. The task machine / classification machine can add or reduce task types by modifying the corresponding module function. The process state machine can adjust the existing state / stage or add a new state / stage. It can also determine whether to start the large model interaction function according to the task configuration. In addition, the process state machine and the voice interaction process node support state communication. When interaction is needed, switch the state to transmit the audio to the model interface. The model returns the result to trigger the next round of tasks and switch to the communication end state. The nodes form a sequence and chain linkage.

[0155] Further, in some other embodiments of the present application, the large language model can also be located in other places other than the server, and the robot can also be other types of robots.

[0156] The embodiments of the present application provide a robot, such as Figure 17 As shown, comprising: a memory 122, for storing a computer program, the computer program comprising program instructions; a processor 121, for executing program instructions. The processor 121 executes the computer execution instructions stored in the memory, so that the processor 121 executes the task processing method in the above-mentioned embodiments, so that the robot realizes the task processing method in the above-mentioned embodiments.

[0157] The processor 121 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0158] The memory 122 is connected with the processor 121 through the system bus and completes mutual communication, and the memory 122 is used for storing computer program instructions.

[0159] The embodiment of the application provides a task processing system, comprising a server and a robot, wherein the robot comprises a classification machine, a task machine and a process state machine, and a large language model is integrated in the server.

[0160] The server is used for generating a target task through the large language model and sending the target task to the robot.

[0161] The robot is used for receiving the target task through the classification machine, determining whether the target task is a process task, sending task information corresponding to the target task to the task machine in the case that the target task is determined to be the process task, determining a task type corresponding to the target task in the case that the target task is determined to be a non-process task, and sending the task information corresponding to the target task to the process state machine according to the task type; the task machine is used for pre-processing the task information corresponding to the target task in the case that the task information corresponding to the target task is received, and sending the pre-processed task information corresponding to the target task to the process state machine; and the process state machine is used for executing a task process corresponding to the target task in the case that the task information corresponding to the target task is received.

[0162] The embodiment of the application provides a computer program product comprising instructions, which, when executed by the robot, cause the robot to perform the task processing method.

[0163] The embodiment of the application provides a computer readable storage medium comprising computer program instructions, which, when executed by the robot, cause the robot to perform the task processing method.

[0164] It should be noted that in addition to the specific embodiments of the application described above, other advantages and effects of the application can be easily understood by those skilled in the art from the disclosure. Although the description of the application is introduced in combination with the preferred embodiments, it does not mean that the features of the application are limited to the embodiments. On the contrary, the purpose of introducing the application in combination with the embodiments is to cover other options or modifications that can be extended based on the claims of the application. In order to provide a deep understanding of the application, many specific details are included in the above description, and the application can also be implemented without using these details. In addition, in order to avoid confusion or obscure the focus of the application, some specific details will be omitted in the description. It should be noted that the embodiments and features in the embodiments of the application can be combined with each other without conflict.

[0165] It should be noted that in this specification similar reference numerals and letters indicate similar items, and thus, once an item is defined in one figure, it should not require further defining and explaining in subsequent figures.

[0166] It should be noted that the terms "first", "second", and so on, are used herein only to distinguish one item from another, and do not imply or suggest relative importance.

[0167] It should be noted that in the drawings, some structural or methodological features can be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order can not be required. Rather, in some embodiments, these features can be arranged in a manner different from that shown in the illustrative drawings. Additionally, inclusion of structural or methodological features in a particular figure is not meant to imply that such features are required in all embodiments, and in some embodiments, these features can not be included or can be combined with other features.

[0168] While the application has been illustrated and described in detail in the drawings and foregoing description, the same is to be considered as illustrative and not restrictive in character, since the application includes all modifications and equivalents falling within the spirit and scope of the application. Accordingly, the application is not limited to the precise embodiments described herein, and variations other than those described herein can be accepted as falling within the scope of the application.

Claims

1. A task processing method characterized by, The method is applied to a robot including a classification machine, a task machine and a process state machine, and comprises the following steps: The classification machine determines whether a target task is a process task, sends task information corresponding to the target task to the task machine if it is determined that the target task is the process task, determines a task type corresponding to the target task, sends task information corresponding to the target task to the process state machine according to the task type, sends corresponding notification information to the task machine if it is unable to determine whether the target task is the process task, the target task is an agent task sent by a large language model corresponding to the robot, the large language model is integrated in a server corresponding to the robot, the process task is a task including multiple tasks of different types, and is a task chain task defined in advance in a configuration file, having a fixed sequence and logical relationship, and is a task sequence composed of multiple tasks of different types, the multiple tasks of different types corresponding to the task sequence need to be executed in turn according to the sequence of the task types, each task in the process task is regarded as a separate execution unit or as a whole, the non-process task is a task including only one task of one type, and is a task triggered randomly at any time and anywhere without being embedded in a fixed task chain in advance, a container for storing a detection task type is preconfigured in the classification machine, one task type corresponds to one container for storing a corresponding task type detection function, each task type detection function is used for detection matching processing of a corresponding task type, and determination of whether the target task is a process task comprises the following steps: calling the task type detection function stored in the container corresponding to each task type, and performing detection matching processing on the target task in turn through task type identification information in task configuration information included in the target task, to determine whether the target task is the process task, wherein if multiple tasks of different types are matched in the target task, it is determined that the target task is the process task, if only one task of one type is matched in the target task, it is determined that the target task is not the process task, and if the type of the task included in the target task is not matched, it is determined that it is unable to determine whether the target task is the process task. The task machine pre-processes the task information corresponding to the target task to make the task information corresponding to the target task meet the task execution requirement corresponding to the process task, and sends the pre-processed task information corresponding to the target task to the process state machine, and in the case of receiving the notification information, determines whether the target task is error information in response to the notification information, does not perform the task processing corresponding to the target task in the case of determining that the target task is error information, and makes the robot present task error prompt information, and determines the task type corresponding to the target task in the case of determining that the target task is non-error information, and sends the task information corresponding to the target task to the process state machine according to the task type; The process state machine executes the task process corresponding to the target task in the case of receiving the task information corresponding to the target task, wherein if the target task is the process task, the process state machine executes each task in turn according to the task execution order based on the multiple tasks of different types included in the target task, until each task included in the target task is executed; if the target task is not the process task, the process state machine executes one task of one type included in the target task.

2. The method of claim 1, wherein, The preprocessing is a task information processing based on a robot operating system, and the task information processing includes task execution logic switching processing, task execution order adjustment processing, encapsulation and packaging processing.

3. The method of claim 2, wherein, The process state machine includes multiple task modules, and the task machine sends the pre-processed task information corresponding to the target task to the process state machine, including: The task machine sends the task information corresponding to each task in turn to the corresponding task module in the process state machine according to the multiple tasks of different types included in the target task.

4. The method of claim 3, wherein, The classification machine sends the task information corresponding to the target task to the process state machine according to the task type, including: The classification machine calls the corresponding service or topic according to the task type to send the task information corresponding to the target task to the corresponding task module in the process state machine.

5. The method of claim 4, wherein, The process state machine executes each task in turn according to the task execution order, including: After executing a preset task in the task, the process state machine determines whether there is an event of interacting with the large language model, uses a free question and answer mode to realize interaction with the large language model in the case of determining that there is an event of interacting with the large language model, and executes the next task in the case of determining that there is no event of interacting with the large language model.

6. The method of claim 5, wherein, Each task includes multiple task implementation stages, and the multiple task implementation stages include an idle stage, a triggered task stage, a task running waiting stage, and a task completion stage, and the process state machine executes each task, including: The process state machine executes each task according to the multiple task implementation stages.

7. A robot, characterized in that including: a memory for storing a computer program, the computer program comprising program instructions; a processor for executing the program instructions to cause the robot to perform the task processing method according to any one of claims 1-6.

Images