Task planning method and device, equipment, storage medium and program product

Abstract

The invention relates to a task planning method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring voice instruction information input by a user for a robot; obtaining at least one piece of task environment information of the robot, wherein the task environment information is used for representing environment characteristics of the current position of the robot; identifying the voice instruction information, determining a target task executed by the robot indicated by the voice instruction information, and decomposing the target task into at least one subtask; calculating task influence parameters of the at least one piece of task environment information on the associated sub-tasks; and determining a task execution sequence of each sub-task in the target task according to the task influence parameter. By adopting the method, the execution effect of the complex instruction can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a task planning method, device, equipment, storage medium and program product. Background Art

[0002] With the development of artificial intelligence technology, the application of robots in different scenarios has gradually increased, especially in service, education, medical and other scenarios, where robots usually need to execute relatively complex instructions. Due to the complexity of the instructions, the robot can generate multiple tasks for the instruction and plan the execution order of the task so that each task can be completed in sequence later.

[0003] In the related art, the tasks corresponding to each instruction and the execution order of each task can be pre-set, and then each task is executed according to the pre-set execution order. However, since the task environment is not static, when the task environment changes, the original task execution order corresponding to each instruction is still used, which may conflict with the current task environment, thereby resulting in poor execution effect of complex instructions. Summary of the invention

[0004] Based on this, it is necessary to provide a task planning method, device, equipment, storage medium and program product that can improve the execution effect of complex instructions in response to the above technical problems.

[0005] In a first aspect, the present application provides a task planning method, comprising:

[0006] Obtaining voice command information input by the user for the robot;

[0007] Acquire at least one task environment information of the robot, where the task environment information is used to characterize the environmental characteristics of the current location of the robot;

[0008] Recognizing the voice command information, determining a target task that the voice command information instructs the robot to perform, and decomposing the target task into at least one subtask;

[0009] Determine task impact parameters of the at least one task environment information on the associated subtasks respectively;

[0010] The task execution order of each subtask in the target task is determined according to the task impact parameter.

[0011] In one embodiment, determining the task execution order of each subtask in the target task according to the task impact parameter includes:

[0012] Obtain the task type of each subtask and the feature type of each task environment information;

[0013] Determining the weight of the task influencing parameter according to the task type and the feature type;

[0014] According to the weight of the task impact parameter, weighted calculation is performed on the task impact parameter of each subtask to obtain a weighted ranking value of each subtask;

[0015] The task execution order of each subtask in the target task is determined according to the weighted ranking values ​​of each subtask.

[0016] In one embodiment, determining the task execution order of each subtask in the target task according to the weighted ranking value of each subtask includes:

[0017] Obtaining the preconfigured initial execution order of each subtask in the target task;

[0018] Determining, according to the execution order restriction conditions between the subtasks, the non-fixed order subtasks in the subtasks and the execution order adjustment range of the non-fixed order subtasks;

[0019] According to the weighted ranking values ​​of the subtasks and the initial execution order, the corresponding non-fixed order subtasks are adjusted in order within the execution order adjustment range to determine the task execution order of the subtasks in the target task.

[0020] In one embodiment, decomposing the target task into at least one subtask includes:

[0021] Describing the at least one task environment information by using a visual language model to obtain environment description information of the current location of the robot;

[0022] Using the environment description information as prior information of a large language model and using the large language model to decompose the target task to obtain at least one subtask corresponding to the target task;

[0023] The large language model is used to decompose the target task into multiple skills in the robot skill library according to the prior information, and combine the multiple skills into at least one subtask.

[0024] In one embodiment, the obtaining of at least one task environment information of the robot includes:

[0025] Determine a plurality of environment collection points corresponding to the robot according to the current location of the robot;

[0026] Acquire multi-dimensional environmental data corresponding to the multiple environmental collection points;

[0027] fusing the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the current location of the robot;

[0028] At least one task environment information of the robot is extracted from the point cloud data of the current position of the robot.

[0029] In one of the embodiments, the multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

[0030] In a second aspect, the present application provides a task planning device, comprising:

[0031] An acquisition module, used to acquire voice command information input by a user for the robot; and to acquire at least one task environment information of the robot, wherein the task environment information is used to characterize the environmental characteristics of the current location of the robot;

[0032] A recognition module, configured to recognize the voice instruction information, determine the target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask;

[0033] The determination module is used to determine the task impact parameters of the at least one task environment information on the associated subtasks respectively; and determine the task execution order of each subtask in the target task according to the task impact parameters.

[0034] In one of the embodiments, the determination module is also used to obtain the task type of each subtask and the feature type of each task environment information; determine the weight of the task impact parameter according to the task type and the feature type; perform weighted calculations on the task impact parameters of each subtask according to the weight of the task impact parameter to obtain a weighted ranking value of each subtask; and determine the task execution order of each subtask in the target task according to the weighted ranking value of each subtask.

[0035] In one of the embodiments, the determination module is also used to obtain the pre-configured initial execution order of each subtask in the target task; determine the non-fixed order subtasks in each subtask and the execution order adjustment range of the non-fixed order subtasks based on the execution order constraints between the subtasks; and adjust the order of the corresponding non-fixed order subtasks within the execution order adjustment range based on the weighted ranking values ​​of the subtasks and the initial execution order to determine the task execution order of each subtask in the target task.

[0036] In one embodiment, the recognition module is further used to describe the at least one task environment information through a visual language model to obtain environment description information of the current location of the robot; use the environment description information as prior information of a large language model and use the large language model to decompose the target task to obtain at least one subtask corresponding to the target task;

[0037] The large language model is used to decompose the target task into multiple skills in the robot skill library according to the prior information, and combine the multiple skills into at least one subtask.

[0038] In one of the embodiments, the acquisition module is further used to determine multiple environmental collection points corresponding to the robot according to the current location of the robot; obtain multi-dimensional environmental data corresponding to the multiple environmental collection points; fuse the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the current location of the robot; and extract at least one task environment information of the robot from the point cloud data of the current location of the robot.

[0039] In one of the embodiments, the multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

[0040] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the task planning method of the first aspect when executing the computer program.

[0041] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the task planning method of the first aspect described above is implemented.

[0042] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the task planning method of the first aspect mentioned above.

[0043] The above-mentioned task planning method, device, equipment, storage medium and program product obtain the voice command information input by the user for the robot; obtain at least one task environment information of the robot, the task environment information is used to characterize the environmental characteristics of the current location of the robot; identify the voice command information, determine the target task that the voice command information instructs the robot to perform, and decompose the target task into at least one subtask; calculate the task impact parameters of at least one task environment information on the associated subtasks; determine the task execution order of each subtask in the target task according to the task impact parameters. Since the task execution order of each subtask in the target task is determined according to the task impact parameters of the associated subtasks of the task environment information, it can ensure that the task execution order is compatible with the current task environment, reduce the possibility of conflict between task execution and the current task environment, and thus ensure the execution effect of complex instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flowchart of a task planning method provided in an embodiment of the present application;

[0046] Figure 2 A flowchart of another task planning method provided in an embodiment of the present application;

[0047] Figure 3 A flowchart of another task planning method provided in an embodiment of the present application;

[0048] Figure 4 A flowchart of another task planning method provided in an embodiment of the present application;

[0049] Figure 5 A structural block diagram of a task planning device provided in an embodiment of the present application;

[0050] Figure 6 An internal structure diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] The task planning method provided in the embodiment of the present application can be applied to a robot. The robot can first obtain the voice command information for the robot input by the user, and obtain at least one task environment information of the robot, and the task environment information is used to characterize the environmental characteristics of the current location of the robot. Secondly, the robot can identify the voice command information, determine the target task that the voice command information instructs the robot to perform, and decompose the target task into at least one subtask. Thirdly, the robot can determine the task impact parameters of the at least one task environment information on the associated subtasks respectively. Finally, the robot can determine the task execution order of each subtask in the target task according to the task impact parameters.

[0053] Among them, the robot can be a humanoid robot or other sports robot including movable joints. The robot can include multiple joints and a visual system. The visual system is generally located at the head of the robot and is used to collect images of the external environment. In addition to the visual system, the robot can also be configured with a voice processing system, a question-and-answer knowledge base, and the like. The robot includes multiple joints, which are electrically connected to a drive mechanism to achieve a certain range of rotation. When the joint angles of each joint of the robot change, the robot can complete different posture changes. Accordingly, the robot can be controlled to perform different actions by controlling the joint angles of each joint, thereby achieving the target task indicated by the user.

[0054] In an exemplary embodiment, Figure 1 As shown, a task planning method is provided, and the method is applied to a robot as an example for explanation, including S201-S205:

[0055] S201: Acquire voice command information input by a user for a robot.

[0056] In the present application, the robot can interact with the user to obtain the voice command data input by the user, thereby performing the target task instructed by the user.

[0057] It should be understood that the embodiments of the present application do not limit how to obtain the voice command data input by the user. In some embodiments, a voice processing system may be provided on the robot to obtain the voice command data input by the user through a voice acquisition system.

[0058] Exemplarily, the audio collection component in the speech processing system can be a microphone array, and the microphone array can be one or more. When multiple microphone arrays are provided, the microphone arrays can be provided at different positions of the robot, so as to collect audio data input by the user from different directions of the robot.

[0059] S202: Obtain at least one task environment information of the robot.

[0060] In this step, after obtaining the voice command information input by the user for the robot, the robot can also obtain at least one task environment information of the robot.

[0061] It should be understood that the embodiments of the present application do not limit how to obtain at least one task environment information of the robot. In some embodiments, the robot can first determine multiple environment collection points corresponding to the robot based on the current location of the robot. Secondly, the robot can obtain multi-dimensional environment data corresponding to multiple environment collection points. Thirdly, the robot can fuse the multi-dimensional environment data corresponding to multiple environment collection points into point cloud data of the current location of the robot. Finally, the robot can extract at least one task environment information of the robot from the point cloud data of the current location of the robot.

[0062] The above-mentioned environment collection points can be set around the robot in a variety of ways. For example, a grid can be set around the robot, and an environment collection point can be set at the vertex of each grid point. The size of the grid can be set according to actual conditions. For example, extension lines can be set around the robot at fixed angle intervals, and an environment collection point can be set on each extension line at fixed length intervals.

[0063] It should be understood that the embodiments of the present application do not limit the above-mentioned multi-dimensional environmental data. In some embodiments, the multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

[0064] For example, a radar sensor, an infrared sensor, and a visual sensor may be provided on the robot, wherein the radar sensor may collect radar data, the infrared image data may collect infrared image data, and the visual sensor may collect visual image data. Subsequently, the robot may fuse the multi-dimensional environmental data to obtain point cloud data of the current location of the robot.

[0065] The task environment information is used to characterize the environment characteristics of the current location of the robot. For example, the environment characteristics may be obstacles at a specific location of the robot, a path for the robot to reach the specific location, and the like.

[0066] It should be understood that the embodiments of the present application do not limit how the robot identifies the environmental features of its current location. In some embodiments, the point cloud data of the robot's current location can be input into an environmental recognition model, and the point cloud data can be extracted through the environmental recognition model to accurately identify the environmental features of the robot's current location and obtain at least one task environment information of the robot.

[0067] S203: Identify the voice command information, determine the target task that the voice command information instructs the robot to perform, and decompose the target task into at least one subtask.

[0068] In this step, after the robot obtains the voice command information and the task environment information, it can identify the voice command information, determine the target task that the voice command information instructs the robot to perform, and decompose the target task into at least one subtask.

[0069] In some embodiments, the robot may first convert voice instruction data into text instruction data, then recognize the text instruction data, and determine the target task that the voice instruction information instructs the robot to perform.

[0070] It should be understood that the embodiments of the present application do not limit how to convert voice command data into text command data. In some embodiments, the robot can first filter the voice command data to remove the ambient sound in the voice command data. Subsequently, the robot can input the filtered voice command data into an automatic speech recognition (ASR) model and obtain the phonemes or words contained in the voice command data output by the ASR model. Finally, the robot can decode the phonemes or words contained in the voice command data and form text command data based on the decoding results.

[0071] The ASR model is used to identify acoustic features in the voice command data and match the phonemes or words contained in the voice command data according to the acoustic features. The above acoustic features may include any features that can reflect the voice, such as pitch, intensity, timbre, etc.

[0072] In some embodiments, the robot can extract the user's interest points from the context information corresponding to the user. Then, the robot recognizes the text instruction data according to the user's interest points and determines the user's task intention information. Finally, the robot determines the task instructions included in the target task according to the user's task intention information.

[0073] The context information corresponding to the user may include historical task data that satisfies the user. The historical task data may include voice instruction data input by the user corresponding to the historical task and feedback data of the user on the task instruction of the historical task.

[0074] Among them, the above-mentioned points of interest can be the types of instructions that frequently appear in the context information corresponding to the user, and the points of interest can include the actions of the robot, the direction of the robot's question and answer, etc.

[0075] Exemplarily, after the robot extracts the user's points of interest from the context information, it can output the user's points of interest together with the text instruction data into the semantic understanding model, and obtain the task intention information inferred by the semantic understanding model.

[0076] Among them, the semantic understanding model can analyze the text content of the text instruction data with reference to the user's interest points, infer or infer the intention or meaning behind the text, and thus obtain task intention information. Exemplarily, the above-mentioned semantic understanding model can include a bag-of-words model, a term frequency-inverse document frequency (TF-IDF) model, a word vector (Word to Vector, Word2Vec) model, a bidirectional encoder representation from transformers (BERT) model, a generative pretrained transformer (GPT) model, a graph neural network model, etc.

[0077] The following describes how to decompose the target task into at least one subtask.

[0078] In some embodiments, the robot may first describe at least one task environment information through a visual language model to obtain the environment description information of the robot's current location. Subsequently, the robot may use the environment description information as prior information of the large language model and use the large language model to decompose the target task to obtain at least one subtask corresponding to the target task.

[0079] Among them, the large language model is used to decompose the target task into multiple skills in the robot skill library based on prior information, and combine multiple skills into at least one subtask.

[0080] It should be understood that the target task mentioned above may be any task executable by the robot, and the subtasks mentioned above may be multiple task execution stages decomposed to complete the target.

[0081] For example, if the target task is to pour water from a cup into a bucket, the generated subtasks may include grabbing the cup, identifying a target bucket for pouring water from multiple buckets, planning a path to the target bucket, moving to the target bucket, pouring water into the target bucket, and placing the cup at a designated location.

[0082] Exemplarily, each subtask may also include multiple skills that the robot can complete. For example, if the subtask is to move to a target bucket, the skills included therein may include moving from an initial point in a first direction to a first point, turning from the first point to a second direction, and moving from the first point in the second direction to a target point where the target bucket is located.

[0083] In some embodiments, since the environmental features corresponding to the task environment information may be information such as obstacles at a specific position of the robot, the path of the robot to reach a specific position, etc., the robot can describe at least one task environment information through a visual language model, and can obtain environmental description information such as obstacle avoidance information and path planning information.

[0084] Correspondingly, after obtaining the environment description information, the robot can use the environment description information as the prior information of the large language model, that is, use the environment description information such as obstacle avoidance information and path planning information as the decomposition condition of the target task, and gradually decompose each step of completing the target task, so as to obtain at least one subtask corresponding to the target task.

[0085] In some embodiments, the large language model can decompose the target task into multiple sets of subtasks. Subsequently, the robot can evaluate the multiple sets of subtasks respectively, determine the evaluation parameters of each set of subtasks corresponding to the target task, and then select the subtask with the highest evaluation parameter from the multiple sets of subtasks as the subtask of the target task finally decomposed.

[0086] Among them, the above-mentioned evaluation parameters can be used to indicate the task execution speed corresponding to the set of tasks, or, can also indicate the task execution difficulty corresponding to the set of tasks, or, can also indicate the task execution fault tolerance corresponding to the set of tasks, and the embodiments of the present application do not limit this.

[0087] S204: Determine task impact parameters of at least one task environment information on the associated subtasks.

[0088] In this step, after the robot determines the target task that the voice command information instructs the robot to perform and decomposes the target task into at least one subtask, it can determine the task impact parameters of at least one task environment information on the associated subtasks.

[0089] It should be understood that in the embodiment of the present application, not all task environment information is associated with each subtask. Therefore, before calculating the task impact parameters of the task environment information on the subtasks respectively, the robot can first determine the association relationship between the task environment information and the subtasks.

[0090] For example, the robot can confirm the association between the task environment information and the subtask from the spatial dimension. That is, the robot can estimate the completion position of each subtask, and if the completion position of the subtask is within the influence range of the environmental feature corresponding to the task environment information, it can be determined that the task environment information and the subtask are associated.

[0091] In some embodiments, the robot determines the task impact parameters of at least one task environment information on the associated subtasks through a large language model. The large language model can evaluate the impact coefficient of each task environment information on the task completion time and the impact coefficient of the task completion difficulty of the associated subtask, and then combine the impact coefficient of the task completion time and the impact coefficient of the task completion difficulty to obtain the task impact parameters of the task environment information on the associated subtasks.

[0092] For example, in the process of completing subtask A, it is necessary to bypass the obstacle indicated by task environment information 1. Since the space blocked by the obstacle is large, the robot needs to spend 10 seconds to bypass the obstacle during the completion of subtask A, and will not encounter additional complex road conditions during the bypass. Therefore, the large language model can evaluate the impact coefficient of completion time as 0.6 and the impact coefficient of task completion difficulty as 0.1. The large language model multiplies the impact coefficient of completion time and the impact coefficient of task completion difficulty, and obtains the task impact parameter of task environment information 1 on subtask A as 0.06.

[0093] S205: Determine the task execution order of each subtask in the target task according to the task impact parameter.

[0094] In this step, after determining the task impact parameters of at least one task environment information on the associated subtasks, the robot may determine the task execution order of each subtask in the target task according to the task impact parameters.

[0095] In some embodiments, the robot may first obtain the task type of each subtask and the feature type of each task environment information, and then determine the weight of the task impact parameter according to the task type and feature type. Subsequently, the robot performs weighted calculations on the task impact parameters of each subtask according to the weight of the task impact parameter to obtain the weighted ranking value of each subtask. Finally, the robot may determine the task execution order of each subtask in the target task according to the weighted ranking value of each subtask.

[0096] Among them, task types may include question-answering tasks, mobile tasks, action tasks, etc., and feature types may include obstacle features, path features, etc. Different task types and feature types have different weights on task influencing parameters.

[0097] Exemplarily, the robot may be pre-set with a combination of task types and feature types, and a mapping relationship with the weights of task influencing parameters. When subtasks need to be sorted, the robot may directly match the weights of each task influencing parameter based on the mapping relationship.

[0098] Exemplarily, since each subtask may include multiple task influencing parameters, after the robot determines the weights of the various task influencing parameters, all task influencing parameters associated with each subtask may be weightedly calculated using the weights to obtain a weighted ranking value for subtask ranking.

[0099] It should be understood that the embodiments of the present application do not limit how to determine the task execution order of each subtask in the target task according to the weighted ranking value of each subtask. In some embodiments, the robot can first obtain the initial execution order of each subtask in the preconfigured target task. Subsequently, the robot can determine the non-fixed order subtasks in each subtask and the execution order adjustment range of the non-fixed order subtasks according to the execution order constraints between the subtasks. Finally, the robot can adjust the order of the corresponding non-fixed order subtasks within the execution order adjustment range according to the weighted ranking value and initial execution order of each subtask to determine the task execution order of each subtask in the target task.

[0100] The preconfigured initial execution order may be a subtask execution order that does not consider environmental influence factors.

[0101] It should be understood that the execution order constraint condition can be a sequence condition of the internal execution logic. For example, if the target task is to pour the water in the cup into the bucket, the subtasks include grabbing the cup, identifying the target bucket for pouring water from multiple buckets, planning the path to the target bucket, moving to the target bucket, pouring water into the target bucket, and placing the cup after pouring water at a designated location. Among them, the execution order constraint condition can include that the subtask "grabbing the cup" needs to be restricted before the subtask "moving to the target bucket", and the subtask "placing the cup after pouring water at a designated location" needs to be restricted after the subtask "pouring water into the target bucket".

[0102] Exemplarily, after determining the non-fixed order subtasks in each subtask and the execution order adjustment range of the non-fixed order subtasks through execution order constraints, the execution order of the non-fixed order subtasks in the initial execution order can be adjusted according to the size of the weighted ranking value of each subtask, thereby determining the final task execution order of each subtask in the target task.

[0103] In some embodiments, when the scene changes, the robot will also synchronize the previously executed subtasks together with the modified task environment information to the large language model, so that the large language model reselects the remaining subtasks based on the executed subtasks and the modified task environment information to generate tasks. Similarly, the task impact parameters can be determined again, and the task execution order of each subtask in the regenerated task can be determined based on the task impact parameters.

[0104] The task planning method provided in the embodiment of the present application obtains the voice command information for the robot input by the user; obtains at least one task environment information of the robot, and the task environment information is used to characterize the environmental characteristics of the current location of the robot; identifies the voice command information, determines the target task that the voice command information instructs the robot to perform, and decomposes the target task into at least one subtask; calculates the task impact parameters of at least one task environment information on the associated subtasks; and determines the task execution order of each subtask in the target task according to the task impact parameters. Since the task execution order of each subtask in the target task is determined according to the task impact parameters of the associated subtasks of the task environment information, it can ensure that the task execution order is compatible with the current task environment, reduce the possibility of conflict between task execution and the current task environment, and thus ensure the execution effect of complex instructions.

[0105] The following describes how to obtain at least one task environment information of the robot. Figure 2 A flowchart of another task planning method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the task planning method includes S301-S308:

[0106] S301: Acquire voice command information input by a user for a robot.

[0107] S302: Determine multiple environment collection points corresponding to the robot according to the current location of the robot.

[0108] The above-mentioned environment collection points can be set around the robot in a variety of ways. For example, a grid can be set around the robot, and an environment collection point can be set at the vertex of each grid point. The size of the grid can be set according to actual conditions. For example, extension lines can be set around the robot at fixed angle intervals, and an environment collection point can be set on each extension line at fixed length intervals.

[0109] S303: Acquire multi-dimensional environmental data corresponding to multiple environmental collection points.

[0110] The multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

[0111] For example, a radar sensor, an infrared sensor, and a visual sensor may be provided on the robot, wherein the radar sensor may collect radar data, the infrared image data may collect infrared image data, and the visual sensor may collect visual image data. Subsequently, the robot may fuse the multi-dimensional environmental data to obtain point cloud data of the current location of the robot.

[0112] S304: Fusing the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the current location of the robot.

[0113] S305: Extract at least one task environment information of the robot from the point cloud data of the current location of the robot.

[0114] The task environment information is used to characterize the environment characteristics of the current location of the robot. For example, the environment characteristics may be obstacles at a specific location of the robot, a path for the robot to reach the specific location, and the like.

[0115] It should be understood that the embodiments of the present application do not limit how the robot identifies the environmental features of its current location. In some embodiments, the point cloud data of the robot's current location can be input into an environmental recognition model, and the point cloud data can be extracted through the environmental recognition model to accurately identify the environmental features of the robot's current location and obtain at least one task environment information of the robot.

[0116] S306: Identify the voice instruction information, determine the target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask.

[0117] S307: Determine task impact parameters of at least one task environment information on the associated subtasks.

[0118] S308: Determine the task execution order of each subtask in the target task according to the task impact parameter.

[0119] The following describes how to decompose the target task into at least one subtask. Figure 3 A flowchart of another task planning method provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the task planning method includes S401-S407:

[0120] S401: Acquire voice command information input by a user for a robot.

[0121] S402: Acquire at least one task environment information of the robot, where the task environment information is used to characterize the environmental characteristics of the current location of the robot.

[0122] S403: Identify the voice command information and determine the target task that the voice command information instructs the robot to perform.

[0123] S404: Describe at least one task environment information through a visual language model to obtain environment description information of the current location of the robot.

[0124] S405: Use the environment description information as prior information of the large language model and use the large language model to decompose the target task to obtain at least one subtask corresponding to the target task.

[0125] Among them, the large language model is used to decompose the target task into multiple skills in the robot skill library based on prior information, and combine multiple skills into at least one subtask.

[0126] It should be understood that the target task mentioned above may be any task executable by the robot, and the subtasks mentioned above may be multiple task execution stages decomposed to complete the target.

[0127] In some embodiments, since the environmental features corresponding to the task environment information may be information such as obstacles at a specific position of the robot, the path of the robot to reach a specific position, etc., the robot can describe at least one task environment information through a visual language model, and can obtain environmental description information such as obstacle avoidance information and path planning information.

[0128] Correspondingly, after obtaining the environment description information, the robot can use the environment description information as the prior information of the large language model, that is, use the environment description information such as obstacle avoidance information and path planning information as the decomposition condition of the target task, and gradually decompose each step of completing the target task, so as to obtain at least one subtask corresponding to the target task.

[0129] S406: Determine task impact parameters of at least one task environment information on the associated subtasks.

[0130] S407: Determine the task execution order of each subtask in the target task according to the task impact parameter.

[0131] The following describes how to determine the task execution order of each subtask in the target task. Figure 4 A flowchart of another task planning method provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the task planning method includes S501-S508:

[0132] S501: Acquire voice command information input by a user for a robot.

[0133] S502: Acquire at least one task environment information of the robot, where the task environment information is used to characterize the environmental characteristics of the current location of the robot.

[0134] S503: Identify the voice instruction information, determine the target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask.

[0135] S504: Determine task impact parameters of at least one task environment information on the associated subtasks.

[0136] S505: Obtain the task type of each subtask and the feature type of each task environment information.

[0137] S506: Determine the weight of the task influencing parameter according to the task type and feature type.

[0138] S507 . Perform weighted calculation on the task impact parameters of each subtask according to the weight of the task impact parameter to obtain a weighted ranking value of each subtask.

[0139] S508: Determine the task execution order of each subtask in the target task according to the weighted ranking value of each subtask.

[0140] The task planning method provided in the embodiment of the present application obtains the voice command information for the robot input by the user; obtains at least one task environment information of the robot, and the task environment information is used to characterize the environmental characteristics of the current location of the robot; identifies the voice command information, determines the target task that the voice command information instructs the robot to perform, and decomposes the target task into at least one subtask; calculates the task impact parameters of at least one task environment information on the associated subtasks; and determines the task execution order of each subtask in the target task according to the task impact parameters. Since the task execution order of each subtask in the target task is determined according to the task impact parameters of the associated subtasks of the task environment information, it can ensure that the task execution order is compatible with the current task environment, reduce the possibility of conflict between task execution and the current task environment, and thus ensure the execution effect of complex instructions.

[0141] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0142] Based on the same inventive concept, the embodiment of the present application also provides a task planning device for implementing the task planning method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more task planning device embodiments provided below can refer to the limitations on the task planning method above, and will not be repeated here.

[0143] In an exemplary embodiment, Figure 5 As shown, a task planning device 600 is provided, comprising: an acquisition module 601, an identification module 602 and a determination module 603, wherein:

[0144] The acquisition module 601 is used to acquire the voice command information input by the user for the robot; and acquire at least one task environment information of the robot, where the task environment information is used to characterize the environmental characteristics of the current location of the robot.

[0145] The recognition module 602 is used to recognize the voice instruction information, determine the target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask.

[0146] The determination module 603 is used to determine the task impact parameters of at least one task environment information on the associated subtasks; and determine the task execution order of each subtask in the target task according to the task impact parameters.

[0147] In one embodiment, the determination module 603 is also used to obtain the task type of each subtask and the feature type of each task environment information; determine the weight of the task impact parameter according to the task type and the feature type; perform weighted calculations on the task impact parameters of each subtask according to the weight of the task impact parameter to obtain the weighted ranking value of each subtask; and determine the task execution order of each subtask in the target task according to the weighted ranking value of each subtask.

[0148] In one of the embodiments, the determination module 603 is also used to obtain the initial execution order of each subtask in the preconfigured target task; determine the non-fixed order subtasks in each subtask and the execution order adjustment range of the non-fixed order subtasks based on the execution order constraints between the subtasks; and adjust the order of the corresponding non-fixed order subtasks within the execution order adjustment range based on the weighted ranking value and the initial execution order of each subtask to determine the task execution order of each subtask in the target task.

[0149] In one embodiment, the recognition module 602 is further used to describe at least one task environment information through a visual language model to obtain environment description information of the robot's current location; use the environment description information as prior information of the large language model and use the large language model to decompose the target task to obtain at least one subtask corresponding to the target task.

[0150] Among them, the large language model is used to decompose the target task into multiple skills in the robot skill library based on prior information, and combine multiple skills into at least one subtask.

[0151] In one of the embodiments, the acquisition module 601 is also used to determine multiple environmental collection points corresponding to the robot according to the current location of the robot; obtain multi-dimensional environmental data corresponding to the multiple environmental collection points; fuse the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the current location of the robot; and extract at least one task environment information of the robot from the point cloud data of the current location of the robot.

[0152] In one of the embodiments, the multi-dimensional environment data includes at least one of the following: visual image data, infrared image data, and radar data.

[0153] Each module in the above-mentioned task planning device can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0154] In an exemplary embodiment, a computer device is provided. The computer device is located on a robot, and its internal structure diagram can be as follows: Figure 6As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, a near field communication (NFC) or other technologies. When the computer program is executed by the processor, a task planning method is implemented.

[0155] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0156] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned task planning method when executing the computer program.

[0157] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned task planning method is implemented.

[0158] In one embodiment, a computer program product is provided, including a computer program, which implements the above-mentioned task planning method when executed by a processor.

[0159] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0160] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0161] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0162] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A task planning method, characterized in that: The method comprises: Obtaining voice command information input by the user for the robot; Acquire at least one task environment information of the robot, where the task environment information is used to characterize the environmental characteristics of the current location of the robot; Recognizing the voice command information, determining a target task that the voice command information instructs the robot to perform, and decomposing the target task into at least one subtask; Determine task impact parameters of the at least one task environment information on the associated subtasks respectively; The task execution order of each subtask in the target task is determined according to the task impact parameter.

2. The method according to claim 1, characterized in that The step of determining the task execution order of each subtask in the target task according to the task impact parameter includes: Obtain the task type of each subtask and the feature type of each task environment information; Determining the weight of the task influencing parameter according to the task type and the feature type; According to the weight of the task impact parameter, weighted calculation is performed on the task impact parameter of each subtask to obtain a weighted ranking value of each subtask; The task execution order of each subtask in the target task is determined according to the weighted ranking values ​​of each subtask.

3. The method according to claim 2, characterized in that Determining the task execution order of each subtask in the target task according to the weighted ranking value of each subtask includes: Obtaining the preconfigured initial execution order of each subtask in the target task; Determining, according to the execution order restriction conditions between the subtasks, the non-fixed order subtasks in the subtasks and the execution order adjustment range of the non-fixed order subtasks; According to the weighted ranking values ​​of the subtasks and the initial execution order, the corresponding non-fixed order subtasks are adjusted in order within the execution order adjustment range to determine the task execution order of the subtasks in the target task.

4. The method according to claim 1, characterized in that: Decomposing the target task into at least one subtask includes: Describing the at least one task environment information by using a visual language model to obtain environment description information of the current location of the robot; Using the environment description information as prior information of a large language model and using the large language model to decompose the target task to obtain at least one subtask corresponding to the target task; The large language model is used to decompose the target task into multiple skills in the robot skill library according to the prior information, and combine the multiple skills into at least one subtask.

5. The method according to claim 1, characterized in that The obtaining of at least one task environment information of the robot comprises: Determine a plurality of environment collection points corresponding to the robot according to the current location of the robot; Acquire multi-dimensional environmental data corresponding to the multiple environmental collection points; fusing the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the current location of the robot; At least one task environment information of the robot is extracted from the point cloud data of the current position of the robot.

6. The method according to claim 5, characterized in that The multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

7. A task planning device, characterized in that: The device comprises: An acquisition module, used to acquire voice command information input by a user for the robot; and to acquire at least one task environment information of the robot, wherein the task environment information is used to characterize the environmental characteristics of the current location of the robot; A recognition module, configured to recognize the voice instruction information, determine the target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask; The determination module is used to determine the task impact parameters of the at least one task environment information on the associated subtasks respectively; and determine the task execution order of each subtask in the target task according to the task impact parameters.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images