Mission planning method, device, equipment, storage medium and program product

Abstract

This application relates to a task planning method, apparatus, device, storage medium, and program product. The method includes: obtaining voice command information input by a user for a robot; obtaining at least one task environment information of the robot, the task environment information being used to characterize the environmental characteristics of the robot's current location; recognizing the voice command information, determining the target task the voice command information instructs the robot to perform, and decomposing the target task into at least one subtask; calculating the task impact parameters of the at least one task environment information on the associated subtasks; and determining the execution order of each subtask within the target task based on the task impact parameters. This method can improve the execution of complex instructions.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a task planning method, apparatus, device, storage medium, and program product. Background Art

[0002] With the development of artificial intelligence (AI), the application of robots in various scenarios is increasing. This is particularly true in service, education, and healthcare settings, where robots often need to execute complex instructions. Due to the complexity of these instructions, robots can generate multiple tasks based on the instructions and plan the execution sequence of these tasks so that they can be completed in sequence.

[0003] In related technologies, the tasks corresponding to each instruction and the execution order of each task can be pre-set, and then each task is executed according to the pre-set execution order. However, because the task environment is not static, when the task environment changes, continuing to use the previously set task execution order corresponding to each instruction may conflict with the current task environment, resulting in poor execution of complex instructions. Summary of the Invention

[0004] Based on this, it is necessary to provide a task planning method, device, equipment, storage medium and program product that can improve the execution effect of complex instructions to address the above technical problems.

[0005] In a first aspect, the present application provides a task planning method, comprising:

[0006] Obtain the voice command information input by the user for the robot;

[0007] Acquire at least one task environment information of the robot, where the task environment information is used to characterize environmental characteristics of the current location of the robot;

[0008] Recognizing the voice command information, determining a target task that the voice command information instructs the robot to perform, and decomposing the target task into at least one subtask;

[0009] determining task impact parameters of the at least one task environment information on the associated subtasks;

[0010] The task execution order of each subtask in the target task is determined according to the task impact parameter.

[0011] In one embodiment, determining the task execution order of each subtask in the target task according to the task impact parameter includes:

[0012] Obtain the task type of each subtask and the feature type of each task environment information;

[0013] Determining the weight of the task impact parameter according to the task type and the feature type;

[0014] According to the weight of the task impact parameter, the task impact parameter of each subtask is weighted and calculated to obtain a weighted ranking value of each subtask;

[0015] The task execution order of each subtask in the target task is determined according to the weighted ranking value of each subtask.

[0016] In one embodiment, determining the task execution order of each subtask in the target task according to the weighted ranking value of each subtask includes:

[0017] Obtaining a pre-configured initial execution order of each subtask in the target task;

[0018] Determining, according to the execution order restriction conditions between the subtasks, the non-fixed-order subtasks in the subtasks and the execution order adjustment range of the non-fixed-order subtasks;

[0019] According to the weighted ranking values ​​of the subtasks and the initial execution order, the corresponding non-fixed order subtasks are adjusted in order within the execution order adjustment range to determine the task execution order of the subtasks in the target task.

[0020] In one embodiment, decomposing the target task into at least one subtask includes:

[0021] Describing the at least one task environment information by using a visual language model to obtain environment description information of the current location of the robot;

[0022] Using the environment description information as prior information of a large language model and using the large language model to decompose the target task to obtain at least one subtask corresponding to the target task;

[0023] The large language model is used to decompose the target task into multiple skills in the robot skill library according to the prior information, and combine the multiple skills into at least one subtask.

[0024] In one embodiment, obtaining at least one task environment information of the robot includes:

[0025] Determining multiple environment collection points corresponding to the robot according to the current location of the robot;

[0026] Acquire multidimensional environmental data corresponding to the multiple environmental collection points;

[0027] fusing the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the current location of the robot;

[0028] At least one task environment information of the robot is extracted from the point cloud data of the current location of the robot.

[0029] In one embodiment, the multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

[0030] In a second aspect, the present application provides a task planning device, comprising:

[0031] an acquisition module, configured to acquire voice command information input by a user for the robot; and to acquire at least one task environment information of the robot, wherein the task environment information is used to characterize environmental characteristics of the current location of the robot;

[0032] a recognition module, configured to recognize the voice instruction information, determine a target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask;

[0033] The determination module is configured to determine task impact parameters of the at least one task environment information on the associated subtasks; and determine a task execution order of the subtasks in the target task according to the task impact parameters.

[0034] In one embodiment, the determination module is further used to obtain the task type of each subtask and the feature type of each task environment information; determine the weight of the task impact parameter based on the task type and the feature type; perform weighted calculation on the task impact parameter of each subtask based on the weight of the task impact parameter to obtain the weighted ranking value of each subtask; and determine the task execution order of each subtask in the target task based on the weighted ranking value of each subtask.

[0035] In one embodiment, the determination module is further used to obtain the pre-configured initial execution order of each subtask in the target task; determine the non-fixed order subtasks in each subtask and the execution order adjustment range of the non-fixed order subtasks based on the execution order constraints between the subtasks; and adjust the corresponding non-fixed order subtasks within the execution order adjustment range based on the weighted ranking values ​​of the subtasks and the initial execution order to determine the task execution order of each subtask in the target task.

[0036] In one embodiment, the recognition module is further configured to describe the at least one task environment information using a visual language model to obtain environment description information of the robot's current location; use the environment description information as prior information of a large language model and use the large language model to decompose the target task to obtain at least one subtask corresponding to the target task;

[0037] The large language model is used to decompose the target task into multiple skills in the robot skill library according to the prior information, and combine the multiple skills into at least one subtask.

[0038] In one embodiment, the acquisition module is further used to determine multiple environmental collection points corresponding to the robot based on the current location of the robot; obtain multi-dimensional environmental data corresponding to the multiple environmental collection points; fuse the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the current location of the robot; and extract at least one task environment information of the robot from the point cloud data of the current location of the robot.

[0039] In one embodiment, the multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

[0040] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the task planning method of the first aspect when executing the computer program.

[0041] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the task planning method of the first aspect described above.

[0042] In a fifth aspect, the present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the task planning method of the first aspect described above.

[0043] The above-mentioned task planning method, apparatus, device, storage medium, and program product obtain voice command information input by a user for a robot; obtain at least one task environment information of the robot, the task environment information being used to characterize the environmental characteristics of the robot's current location; recognize the voice command information, determine the target task that the voice command information instructs the robot to perform, and decompose the target task into at least one subtask; calculate the task impact parameters of the at least one task environment information on the associated subtasks; and determine the task execution order of each subtask in the target task based on the task impact parameters. Since the task execution order of each subtask in the target task is determined based on the task impact parameters of the task environment information on the associated subtasks, the task execution order can be ensured to be compatible with the current task environment, reducing the possibility of conflict between task execution and the current task environment, thereby ensuring the execution effect of complex instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A flowchart of a task planning method provided in an embodiment of the present application;

[0046] Figure 2 A flowchart of another task planning method provided in an embodiment of the present application;

[0047] Figure 3 A flowchart of another task planning method provided in an embodiment of the present application;

[0048] Figure 4 A flowchart of another task planning method provided in an embodiment of the present application;

[0049] Figure 5 A structural block diagram of a task planning device provided in an embodiment of the present application;

[0050] Figure 6 This is a diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0052] The task planning method provided in the embodiment of the present application can be applied to a robot. The robot can first obtain the voice command information input by the user for the robot, and obtain at least one task environment information of the robot, where the task environment information is used to characterize the environmental characteristics of the robot's current location. Secondly, the robot can recognize the voice command information, determine the target task that the voice command information instructs the robot to perform, and decompose the target task into at least one subtask. Thirdly, the robot can determine the task impact parameters of the at least one task environment information on the associated subtasks. Finally, the robot can determine the task execution order of each subtask in the target task based on the task impact parameters.

[0053] Among them, the robot can be a humanoid robot or other sports robot with movable joints. The robot can include multiple joints and a visual system. The visual system is generally located at the head of the robot and is used to collect images of the external environment. In addition to the visual system, the robot can also be equipped with a voice processing system, a question-and-answer knowledge base, etc. The robot includes multiple joints, which are electrically connected to the drive mechanism to achieve a certain range of rotation. When the joint angles of each joint of the robot change, the robot can complete different posture changes. Accordingly, the robot can be controlled to perform different actions by controlling the joint angles of each joint, thereby achieving the target task instructed by the user.

[0054] In an exemplary embodiment, Figure 1 As shown, a task planning method is provided, which is described by taking the application of the method to a robot as an example, including S201-S205:

[0055] S201: Acquire voice command information input by a user for a robot.

[0056] In this application, the robot can interact with the user to obtain voice command data input by the user, thereby performing the target task instructed by the user.

[0057] It should be understood that the embodiments of the present application do not limit how to obtain the voice command data input by the user. In some embodiments, a voice processing system may be provided on the robot to obtain the voice command data input by the user through a voice acquisition system.

[0058] For example, the audio collection component in the speech processing system can be a microphone array, which can be one or more. When multiple microphone arrays are provided, the microphone arrays can be placed at different positions on the robot to collect audio data input by the user from different directions of the robot.

[0059] S202: Obtain at least one task environment information of the robot.

[0060] In this step, after obtaining the voice command information input by the user for the robot, the robot may also obtain at least one task environment information of the robot.

[0061] It should be understood that the embodiments of the present application do not limit how to obtain at least one task environment information of the robot. In some embodiments, the robot may first determine multiple environment collection points corresponding to the robot based on the robot's current location. Second, the robot may obtain multi-dimensional environment data corresponding to the multiple environment collection points. Third, the robot may fuse the multi-dimensional environment data corresponding to the multiple environment collection points into point cloud data of the robot's current location. Finally, the robot may extract at least one task environment information of the robot from the point cloud data of the robot's current location.

[0062] The aforementioned environmental collection points can be arranged around the robot in a variety of ways. For example, a grid can be arranged around the robot, with each vertex of the grid being an environmental collection point. The size of the grid can be customized based on actual conditions. For example, extension lines can be arranged around the robot at fixed angular intervals, with each extension line being provided with an environmental collection point at fixed length intervals.

[0063] It should be understood that the embodiments of the present application do not limit the above-mentioned multi-dimensional environmental data. In some embodiments, the multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

[0064] For example, the robot can be equipped with a radar sensor, an infrared sensor, and a visual sensor. The radar sensor can collect radar data, the infrared image data can collect infrared image data, and the visual sensor can collect visual image data. The robot can then fuse the multi-dimensional environmental data to obtain point cloud data of the robot's current location.

[0065] The task environment information is used to characterize the environment characteristics of the robot's current location. For example, the environment characteristics may be obstacles at a specific location of the robot, the path the robot takes to reach the specific location, and the like.

[0066] It should be understood that the embodiments of the present application do not limit how the robot identifies the environmental features of its current location. In some embodiments, the point cloud data of the robot's current location can be input into an environmental recognition model, and the point cloud data can be extracted through the environmental recognition model to accurately identify the environmental features of the robot's current location and obtain at least one task environment information of the robot.

[0067] S203: Identify the voice instruction information, determine the target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask.

[0068] In this step, after the robot obtains the voice command information and task environment information, it can identify the voice command information, determine the target task that the voice command information instructs the robot to perform, and decompose the target task into at least one subtask.

[0069] In some embodiments, the robot may first convert the voice instruction data into text instruction data, then recognize the text instruction data, and determine the target task that the voice instruction information instructs the robot to perform.

[0070] It should be understood that the embodiments of the present application do not limit how voice command data is converted into text command data. In some embodiments, the robot may first filter the voice command data to remove ambient sound from the voice command data. Subsequently, the robot may input the filtered voice command data into an Automatic Speech Recognition (ASR) model and obtain the phonemes or words contained in the voice command data output by the ASR model. Finally, the robot may decode the phonemes or words contained in the voice command data and generate text command data based on the decoding results.

[0071] The ASR model is used to identify acoustic features in voice command data and match the phonemes or words contained in the voice command data based on the acoustic features. The acoustic features can include any characteristics that can reflect speech, such as pitch, intensity, timbre, etc.

[0072] In some embodiments, the robot can extract the user's interests from the user's corresponding contextual information. Subsequently, the robot identifies the textual instruction data based on the user's interests and determines the user's task intent. Finally, the robot determines the task instructions included in the target task based on the user's task intent.

[0073] The context information corresponding to the user may include historical task data that satisfies the user. The historical task data may include voice instruction data input by the user corresponding to the historical task and feedback data of the user on the task instruction of the historical task.

[0074] The above-mentioned points of interest may be instruction types that frequently appear in the context information corresponding to the user, and the points of interest may include the actions of the robot, the direction of the robot's question and answer, and the like.

[0075] For example, after the robot extracts the user's points of interest from the context information, it can output the user's points of interest together with the text instruction data into the semantic understanding model, and obtain the task intention information inferred by the semantic understanding model.

[0076] The semantic understanding model can analyze the text content of text instruction data based on the user's interests, inferring or inferring the intent or meaning behind the text, thereby obtaining task intent information. Exemplary semantic understanding models may include bag-of-words models, term frequency-inverse document frequency (TF-IDF) models, word vectors (Word2Vec) models, bidirectional encoder representations from transformers (BERT) models, generative pretrained transformer (GPT) models, graph neural network models, and so on.

[0077] The following describes how to decompose a target task into at least one subtask.

[0078] In some embodiments, the robot may first describe at least one task environment using a visual language model to obtain a description of the environment at the robot's current location. The robot may then use this description as prior information for a large language model and decompose the target task using the large language model to obtain at least one subtask corresponding to the target task.

[0079] Among them, the large language model is used to decompose the target task into multiple skills in the robot skill library based on prior information, and combine multiple skills into at least one subtask.

[0080] It should be understood that the target task may be any task that can be executed by the robot, and the subtasks may be multiple task execution stages decomposed to complete the target.

[0081] For example, if the target task is to pour water from a cup into a bucket, the generated subtasks may include grabbing the cup, identifying the target bucket for pouring water from multiple buckets, planning a path to the target bucket, moving to the target bucket, pouring water into the target bucket, and placing the cup after pouring water at a designated location.

[0082] Exemplarily, each subtask may also include multiple skills that the robot can complete. For example, if the subtask is to move to the target bucket, the skills included therein may be moving from the initial point to the first point in a first direction, turning from the first point to the second direction, and moving from the first point to the target point where the target bucket is located in the second direction.

[0083] In some embodiments, since the environmental features corresponding to the task environment information may be information such as obstacles at a specific position of the robot, the path of the robot to reach a specific position, etc., the robot can describe at least one task environment information through a visual language model, and can obtain environmental description information such as obstacle avoidance information and path planning information.

[0084] Correspondingly, after obtaining the environment description information, the robot can use the environment description information as the prior information of the large language model, that is, use the environment description information such as obstacle avoidance information and path planning information as the decomposition conditions of the target task, and gradually decompose each step of completing the target task, so as to obtain at least one subtask corresponding to the target task.

[0085] In some embodiments, the large language model can decompose the target task into multiple sets of subtasks. The robot can then evaluate each set of subtasks separately, determine the evaluation parameters for each set of subtasks corresponding to the target task, and then select the subtask with the highest evaluation parameter from the multiple sets as the final subtask of the target task.

[0086] Among them, the above-mentioned evaluation parameters can be used to indicate the task execution speed corresponding to the set of subtasks, or, can also indicate the task execution difficulty corresponding to the set of subtasks, or, can also indicate the task execution fault tolerance corresponding to the set of subtasks. The embodiments of the present application do not limit this.

[0087] S204: Determine task impact parameters of at least one task environment information on associated subtasks.

[0088] In this step, after the robot determines the target task that the voice instruction information instructs the robot to perform and decomposes the target task into at least one subtask, it can determine the task impact parameters of at least one task environment information on the associated subtasks.

[0089] It should be understood that in the embodiments of the present application, not all task environment information is associated with each subtask. Therefore, before calculating the task impact parameters of the task environment information on each subtask, the robot can first determine the association relationship between the task environment information and the subtask.

[0090] For example, the robot can determine the relationship between task environment information and subtasks from a spatial perspective. That is, the robot can estimate the completion location of each subtask. If the completion location of the subtask is within the influence range of the environmental characteristics corresponding to the task environment information, then the robot can determine that the task environment information and the subtask are related.

[0091] In some embodiments, the robot uses a large language model to determine the task impact parameters of at least one task environment information on each associated subtask. The large language model can separately evaluate the impact coefficient of each task environment information on the task completion time and the task completion difficulty of the associated subtask, and then combine the impact coefficients of the task completion time and task completion difficulty to determine the task impact parameters of each task environment information on the associated subtask.

[0092] For example, completing Subtask A requires circumventing an obstacle indicated by Task Environment Information 1. Because the obstacle obstructs a large area, the robot spends 10 seconds detouring without encountering any additional complex road conditions. Therefore, the large language model can estimate the impact coefficient of completion time to be 0.6 and the impact coefficient of task difficulty to be 0.1. By multiplying the impact coefficients of completion time and task difficulty, the large language model calculates the impact parameter of Task Environment Information 1 on Subtask A to be 0.06.

[0093] S205: Determine the task execution order of each subtask in the target task according to the task impact parameter.

[0094] In this step, after determining the task impact parameters of at least one task environment information on the associated subtasks, the robot may determine the task execution order of each subtask in the target task according to the task impact parameters.

[0095] In some embodiments, the robot may first obtain the task type of each subtask and the characteristic type of each task environment information, and then determine the weight of the task impact parameter based on the task type and characteristic type. Subsequently, the robot performs a weighted calculation on the task impact parameter of each subtask based on the weight of the task impact parameter to obtain a weighted ranking value for each subtask. Finally, the robot may determine the task execution order of each subtask within the target task based on the weighted ranking value of each subtask.

[0096] Among them, task types can include question-answering tasks, mobile tasks, action tasks, etc., and feature types can include obstacle features, path features, etc. Different task types and feature types have different weights on task influencing parameters.

[0097] For example, the robot can be pre-set with a combination of task types and feature types, and a mapping relationship with the weights of task influencing parameters. When sub-tasks need to be sorted, the robot can directly match the weights of each task influencing parameter based on the mapping relationship.

[0098] For example, since each subtask may include multiple task impact parameters, after the robot determines the weight of each task impact parameter, all task impact parameters associated with each subtask can be weightedly calculated using the weights to obtain a weighted ranking value for subtask sorting.

[0099] It should be understood that the embodiments of the present application do not limit how to determine the task execution order of each subtask in the target task based on the weighted ranking value of each subtask. In some embodiments, the robot can first obtain the initial execution order of each subtask in the preconfigured target task. Subsequently, the robot can determine the non-fixed order subtasks in each subtask and the execution order adjustment range of the non-fixed order subtasks based on the execution order restriction conditions between the subtasks. Finally, the robot can adjust the order of the corresponding non-fixed order subtasks within the execution order adjustment range based on the weighted ranking value and initial execution order of each subtask to determine the task execution order of each subtask in the target task.

[0100] The preconfigured initial execution order may be a subtask execution order that does not consider environmental influence factors.

[0101] It should be understood that the execution order constraint can be a sequence condition of the internal execution logic. For example, if the target task is to pour water from a cup into a bucket, the subtasks include grabbing the cup, identifying the target bucket for pouring water from multiple buckets, planning a path to the target bucket, moving to the target bucket, pouring water into the target bucket, and placing the cup after pouring water at a designated location. Among them, the execution order constraint can include that the subtask "grabbing the cup" needs to be restricted before the subtask "moving to the target bucket", and the subtask "placing the cup after pouring water at a designated location" needs to be restricted after the subtask "pouring water into the target bucket".

[0102] For example, after determining the non-fixed order subtasks in each subtask and the execution order adjustment range of the non-fixed order subtasks through execution order constraints, the execution order of the non-fixed order subtasks in the initial execution order can be adjusted according to the size of the weighted ranking value of each subtask, thereby determining the final task execution order of each subtask in the target task.

[0103] In some embodiments, when the scenario changes, the robot will also synchronize previously executed subtasks along with the modified task environment information to the large language model, so that the large language model can reselect the remaining subtasks based on the executed subtasks and the modified task environment information and regenerate tasks. Similarly, the task impact parameters can also be re-determined, and the task execution order of each subtask in the regenerated task can be determined based on the task impact parameters.

[0104] The task planning method provided in the embodiment of the present application obtains voice command information input by a user for a robot; obtains at least one task environment information of the robot, the task environment information being used to characterize the environmental characteristics of the robot's current location; recognizes the voice command information, determines the target task that the voice command information instructs the robot to perform, and decomposes the target task into at least one subtask; calculates the task impact parameters of at least one task environment information on the associated subtasks; and determines the task execution order of each subtask in the target task based on the task impact parameters. Since the task execution order of each subtask in the target task is determined based on the task impact parameters of the associated subtasks based on the task environment information, it is possible to ensure that the task execution order is compatible with the current task environment, reduce the possibility of conflict between task execution and the current task environment, and thus ensure the execution effect of complex instructions.

[0105] The following describes how to obtain at least one task environment information of the robot. Figure 2 A flowchart of another task planning method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the task planning method includes S301-S308:

[0106] S301: Acquire voice command information input by a user for a robot.

[0107] S302: Determine multiple environment collection points corresponding to the robot according to the current location of the robot.

[0108] The aforementioned environmental collection points can be arranged around the robot in a variety of ways. For example, a grid can be arranged around the robot, with each vertex of the grid being an environmental collection point. The size of the grid can be customized based on actual conditions. For example, extension lines can be arranged around the robot at fixed angular intervals, with each extension line being provided with an environmental collection point at fixed length intervals.

[0109] S303: Acquire multi-dimensional environmental data corresponding to multiple environmental collection points.

[0110] The multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

[0111] For example, the robot can be equipped with a radar sensor, an infrared sensor, and a visual sensor. The radar sensor can collect radar data, the infrared image data can collect infrared image data, and the visual sensor can collect visual image data. The robot can then fuse the multi-dimensional environmental data to obtain point cloud data of the robot's current location.

[0112] S304: Fusing the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the current location of the robot.

[0113] S305: Extract at least one task environment information of the robot from the point cloud data of the current location of the robot.

[0114] The task environment information is used to characterize the environment characteristics of the robot's current location. For example, the environment characteristics may be obstacles at a specific location of the robot, the path the robot takes to reach the specific location, and the like.

[0115] It should be understood that the embodiments of the present application do not limit how the robot identifies the environmental features of its current location. In some embodiments, the point cloud data of the robot's current location can be input into an environmental recognition model, and the point cloud data can be extracted through the environmental recognition model to accurately identify the environmental features of the robot's current location and obtain at least one task environment information of the robot.

[0116] S306: Identify the voice instruction information, determine the target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask.

[0117] S307: Determine task impact parameters of at least one task environment information on associated subtasks.

[0118] S308: Determine the task execution order of each subtask in the target task according to the task impact parameter.

[0119] The following describes how to decompose a target task into at least one subtask. Figure 3 A flowchart of another task planning method provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the task planning method includes S401-S407:

[0120] S401: Acquire voice command information input by a user for a robot.

[0121] S402: Acquire at least one task environment information of the robot, where the task environment information is used to characterize the environmental characteristics of the current location of the robot.

[0122] S403: Identify the voice command information and determine the target task that the voice command information instructs the robot to perform.

[0123] S404: Describe at least one task environment information using a visual language model to obtain environment description information of the robot's current location.

[0124] S405 : Using the environment description information as prior information of the large language model and using the large language model to decompose the target task to obtain at least one subtask corresponding to the target task.

[0125] Among them, the large language model is used to decompose the target task into multiple skills in the robot skill library based on prior information, and combine multiple skills into at least one subtask.

[0126] It should be understood that the target task may be any task that can be executed by the robot, and the subtasks may be multiple task execution stages decomposed to complete the target.

[0127] In some embodiments, since the environmental features corresponding to the task environment information may be information such as obstacles at a specific position of the robot, the path of the robot to reach a specific position, etc., the robot can describe at least one task environment information through a visual language model, and can obtain environmental description information such as obstacle avoidance information and path planning information.

[0128] Correspondingly, after obtaining the environment description information, the robot can use the environment description information as the prior information of the large language model, that is, use the environment description information such as obstacle avoidance information and path planning information as the decomposition conditions of the target task, and gradually decompose each step of completing the target task, so as to obtain at least one subtask corresponding to the target task.

[0129] S406: Determine task impact parameters of at least one task environment information on associated subtasks.

[0130] S407: Determine the task execution order of each subtask in the target task according to the task impact parameter.

[0131] The following describes how to determine the execution order of each subtask in the target task. Figure 4 A flowchart of another task planning method provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the task planning method includes S501-S508:

[0132] S501: Acquire voice command information input by a user for a robot.

[0133] S502: Acquire at least one task environment information of the robot, where the task environment information is used to characterize the environmental characteristics of the current location of the robot.

[0134] S503: Identify the voice instruction information, determine the target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask.

[0135] S504: Determine task impact parameters of at least one task environment information on associated subtasks.

[0136] S505: Obtain the task type of each subtask and the feature type of each task environment information.

[0137] S506: Determine the weight of the task impact parameter according to the task type and feature type.

[0138] S507 : Perform weighted calculation on the task impact parameters of each subtask according to the weight of the task impact parameter to obtain a weighted ranking value of each subtask.

[0139] S508: Determine the task execution order of each subtask in the target task according to the weighted ranking value of each subtask.

[0140] The task planning method provided in the embodiment of the present application obtains voice command information input by a user for a robot; obtains at least one task environment information of the robot, the task environment information being used to characterize the environmental characteristics of the robot's current location; recognizes the voice command information, determines the target task that the voice command information instructs the robot to perform, and decomposes the target task into at least one subtask; calculates the task impact parameters of at least one task environment information on the associated subtasks; and determines the task execution order of each subtask in the target task based on the task impact parameters. Since the task execution order of each subtask in the target task is determined based on the task impact parameters of the associated subtasks based on the task environment information, it is possible to ensure that the task execution order is compatible with the current task environment, reduce the possibility of conflict between task execution and the current task environment, and thus ensure the execution effect of complex instructions.

[0141] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0142] Based on the same inventive concept, the present application also provides a task planning device for implementing the task planning method described above. The implementation solution provided by the device is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more task planning device embodiments provided below can be found in the above-mentioned limitations of the task planning method and will not be repeated here.

[0143] In an exemplary embodiment, Figure 5 As shown, a task planning device 600 is provided, comprising: an acquisition module 601, an identification module 602 and a determination module 603, wherein:

[0144] The acquisition module 601 is used to acquire the voice command information input by the user for the robot; and acquire at least one task environment information of the robot, where the task environment information is used to characterize the environmental characteristics of the current location of the robot.

[0145] The recognition module 602 is used to recognize the voice instruction information, determine the target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask.

[0146] The determination module 603 is configured to determine task impact parameters of at least one task environment information on the associated subtasks; and determine the task execution order of each subtask in the target task according to the task impact parameters.

[0147] In one embodiment, the determination module 603 is also used to obtain the task type of each subtask and the feature type of each task environment information; determine the weight of the task impact parameter based on the task type and feature type; perform weighted calculations on the task impact parameters of each subtask based on the weight of the task impact parameter to obtain the weighted ranking value of each subtask; and determine the task execution order of each subtask in the target task based on the weighted ranking value of each subtask.

[0148] In one embodiment, the determination module 603 is also used to obtain the initial execution order of each subtask in the preconfigured target task; determine the non-fixed order subtasks in each subtask and the execution order adjustment range of the non-fixed order subtasks based on the execution order constraints between the subtasks; and adjust the corresponding non-fixed order subtasks within the execution order adjustment range based on the weighted ranking value and initial execution order of each subtask to determine the task execution order of each subtask in the target task.

[0149] In one embodiment, the recognition module 602 is further used to describe at least one task environment information through a visual language model to obtain environment description information of the robot's current location; use the environment description information as prior information of the large language model and use the large language model to decompose the target task to obtain at least one subtask corresponding to the target task.

[0150] Among them, the large language model is used to decompose the target task into multiple skills in the robot skill library based on prior information, and combine multiple skills into at least one subtask.

[0151] In one embodiment, the acquisition module 601 is further used to determine multiple environmental collection points corresponding to the robot based on the current location of the robot; obtain multi-dimensional environmental data corresponding to the multiple environmental collection points; fuse the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the current location of the robot; and extract at least one task environment information of the robot from the point cloud data of the current location of the robot.

[0152] In one embodiment, the multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

[0153] Each module in the aforementioned task planning device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0154] In an exemplary embodiment, a computer device is provided. The computer device is located on a robot, and its internal structure diagram can be shown as follows: Figure 6As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a task planning method is implemented.

[0155] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0156] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned task planning method when executing the computer program.

[0157] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned task planning method is implemented.

[0158] In one embodiment, a computer program product is provided, comprising a computer program, which implements the above-mentioned task planning method when executed by a processor.

[0159] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0160] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0161] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0162] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A task planning method, characterized in that: The method comprises: Obtain the voice command information input by the user for the robot; Acquire at least one task environment information of the robot, where the task environment information is used to characterize environmental characteristics of the current location of the robot; Recognizing the voice command information, determining a target task that the voice command information instructs the robot to perform, and decomposing the target task into at least one subtask; Determining task impact parameters of the at least one task environment information on the associated subtasks, wherein the task impact parameters are obtained based on an impact coefficient of the task completion time and an impact coefficient of the task completion difficulty associated with the task environment information; Obtain the task type of each subtask and the feature type of each task environment information; Determining the weight of the task influencing parameter according to the task type and the feature type; task types include question-answering tasks, movement tasks, and action tasks; feature types include obstacle features and path features; different task types and feature types have different weights for task influencing parameters; According to the weight of the task impact parameter, the task impact parameter of each subtask is weighted and calculated to obtain a weighted ranking value of each subtask; Obtaining a pre-configured initial execution order of each subtask in the target task, wherein the initial execution order is the subtask execution order without considering environmental influence factors; Determining the non-fixed order subtasks in each subtask and the execution order adjustment range of the non-fixed order subtasks according to the execution order constraint conditions between the subtasks, wherein the execution order constraint conditions are the sequence conditions of the internal execution logic; According to the weighted ranking values ​​of the subtasks and the initial execution order, the corresponding non-fixed order subtasks are adjusted in order within the execution order adjustment range to determine the task execution order of the subtasks in the target task.

2. The method according to claim 1, characterized in that Decomposing the target task into at least one subtask includes: Describing the at least one task environment information by using a visual language model to obtain environment description information of the current location of the robot; Using the environment description information as prior information of a large language model and using the large language model to decompose the target task to obtain at least one subtask corresponding to the target task; The large language model is used to decompose the target task into multiple skills in the robot skill library according to the prior information, and combine the multiple skills into at least one subtask.

3. The method according to claim 1, characterized in that The obtaining of at least one task environment information of the robot includes: Determining multiple environment collection points corresponding to the robot according to the current location of the robot; Acquire multidimensional environmental data corresponding to the multiple environmental collection points; fusing the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the current location of the robot; At least one task environment information of the robot is extracted from the point cloud data of the current location of the robot.

4. The method according to claim 3, characterized in that The multi-dimensional environmental data includes at least one of the following: visual image data, infrared image data, and radar data.

5. A task planning device, characterized in that: The device comprises: an acquisition module, configured to acquire voice command information input by a user for the robot; and to acquire at least one task environment information of the robot, wherein the task environment information is used to characterize environmental characteristics of the current location of the robot; a recognition module, configured to recognize the voice instruction information, determine a target task that the voice instruction information instructs the robot to perform, and decompose the target task into at least one subtask; A determination module is used to determine the task impact parameters of the at least one task environment information on the associated subtasks, wherein the task impact parameters are obtained based on the impact coefficient of the task completion time and the impact coefficient of the task completion difficulty associated with the task environment information; obtain the task type of each subtask and the feature type of each task environment information; the task type includes question-answering tasks, mobile tasks, and action tasks, and the feature type includes obstacle features and path features. Different task types and feature types have different weights for task impact parameters; according to the task type and feature type, determine the weight of the task impact parameter; according to the weight of the task impact parameter, the task impact parameter of each subtask is determined. Perform weighted calculations separately to obtain weighted ranking values ​​of each subtask; obtain the initial execution order of each subtask in the preconfigured target task, wherein the initial execution order is the subtask execution order without considering environmental factors; determine the non-fixed order subtasks in each subtask and the execution order adjustment range of the non-fixed order subtasks according to the execution order restriction conditions between the subtasks, wherein the execution order restriction conditions are the sequence conditions of the internal execution logic; according to the weighted ranking values ​​and initial execution order of each subtask, adjust the corresponding non-fixed order subtasks in order within the execution order adjustment range to determine the task execution order of each subtask in the target task.

6. The device according to claim 5, characterized in that The recognition module is further used to describe at least one task environment information through a visual language model to obtain environment description information of the robot's current location; use the environment description information as prior information of the large language model and use the large language model to decompose the target task to obtain at least one subtask corresponding to the target task; wherein the large language model is used to decompose the target task into multiple skills in the robot skill library based on the prior information, and combine the multiple skills into at least one subtask.

7. The device according to claim 5, characterized in that The acquisition module is further used to determine multiple environmental collection points corresponding to the robot based on the robot's current location; obtain multi-dimensional environmental data corresponding to the multiple environmental collection points; fuse the multi-dimensional environmental data corresponding to the multiple environmental collection points into point cloud data of the robot's current location; and extract at least one task environment information of the robot from the point cloud data of the robot's current location.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images