Task planning method and device based on large language model, electronic equipment, storage medium and computer program product

Abstract

The present disclosure relates to a task planning method and device based on a large language model, an electronic device, a storage medium and a computer program product. The method comprises: for each task in a plurality of robot tasks, randomly sampling target parameters from a parameter set corresponding to the task; generating a target task instruction in natural language form based on the target parameters; constructing a structured step true value sequence corresponding to the target task instruction; inputting the target task instruction into a pre-trained large language model; comparing a structured step prediction sequence with the structured step true value sequence; and based on the comparison result, evaluating the inference performance of the large language model. In this way, since the present disclosure can automatically and quantitatively evaluate the structured inference ability of the large language model under fixed task constraints, the adverse effects of human factors on the inference performance evaluation of the large language model caused by relying on artificial labeling or subjective judgment can be effectively avoided.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to task planning methods, devices, electronic devices, storage media, and computer program products based on large language models. Background Technology

[0002] With the development of technology, large language models have been widely used in robot control, task planning and human-computer interaction. Using natural language to describe robots performing complex multi-step tasks is becoming an important research direction.

[0003] In related technologies, large language models can typically generate coherent text responses. However, in task scenarios involving multiple objectives, multiple stages, and strong process constraints, the output of large language models often suffers from issues such as missing steps, disordered order, and semantic inconsistencies. Furthermore, related technologies primarily rely on manual annotation or subjective judgment for performance evaluation of large language models. Due to the introduction of human factors, this can lead to poor accuracy and low efficiency in performance evaluation. Summary of the Invention

[0004] This disclosure provides a task planning method, apparatus, electronic device, storage medium, and computer program product based on a large language model, to at least solve the problem in the aforementioned related technologies that the introduction of human factors may lead to poor accuracy and low efficiency in performance evaluation of large language models.

[0005] According to a first aspect of the present disclosure, a task planning method based on a large language model is provided, comprising: for each of a variety of robot tasks, randomly sampling target parameters from a parameter set corresponding to that task, wherein the variety of robot tasks includes at least a single-robot multi-target retrieval task and a multi-robot cooperative transport task; generating target task instructions in natural language form based on the target parameters; constructing a structured step truth sequence corresponding to the target task instructions; inputting the target task instructions into the pre-trained large language model to obtain a structured step prediction sequence; comparing the structured step prediction sequence with the structured step truth sequence to obtain a comparison result; and evaluating the inference performance of the large language model based on the comparison result to obtain a performance evaluation result.

[0006] According to an exemplary embodiment of this disclosure, the structured step truth sequence is represented in key-value pair form, wherein the key contained in the key-value pair is used to represent a task stage, and the value contained in the key-value pair is used to represent the corresponding operation semantic description.

[0007] According to an exemplary embodiment of this disclosure, comparing the structured step prediction sequence with the structured step truth sequence to obtain a comparison result includes: determining whether the predicted step keys contained in the structured step prediction sequence are consistent with the true step keys contained in the structured step truth sequence; determining whether the arrangement order of the predicted step keys is consistent with the arrangement order of the true step keys; determining whether the predicted value corresponding to the predicted step key is consistent with the true value corresponding to the true step key, and obtaining the comparison result; the evaluation of the inference performance of the large language model based on the comparison result includes: determining the number of consistent results indicated by the comparison result; evaluating the inference performance of the large language model based on the number of consistent results, wherein the more consistent results there are, the better the inference performance.

[0008] According to an exemplary embodiment of this disclosure, after evaluating the inference performance of the large language model based on the comparison results, the method further includes: writing the target task instruction, the structured step truth sequence, the structured step prediction sequence, and the performance evaluation results into a log file and storing the log file.

[0009] According to an exemplary embodiment of this disclosure, the step of evaluating the reasoning performance of the large language model based on the comparison results and obtaining performance evaluation results includes: calculating the reasoning accuracy of the large language model based on the comparison results using the following formula, wherein the reasoning accuracy is used to characterize the reasoning performance of the large language model, and the higher the reasoning accuracy, the better the reasoning performance; The formula is expressed as follows:

[0010] in, Let N represent the inference accuracy, and let N represent the number of step keys contained in the structured step truth sequence. This represents the actual value of the key in the i-th step. I( represents the predicted value of the key in the i-th step) ) is an indicator function.

[0011] According to a second aspect of the present disclosure, a task planning apparatus based on a large language model is provided, comprising: a parameter sampling module configured to randomly sample target parameters from a parameter set corresponding to each of a variety of robot tasks, wherein the variety of robot tasks includes at least a single-robot multi-target retrieval task and a multi-robot cooperative transport task; a task instruction generation module configured to generate target task instructions in natural language form based on the target parameters; a truth sequence construction module configured to construct a structured step truth sequence corresponding to the target task instructions; a prediction sequence acquisition module configured to input the target task instructions into the pre-trained large language model to obtain a structured step prediction sequence; a comparison module configured to compare the structured step prediction sequence with the structured step truth sequence to obtain a comparison result; and a performance evaluation module configured to evaluate the inference performance of the large language model based on the comparison result to obtain a performance evaluation result.

[0012] According to an exemplary embodiment of this disclosure, the structured step truth sequence is represented in key-value pair form, wherein the key contained in the key-value pair is used to represent a task stage, and the value contained in the key-value pair is used to represent the corresponding operation semantic description.

[0013] According to an exemplary embodiment of this disclosure, the comparison module is configured to: determine whether the predicted step keys contained in the structured step prediction sequence are consistent with the true step keys contained in the structured step truth sequence; determine whether the arrangement order of the predicted step keys is consistent with the arrangement order of the true step keys; determine whether the predicted value corresponding to the predicted step key is consistent with the true value corresponding to the true step key, and obtain the comparison result; the performance evaluation module is configured to: determine the number of consistent results indicated by the comparison result; and evaluate the inference performance of the large language model based on the number of consistent results, wherein the more consistent results there are, the better the inference performance.

[0014] According to an exemplary embodiment of the present disclosure, the task planning apparatus further includes: a writing module configured to write the target task instruction, the structured step truth sequence, the structured step prediction sequence, and the performance evaluation result into a log file and store the log file.

[0015] According to an exemplary embodiment of this disclosure, the performance evaluation module is configured to: calculate the inference accuracy of the large language model based on the comparison results using the following formula, wherein the inference accuracy is used to characterize the inference performance of the large language model, and the higher the inference accuracy, the better the inference performance; The formula is expressed as follows:

[0016] in, Let N represent the inference accuracy, and let N represent the number of step keys contained in the structured step truth sequence. This represents the actual value of the key in the i-th step. I( represents the predicted value of the key in the i-th step) ) is an indicator function.

[0017] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement a task planning method based on a large language model according to the present disclosure.

[0018] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform a task planning method based on a large language model according to the present disclosure.

[0019] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a task planning method based on a large language model according to the present disclosure.

[0020] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects: In this disclosure, by adopting step-level structured representation for each robot task and evaluating the reasoning performance of the large language model based on the consistency of step structure, it is possible to ensure that the evaluation results are close to the actual reasoning performance of the large language model. That is, it can ensure the reliability and consistency of the large language model in the decomposition of complex robot tasks and the understanding of multi-step instructions. In other words, it can achieve a quantitative evaluation of the structured reasoning ability of the large language model in different task scenarios.

[0021] Furthermore, since this disclosure can automatically and quantify the structured reasoning ability of large language models under fixed task constraints, it can effectively avoid the adverse effects of human factors introduced by relying on manual annotation or subjective judgment on the reasoning performance evaluation of large language models, and can improve the accuracy and efficiency of reasoning performance evaluation.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0024] Figure 1 This is a flowchart illustrating a task planning method based on a large language model according to an exemplary embodiment of the present disclosure; Figure 2 This is a schematic diagram illustrating the instruction understanding and structured execution process in a "single robot multi-target retrieval task" scenario according to an exemplary embodiment of the present disclosure; Figure 3 This is a schematic diagram illustrating a reasoning example in a "single-robot multi-target retrieval task" scenario according to an exemplary embodiment of the present disclosure; Figure 4 This is a schematic diagram illustrating a reasoning example in a "multi-robot collaborative handling task" scenario according to exemplary embodiments of the present disclosure; Figure 5 This is a block diagram illustrating a task planning apparatus based on a large language model according to an exemplary embodiment of the present disclosure; Figure 6 This is a block diagram illustrating an electronic device according to exemplary embodiments of the present disclosure. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0026] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0027] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0028] Figure 1 This is a flowchart illustrating a task planning method based on a large language model according to an exemplary embodiment of the present disclosure.

[0029] Reference Figure 1 In step 101, for each of the various robot tasks, target parameters can be randomly sampled from the parameter set corresponding to that task. The various robot tasks may include at least a single-robot multi-target retrieval task and a multi-robot collaborative transport task. A single-robot multi-target retrieval task corresponds to a multi-target retrieval scenario, and its instructions may include at least two spatial position parameters and two target item parameters. A multi-robot collaborative transport task corresponds to a multi-robot collaborative transport scenario, and its instructions may include at least a starting position parameter, a target position parameter, and intermediate collaborative node parameters.

[0030] It should be noted that in this disclosure, the system can construct a multi-task instruction dataset for model training and evaluation through a template-driven data generation mechanism. This process can be based on a predefined set of natural language instruction templates and achieve large-scale, low-redundancy data synthesis by randomly sampling the task parameter space.

[0031] In step 102, target task instructions in natural language form can be generated based on the target parameters.

[0032] As previously mentioned, the robot tasks in this disclosure can include at least two types: "single-robot multi-target retrieval tasks" and "multi-robot collaborative handling tasks." For "single-robot multi-target retrieval tasks," this scenario mainly describes the process of a robot sequentially going to two different shelves to retrieve different parts according to a single instruction and returning for delivery. The natural language form of the target task instruction in this scenario is characterized by: containing two spatial targets and two object targets, with a clear temporal relationship; the output structure in this scenario can be: fixedly including two task steps of "navigation—recognition—grasping—temporary storage," as well as "return" and "delivery" steps.

[0033] In addition, in the scenario of "single robot multi-target retrieval task", a set of multiple semantically equivalent but different expression forms of instruction templates can be predefined. Each instruction template can contain two shelf number parameters and two part number parameters, thereby ensuring that the task instructions meet the task constraints of "dual objectives and sequential correlation" at the semantic level.

[0034] Let the set of shelf number parameters be: .

[0035] The part number parameter set is as follows: .

[0036] When generating data, the system may not necessarily... Instead of performing a full combinatorial enumeration, a random sampling strategy can be used to generate samples under the following constraints: .

[0037] The introduction of this strategy avoids the data generation blockage problem caused by the exponential growth of the combination space size with the number of parameters, while ensuring the diversity of task scenarios at both the spatial and objective levels.

[0038] For each set of sampled parameters The system can sequentially substitute these values ​​into multiple natural language templates, thereby generating multiple instruction texts with different expressions but consistent task semantics. For example, the resulting data sample can be represented as: .

[0039] For the "multi-robot collaborative handling task" scenario, this scenario mainly describes the process of multiple robots transferring parts from a table to a shelf through relay cooperation. The characteristics of the target task instructions in natural language form in this scenario are: involving the switching of multiple robot roles and the handover of objects; the output structure in this scenario can be: containing the navigation, grasping and placement steps of each of the multiple robots, with a strict execution order.

[0040] Furthermore, in the "multi-robot collaborative handling task" scenario, the target task instruction in natural language form can include three core parameters: desktop number, part number, and target shelf number. And, similar to the "single-robot multi-target retrieval task" scenario, the "multi-robot collaborative handling task" can generate parameter triples through random sampling. This can be injected into various natural language templates to enhance the diversity of instructions at the sentence level. "Parameter triples" In the figure, 't' represents the desktop number parameter; 'p' represents the part number parameter. This indicates the target shelf number parameter.

[0041] In step 103, a structured truth sequence of steps corresponding to the target task instructions can be constructed. It should be noted that existing methods can be used to construct the structured truth sequence of steps in this disclosure, which will not be elaborated upon here. Thus, by employing step-level structured representation for different task scenarios, execution stability can be significantly improved.

[0042] According to exemplary embodiments of this disclosure, the structured step truth sequence described above can be represented in key-value pair form. The key in each key-value pair can represent a task stage; for example, the key can be a numbered step in the format of step x. The value in each key-value pair can represent the corresponding operational semantic description; for example, the value can be the actual detailed operation of the corresponding numbered step. That is, in this disclosure, a step-level key-value pair structure can be used to represent a robot execution plan, and each step can contain a step key and a step value. The step key can be used to represent the execution order and execution subject; the step value can be used to describe the specific operational semantics of the corresponding step. Furthermore, different task scenarios can each use a predefined step key sequence to ensure that each type of task has a fixed and predictable output structure and that the model output can be directly parsed into a robot execution plan.

[0043] For example, for each task instruction in natural language form, the system can automatically construct a structured truth sequence that corresponds one-to-one with the task flow through a predefined step generation function. This structured truth sequence can be represented by an ordered mapping of a fixed key set to clearly characterize the action sequence and operational semantics that the robot should follow during task execution.

[0044] For the aforementioned "single robot, multi-target retrieval task" scenario, the set of step keys can be set as follows:

[0045] The corresponding structured truth value can then be formally represented as:

[0046] in, This refers to the operation description text directly generated from the task parameters. For example, the semantics of the "Navigation 1" step can be strictly defined as "planning a route and moving to the shelf". This ensures that the meaning of the steps remains consistent across different samples, with only the parameter values ​​changing. This design allows subsequent model inference results to be compared with the truth at both the key and semantic levels.

[0047] Figure 2This is a schematic diagram illustrating the instruction understanding and structured execution process in a "single-robot multi-target retrieval task" scenario according to an exemplary embodiment of the present disclosure. (Refer to...) Figure 2 After randomly sampling target parameters from the parameter set corresponding to the task, and generating target task instructions in natural language form based on the target parameters, a structured step truth value sequence corresponding to the target task instructions can be constructed. The step keys included in the structured step truth value sequence can include, but are not limited to: navigation, recognition, fetching, temporary storage, return, and delivery. Each step key can also have its own value. For example, the value of the "navigation" step key could be: "to shelf x"; the value of the "recognition" step key could be: "part i"; the value of the "return" step key could be: "to user"; and the value of the "delivery" step key could be: "deliver part i, etc., to the user".

[0048] For the aforementioned "multi-robot collaborative handling task" scenario, the structured truth steps in this scenario can explicitly distinguish the action subjects of different robots. For example, its step key set can be defined as:

[0049]

[0050]

[0051] The corresponding structured truth can be expressed as:

[0052] in, This refers to the operation description text generated directly from the task parameters.

[0053] This representation explicitly encodes the task execution subject, action type, and stage sequence at the data level, thus providing clear supervision signals for the model to learn the semantics of multi-robot collaboration.

[0054] It should be noted that, in this disclosure, after generating the original instructions and structured truth values, the system can further perform unified formatting on the data to construct input samples suitable for training an autoregressive language model. For example, at this stage, the system can convert each sample into the following text format:

[0055] in, Representing the target task instructions in natural language form. Let represent the truth sequence of the structured steps, and It can be expanded into a JSON-style string, and the order of the step keys must strictly follow a predefined fixed order set. or , This indicates a newline character to start the next line. The introduction of this sequence constraint allows the model to learn the relative positions of steps during training, thereby reducing the probability of missed steps or incorrect sequence during the inference phase.

[0056] For example, in this disclosure, the following operations can be performed for different task scenarios: 1. Automatically generate a large number of natural language instructions based on parameterized templates; 2. Synchronously generate step-level structured truth values ​​that correspond one-to-one with the instructions; 3. Through a unified data preprocessing workflow, natural language instructions and step-level structured truth values ​​are organized as follows: Instruction: <Natural Language Instruction> Output: { Step 1: "...", Step 2":...", ... } This format ensures that different tasks have a consistent representation in the model input space, thereby avoiding semantic conflicts during multi-task training.

[0057] In addition, this disclosure employs a parameter-efficient fine-tuning method to perform multi-task joint training on a large-scale pre-trained language model. The characteristics of this multi-task joint training are: freezing the basic model parameters to retain its general language understanding ability; learning the task-specific instruction-to-step mapping relationship through a lightweight parameter injection method (e.g., a low-rank adaptation module); and training with mixed multi-task data so that the model can simultaneously master multiple task execution modes within the same parameter space.

[0058] In this way, through this multi-task joint training method—that is, through efficient fine-tuning of parameters using a unified format—knowledge sharing across multiple tasks can be achieved, reducing the cost of repetitive training. Furthermore, unified modeling across task scenarios can be achieved without significantly increasing the model size. Additionally, the inference results can be directly executed, meaning the model output can be used in the robot control system without additional parsing.

[0059] For example, during the model training phase, this disclosure can employ the LoRA technique in Parameter Efficient Fine-Tuning (PEFT) to adapt the basic large language model to the task. Furthermore, the basic model parameters can be frozen during training, with low-rank trainable parameter matrices introduced only in the attention projection layer.

[0060] Let the basic model parameters be The low-rank parameter introduced by LoRA is Then, the model weight update form during training can be:

[0061] in, The original weight matrix is ​​frozen. , ,and , Represents the space of real numbers. Represents low-rank parameters The dimension is , Represents low-rank parameters The dimension is .

[0062] In this way, the model can efficiently absorb task-specific structured output patterns while maintaining its original language comprehension capabilities.

[0063] In step 104, the target task instructions in natural language form can be input into a pre-trained large language model to obtain a structured step prediction sequence. That is, the input instructions can be structured using a large language model with efficiently fine-tuned parameters, thereby outputting step-by-step instruction results that correspond one-to-one with the task flow. It should be noted that using a pre-trained large language model to output structured step prediction sequences is existing technology, and will not be described in detail here.

[0064] In step 105, the predicted sequence of the aforementioned structured steps can be compared with the true value sequence of the aforementioned structured steps to obtain the comparison result, that is, the model prediction result can be compared with the true value item by item.

[0065] In step 106, the inference performance of the large language model can be evaluated based on the comparison results to obtain performance evaluation results.

[0066] It should be noted that in this disclosure, the system can use a strict key order consistency criterion to evaluate the prediction results: for example, when the prediction result is a valid JSON and its step key set is completely consistent with the expected fixed key set, the sample can be judged as correct.

[0067] According to exemplary embodiments of this disclosure, it can be determined whether the predicted step keys contained in the structured step prediction sequence are consistent with the true step keys contained in the structured step truth sequence, i.e., whether the step key set is complete; it can also be determined whether the order of the predicted step keys is consistent with the order of the true step keys, i.e., whether the step order is consistent; and it can further be determined whether the predicted value corresponding to the predicted step key is consistent with the true value corresponding to the true step key, thus obtaining a comparison result. Next, the number of consistent results indicated by the comparison result can be determined. Then, based on the number of consistent results, the inference performance of the large language model can be evaluated, wherein the more consistent results, the better the inference performance.

[0068] According to an exemplary embodiment of this disclosure, the reasoning accuracy of the large language model can also be calculated based on the comparison results using the following formula, wherein the reasoning accuracy can be used to characterize the reasoning performance of the large language model, and the higher the reasoning accuracy, the better the reasoning performance. The formula can be expressed as:

[0069] in, For inference accuracy, N represents the number of step keys contained in the structured step truth sequence. This represents the actual value of the key in the i-th step. I( represents the predicted value of the key in the i-th step) ) is an indicator function.

[0070] Additionally, regarding the indicator function I( ),exist The condition inside the square brackets is true, that is... If true, I( The value of ) can be 1; otherwise, that is, in The condition inside the square brackets is false, that is... If not true, I( The value of ) can be 0.

[0071] In this way, evaluating the inference performance of a large language model by using accuracy based on the consistency of the step structure can more realistically reflect the system's reliability, that is, it can more accurately reflect the real execution requirements of the large language model.

[0072] Additionally, let the total number of test samples be... The number of correctly predicted samples is The scene reasoning accuracy can then be defined as:

[0073] Furthermore, the reasoning accuracy rate can be calculated separately for each task scenario, and then the overall reasoning accuracy rate can be calculated based on the reasoning accuracy rate for each task scenario. In this way, the usability of the large language model at the robot execution level can be directly reflected.

[0074] Figure 3 This is a schematic diagram illustrating a reasoning example in a "single-robot multi-target retrieval task" scenario according to an exemplary embodiment of the present disclosure. (Refer to...) Figure 3 In the "single robot multi-target retrieval task" scenario, the randomly sampled target parameters can be: "shelf 19", "shelf 46", "part A124", and "part A123". Furthermore, the natural language target task instruction generated based on the randomly sampled target parameters is: "Go to shelf 19 and shelf 46 and bring me one part A124 and one part A123 respectively." Additionally, by comparing the structured step prediction sequence (pred) with the structured step truth sequence (GT), it can be seen that the step key set is complete, the order of the step keys is consistent, and the values ​​corresponding to the step keys are consistent. Therefore, this inference is completely correct.

[0075] Figure 4 This is a schematic diagram illustrating a reasoning example in a "multi-robot collaborative handling task" scenario according to an exemplary embodiment of this disclosure. (Refer to...) Figure 4 In the scenario of "multi-robot collaborative handling task", the randomly sampled target parameters can be: "table A46", "part a029", and "shelf n45". Furthermore, the natural language target task instruction generated based on the randomly sampled target parameters is: "Move part a029 from table A46 to shelf n45". Additionally, by comparing the structured step prediction sequence (pred) with the structured step truth sequence (GT), it can be seen that the step key set is complete, the order of the step keys is consistent, and the values ​​corresponding to the step keys are consistent. Therefore, this inference is completely correct.

[0076] According to an exemplary embodiment of this disclosure, after evaluating the inference performance of the large language model based on the comparison results, the aforementioned target task instructions, the truth sequence of structured steps, the prediction sequence of structured steps, and the performance evaluation results can be written into a log file and stored.

[0077] In this way, by writing all task instructions, truth steps, prediction steps, and performance evaluation results generated during the testing process into a log file, it is possible to effectively support result verification and engineering traceability.

[0078] This disclosure forms a complete closed loop in the stages of data generation, structured modeling, model training, and inference evaluation, and each stage can be directly mapped to specific engineering code implementation. Therefore, this disclosure has clear reproducibility and industrial applicability.

[0079] Figure 5 This is a block diagram illustrating a task planning apparatus 500 based on a large language model according to an exemplary embodiment of the present disclosure.

[0080] Reference Figure 5 The task planning device 500 may include a parameter sampling module 501, a task instruction generation module 502, a truth sequence construction module 503, a prediction sequence acquisition module 504, a comparison module 505, and a performance evaluation module 506.

[0081] For each of the various robot tasks, the parameter sampling module 501 can randomly sample target parameters from the parameter set corresponding to that task. The various robot tasks may include at least single-robot multi-target retrieval tasks and multi-robot collaborative transport tasks. Single-robot multi-target retrieval tasks correspond to multi-target retrieval scenarios, and their instructions may include at least two spatial position parameters and two target item parameters. Multi-robot collaborative transport tasks correspond to multi-robot collaborative transport scenarios, and their instructions may include at least a starting position parameter, a target position parameter, and intermediate collaborative node parameters.

[0082] The task instruction generation module 502 can generate target task instructions in natural language form based on target parameters.

[0083] The truth sequence construction module 503 can construct a structured step truth sequence corresponding to the target task instruction.

[0084] According to exemplary embodiments of this disclosure, the structured step truth sequence described above can be represented in key-value pair form. The key in each key-value pair can represent a task stage; for example, the key can be a numbered step in the format of step x. The value in each key-value pair can represent the corresponding operational semantic description; for example, the value can be the actual detailed operation of the corresponding numbered step. That is, in this disclosure, a step-level key-value pair structure can be used to represent a robot execution plan, and each step can contain a step key and a step value. The step key can be used to represent the execution order and execution subject; the step value can be used to describe the specific operational semantics of the corresponding step. Furthermore, different task scenarios can each use a predefined step key sequence to ensure that each type of task has a fixed and predictable output structure and that the model output can be directly parsed into a robot execution plan.

[0085] The prediction sequence acquisition module 504 can input the target task instructions in natural language form mentioned above into a pre-trained large language model to obtain a structured step prediction sequence. That is, the large language model, with its parameters efficiently fine-tuned, can perform structured reasoning on the input instructions, and then output step-by-step instruction results that correspond one-to-one with the task flow.

[0086] The comparison module 505 can compare the predicted sequence of the aforementioned structured steps with the true value sequence of the aforementioned structured steps to obtain the comparison result, that is, it can compare the model prediction result with the true value item by item.

[0087] The performance evaluation module 506 can evaluate the inference performance of a large language model based on the comparison results and obtain the performance evaluation results.

[0088] According to an exemplary embodiment of this disclosure, the comparison module 505 can determine whether the predicted step keys contained in the structured step prediction sequence are consistent with the true step keys contained in the structured step truth sequence, that is, it can determine whether the set of step keys is complete; the comparison module 505 can also determine whether the arrangement order of the predicted step keys is consistent with the arrangement order of the true step keys, that is, it can determine whether the step order is consistent; the comparison module 505 can also determine whether the predicted value corresponding to the predicted step key is consistent with the true value corresponding to the true step key, and obtain a comparison result. Next, the performance evaluation module 506 can determine the number of consistent results indicated by the comparison result. Then, the performance evaluation module 506 can evaluate the inference performance of the large language model based on the number of consistent results, wherein the more consistent results, the better the inference performance.

[0089] According to an exemplary embodiment of this disclosure, the performance evaluation module 506 can also calculate the inference accuracy of the large language model based on the comparison results using the following formula, wherein the inference accuracy can be used to characterize the inference performance of the large language model, and the higher the inference accuracy, the better the inference performance.

[0090] The formula can be expressed as:

[0091] in, For inference accuracy, N represents the number of step keys contained in the structured step truth sequence. This represents the actual value of the key in the i-th step. I( represents the predicted value of the key in the i-th step) ) is an indicator function.

[0092] Additionally, regarding the indicator function I( ),exist The condition inside the square brackets is true, that is... If true, I( The value of ) can be 1; otherwise, that is, in The condition inside the square brackets is false, that is... If not true, I( The value of ) can be 0.

[0093] According to an exemplary embodiment of this disclosure, the task planning apparatus 500 may further include a writing module. This writing module can write the aforementioned target task instructions, the structured step truth sequence, the structured step prediction sequence, and the performance evaluation results into a log file and store the log file.

[0094] Figure 6 This is a block diagram illustrating an electronic device 600 according to an exemplary embodiment of the present disclosure.

[0095] Reference Figure 6 The electronic device 600 includes at least one memory 601 and at least one processor 602. The at least one memory 601 stores instructions that, when executed by the at least one processor 602, execute a task planning method based on a large language model according to an exemplary embodiment of the present disclosure.

[0096] As an example, electronic device 600 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned instructions. Here, electronic device 600 is not necessarily a single electronic device, but may be a collection of any devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 600 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0097] In electronic device 600, processor 602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0098] The processor 602 can execute instructions or code stored in the memory 601, which can also store data. Instructions and data can also be sent and received via a network through a network interface device, which can employ any known transmission protocol.

[0099] The memory 601 may be integrated with the processor 602, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 601 may include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory 601 and the processor 602 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 602 to read files stored in the memory.

[0100] In addition, the electronic device 600 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device 600 can be interconnected via a bus and / or network.

[0101] According to exemplary embodiments of this disclosure, a computer-readable storage medium may also be provided, which, when executed by a processor of an electronic device, enables the electronic device to perform the aforementioned task planning method based on a large language model. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0102] According to exemplary embodiments of the present disclosure, a computer program product may also be provided, including a computer program that, when executed by a processor, implements the task planning method based on a large language model according to the present disclosure.

[0103] According to the task planning method, apparatus, electronic device, storage medium and computer program products based on the large language model disclosed herein, by adopting step-level structured representation for each robot task and evaluating the reasoning performance of the large language model based on the consistency of step structure, it can be ensured that the evaluation results are close to the actual reasoning performance of the large language model. That is, it can ensure the reliability and consistency of the large language model in the decomposition of complex robot tasks and the understanding of multi-step instructions. In other words, it can realize the quantitative evaluation of the structured reasoning ability of the large language model in different task scenarios.

[0104] Furthermore, since this disclosure can automatically and quantify the structured reasoning ability of large language models under fixed task constraints, it can effectively avoid the adverse effects of human factors introduced by relying on manual annotation or subjective judgment on the reasoning performance evaluation of large language models, and can improve the accuracy and efficiency of reasoning performance evaluation.

[0105] According to exemplary embodiments of this disclosure, execution stability can be significantly improved by adopting step-level structured representation for different task scenarios.

[0106] According to exemplary embodiments of this disclosure, by introducing sequence constraints, the model can learn the relative positional relationships between steps during training, thereby reducing the probability of missing steps or disordered sequence during the inference phase.

[0107] According to exemplary embodiments of this disclosure, a unified data preprocessing process can ensure that different tasks have a consistent representation in the model input space, thereby avoiding semantic conflicts during multi-task training.

[0108] According to exemplary embodiments of this disclosure, multi-task joint training, i.e., efficient fine-tuning through a unified format and parameters, can achieve multi-task knowledge sharing and reduce the cost of repetitive training. Furthermore, it can achieve unified modeling across task scenarios without significantly increasing the model size. Additionally, it enables direct execution of inference results, meaning the model output can be used in the robot control system without additional parsing.

[0109] According to exemplary embodiments of this disclosure, evaluating the inference performance of a large language model by using accuracy based on the consistency of step structure can more realistically reflect the system reliability, that is, it can more accurately reflect the real execution requirements of the large language model.

[0110] According to exemplary embodiments of this disclosure, the reasoning accuracy in each task scenario can be statistically analyzed separately, and then the overall reasoning accuracy can be calculated based on the reasoning accuracy in each task scenario. In this way, the availability of the large language model at the robot execution level can be directly reflected.

[0111] According to an exemplary embodiment of this disclosure, by writing all task instructions, truth steps, prediction steps, and performance evaluation results generated during the testing process into a log file, effective support for result verification and engineering traceability can be achieved.

[0112] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0113] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A task planning method based on a large language model, characterized in that, include: For each of the various robot tasks, target parameters are randomly sampled from the parameter set corresponding to that task. The various robot tasks include at least single-robot multi-target retrieval tasks and multi-robot cooperative transport tasks. Based on the target parameters, generate target task instructions in natural language form; Construct a structured truth value sequence of steps corresponding to the target task instruction; The target task instruction is input into the pre-trained large language model to obtain a structured step prediction sequence; The predicted sequence of the structured steps is compared with the true sequence of the structured steps to obtain the comparison result; Based on the comparison results, the reasoning performance of the large language model is evaluated to obtain performance evaluation results.

2. The task planning method as described in claim 1, characterized in that, The structured step truth sequence is represented in key-value pair form, wherein the key in the key-value pair is used to represent the task stage, and the value in the key-value pair is used to represent the corresponding operation semantic description.

3. The task planning method as described in claim 2, characterized in that, The step of comparing the predicted sequence of the structured steps with the true sequence of the structured steps to obtain the comparison result includes: Determine whether the predicted step keys contained in the structured step prediction sequence are consistent with the true step keys contained in the structured step truth sequence; determine whether the arrangement order of the predicted step keys is consistent with the arrangement order of the true step keys; determine whether the predicted value corresponding to the predicted step key is consistent with the true value corresponding to the true step key, and obtain the comparison result. The evaluation of the reasoning performance of the large language model based on the comparison results includes: Determine the number of consistent results indicated by the comparison results; The reasoning performance of the large language model is evaluated based on the number of consistent results, wherein the more consistent results there are, the better the reasoning performance.

4. The task planning method as described in claim 1, characterized in that, After evaluating the inference performance of the large language model based on the comparison results, the method further includes: The target task instruction, the truth sequence of the structured steps, the prediction sequence of the structured steps, and the performance evaluation results are written into a log file and stored.

5. The task planning method as described in claim 1, characterized in that, The process of evaluating the inference performance of the large language model based on the comparison results, and obtaining performance evaluation results, includes: Based on the comparison results, the reasoning accuracy of the large language model is calculated using the following formula, wherein the reasoning accuracy is used to characterize the reasoning performance of the large language model, and the higher the reasoning accuracy, the better the reasoning performance. The formula is expressed as follows: in, Let N represent the inference accuracy, and let N represent the number of step keys contained in the structured step truth sequence. This represents the actual value of the key in the i-th step. I( represents the predicted value of the key in the i-th step) ) is an indicator function.

6. A task planning device based on a large language model, characterized in that, include: The parameter sampling module is configured to randomly sample target parameters from the parameter set corresponding to each of the various robot tasks, wherein the various robot tasks include at least single-robot multi-target retrieval tasks and multi-robot cooperative handling tasks. The task instruction generation module is configured to generate target task instructions in natural language form based on the target parameters; The truth sequence construction module is configured to construct a structured step truth sequence corresponding to the target task instruction; The prediction sequence acquisition module is configured to input the target task instruction into the pre-trained large language model to obtain a structured step prediction sequence; The comparison module is configured to compare the predicted sequence of the structured steps with the true sequence of the structured steps to obtain the comparison result; The performance evaluation module is configured to evaluate the inference performance of the large language model based on the comparison results and obtain performance evaluation results.

7. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the task planning method based on a large language model as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the task planning method based on a large language model as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the task planning method based on a large language model as described in any one of claims 1 to 5.

Images