Task planning method and device based on large model feedback optimization

Abstract

The application belongs to the technical field of artificial intelligence, and provides a task planning method and device based on large model feedback optimization, which comprises the following steps: converting a natural language description of a problem into a PDDL symbol sequence, and calculating an output probability corresponding to the PDDL symbol sequence according to LoRA parameters; calculating a loss value between the output probability and a target PDDL symbol sequence, updating the LoRA parameters, and thereby training a large language model with planning generation capability; verifying the executability of a candidate PDDL planning scheme output by the model, adjusting input context information of the input model by using an error log, and obtaining an executable planning scheme through an iterative verification process. The method and device avoid new errors caused by full rewriting by limiting the modification range, and the model uses the solvability of the planner and the verification pass as the stop condition when iterating, so that the result has evaluability and reproducibility.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a task planning method and apparatus based on large model feedback optimization. Background Technology

[0002] The most commonly used description language in the field of symbolic programming is PDDL (Planning Domain Description Language). PDDL is used to transcribe the user's task description into symbolic programming language, which is then used by a classic planner to solve the problem and obtain an executable sequence of actions.

[0003] In related technologies, the generation of natural language into PDDL typically employs an end-to-end generation method centered around a large model. This method directly outputs domain or problem files from the model, supplemented by a small number of examples, retrieval enhancements, or chained reasoning hints to improve structural correctness and constraint consistency. However, during file verification and patching, the model is simply required to be rewritten, or error snippets are only attached to the hints. This leads to an uncontrollable patching process, an excessively large scope of modifications, and a tendency to introduce new type conflicts, predicate signature errors, or object reference errors. Ultimately, this results in an increased parsing rate but an unstable solvability rate, or rapid degradation on more difficult instances, and poor reproducibility. Summary of the Invention

[0004] This invention provides a task planning method and apparatus based on large model feedback optimization to solve the defects of existing natural language to PDDL generation methods, which directly output domain or problem files from the model. These files are uncontrollable during the verification and repair process, easily introducing new type conflicts, predicate signature errors, or object reference errors, ultimately leading to unstable file parsing rates and poor reproducibility.

[0005] This invention provides a task planning method based on large model feedback optimization, comprising:

[0006] The natural language description of the problem is converted into a sequence of symbols in the planning domain definition language (PDDL), and the output probability corresponding to the PDDL symbol sequence is calculated based on the LoRA parameters of the large language model.

[0007] Calculate the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task, and update the LoRA parameter according to the loss value. If the large language model training reaches a preset number of iterations or the loss converges, a large language model with planning and generation capabilities is trained.

[0008] The feasibility of the candidate PDDL planning schemes output by the large language model with planning generation capability is verified. If the verification fails, the input context information of the large language model with planning generation capability is optimized according to the error logs corresponding to the candidate PDDL planning schemes, and an executable planning scheme is obtained through an iterative verification process. The error logs include at least one of the following: predicate signature inconsistency, object not declared, type mismatch, missing key facts in the initial state, and unreachable target conditions.

[0009] According to the task planning method based on large model feedback optimization provided by the present invention, the calculation of the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task includes:

[0010] The loss value between the output probability and the target PDDL symbol sequence corresponding to the target task is calculated based on the negative log-likelihood loss function.

[0011] The negative log-likelihood loss function is expressed by the following formula:

[0012] ;

[0013] in, Let be the negative log-likelihood loss function. The length of the target PDDL symbol sequence. For the first t One target token; Is the model in parameters Below, based on x With input context information Generate target The probability, x This is a description of a natural language task.

[0014] According to the task planning method based on large model feedback optimization provided by the present invention, after training to obtain a large language model with planning generation capabilities, the method further includes:

[0015] The multiple candidate PDDL planning schemes output by the large language model with planning generation capability are sampled to obtain a set of candidate PDDL planning schemes.

[0016] Each candidate PDDL planning scheme in the candidate PDDL planning scheme set is tested for feasibility. The candidate PDDL planning schemes that pass the test are used as positive samples, and the candidate PDDL planning schemes that fail the test are used as negative samples. A reward function is then constructed.

[0017] The relative advantage of each candidate PDDL planning scheme is calculated based on the reward function. A joint loss is constructed using the relative advantage, KL constraint, and negative log-likelihood loss function. The LoRA parameters of the large language model with planning generation capability are updated based on the joint loss. A new large language model is obtained when the large language model with planning generation capability reaches a preset number of iterations or the loss converges.

[0018] According to the task planning method based on large model feedback optimization provided by the present invention, the step of optimizing the input context information of the large language model with planning generation capability according to the error log corresponding to the candidate PDDL planning scheme, and obtaining an executable planning scheme through an iterative verification process includes:

[0019] Error features are extracted from the error log and converted into repair instructions. The repair instructions are used to instruct the large language model with planning generation capabilities to adjust the error content in the candidate PDDL planning scheme.

[0020] The repair instructions and the PDDL symbol sequence are input into the large language model with planning generation capability to obtain a new candidate PDDL planning scheme. If the new candidate PDDL planning scheme passes the executability verification or the large language model with planning generation capability reaches the maximum number of training iterations, an executable planning scheme is obtained.

[0021] According to the task planning method based on large model feedback optimization provided by the present invention, the feasibility verification of the candidate PDDL planning scheme output by the large language model with planning generation capability includes:

[0022] The candidate PDDL planning scheme is parsed using a Fast Downward planner to obtain the parsed task state and the parsed task objective;

[0023] If the parsed task state matches the initial state corresponding to the target task, and the parsed task objective is consistent with the target task, then the candidate PDDL planning scheme is determined to be an executable scheme.

[0024] According to a task planning method based on large model feedback optimization provided by the present invention, after obtaining an executable planning scheme through an iterative verification process, the method further includes:

[0025] The parsing rate and solvability rate are calculated based on the executability verification success rate of the PDDL planning scheme output by the large language model with planning generation capability.

[0026] Calculate the semantic similarity between the PDDL planning scheme output by the large language model with planning generation capability and the real PDDL planning scheme;

[0027] The PDDL planning scheme output by the large language model with planning generation capability is analyzed and the corresponding real PDDL planning scheme is analyzed respectively. The consistency of the two sets of analysis results is compared to obtain the consistency rate.

[0028] The large language model with planning generation capability is evaluated based on at least one of the parsable rate, the solvable rate, the semantic similarity, and the consistency rate to obtain the model evaluation result.

[0029] According to the task planning method based on large model feedback optimization provided by the present invention, the natural language description of the problem is obtained through the following steps:

[0030] The natural language description information of the target task is formatted using a structural template to obtain a natural language description of the problem. The structural template includes object declarations, initial state facts, and a set of target conditions. Problem files corresponding to different target tasks adopt the same writing standards, which include object naming rules, type annotation style, consistency of predicate parameter order, and completeness checks of initial facts.

[0031] The present invention also provides a task planning device based on large model feedback optimization, comprising:

[0032] The computation module is used to convert the natural language description of the problem into a sequence of symbols in the planning domain definition language (PDDL), and to calculate the output probability corresponding to the PDDL symbol sequence based on the LoRA parameters of the large language model.

[0033] The training module is used to calculate the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task, and update the LoRA parameter according to the loss value. When the large language model training reaches a preset number of iterations or the loss converges, a large language model with planning and generation capabilities is trained.

[0034] The optimization module is used to perform executability verification on the candidate PDDL planning schemes output by the large language model with planning generation capability. If the verification fails, the module optimizes the input context information of the large language model with planning generation capability based on the error logs corresponding to the candidate PDDL planning schemes, and obtains an executable planning scheme through an iterative verification process. The error logs include at least one of the following: predicate signature inconsistency, object not declared, type mismatch, missing key facts in the initial state, and unreachable target conditions.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the task planning method based on large model feedback optimization as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the task planning method based on large model feedback optimization as described above.

[0037] The task planning method and apparatus based on large model feedback optimization provided by this invention converts the natural language description of the problem into a PDDL symbol sequence, calculates the output probability corresponding to the PDDL symbol sequence according to the LoRA parameter, and then calculates the loss value between the output probability and the target PDDL symbol sequence to update the LoRA parameter. Finally, the feasibility of the candidate PDDL planning scheme output by the model is verified. If the verification fails, the input context information of the large language model with planning generation capability is optimized according to the error log, and an executable planning scheme is obtained through an iterative verification process. This method maps the locationable information output when the verification fails to a reusable error category and defines a repair strategy for each type of error. By limiting the scope of modification to only repair the smallest set that leads to unsolvability, new errors caused by full rewriting are avoided. In each iteration of the large language model, the planner's solvability and verification pass are used as objective stopping conditions, making the results evaluable and reproducible. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is one of the flowcharts of the task planning method based on large model feedback optimization provided by the present invention.

[0040] Figure 2 This is the second flowchart of the task planning method based on large model feedback optimization provided by the present invention.

[0041] Figure 3 This is a schematic diagram of the task planning device based on large model feedback optimization provided by the present invention.

[0042] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0044] The following is combined with Figures 1-3 This invention describes a task planning method and apparatus based on large model feedback optimization.

[0045] Figure 1 This is one of the flowcharts illustrating the task planning method based on large model feedback optimization provided by this invention, such as... Figure 1 As shown, the method includes the following steps:

[0046] Step 110: Convert the natural language description of the problem into a sequence of symbols in the Planning Domain Definition Language (PDDL), and calculate the output probability corresponding to the PDDL symbol sequence based on the LoRA parameters of the large language model.

[0047] In this step, the user-provided natural language task description is first received, the task objective and constraints are parsed out, and the corresponding PDDL symbol sequence is obtained.

[0048] For example, the task is described in natural language as follows: "Transport package pkg1 from warehouse wh1 to shop1. There is a truck truck1 initially in the warehouse, and the package is initially in the warehouse. The truck can travel between locations connected by roads. The truck needs to load the package before transporting it, and finally unload the package at the shop. The goal is to have the package in the shop."

[0049] It should be noted that in this planning domain, domain documents typically define the following predicates and actions. Examples of predicates include `at` to indicate that an object is at a location, `in` to indicate that a package is inside a vehicle, and `connected` to indicate connectivity between locations. Examples of actions include `drive` to drive from one location to another, `load` to load a package into a truck at the same location, and `unload` to unload a package from a truck at the same location.

[0050] In this embodiment, the problem file needs to determine the object set, initial state, and target based on natural language.

[0051] In this embodiment, under the zero-shot learning framework, the large language model needs to learn the mapping function from natural language input to PDDL. As shown below:

[0052] ;

[0053] in, x Represents a natural language task description. y This represents the corresponding target PDDL symbol sequence. The trainable parameters of the model, namely the low-rank matrices A and B, are LoRA (Low-Rank Adaptation) parameters.

[0054] In this embodiment, ( (where is the length of the target PDDL symbol sequence). The large language model can calculate the probability distribution of the output token using the following formula:

[0055] ;

[0056] in, It is the first in the target PDDL symbol sequence t One token; It is the first in the target PDDL symbol sequence t All tokens preceding this token (i.e., context information); It is the first t The hidden state of each location; and It is a pair of low-rank matrices; It is the scaling factor. The basic weights are used; in this embodiment, the softmax function is used to map the product of the weights and the hidden states to a probability distribution.

[0057] Step 120: Calculate the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task, and update the LoRA parameter according to the loss value. If the large language model training reaches the preset number of iterations or the loss converges, a large language model with planning and generation capabilities is trained.

[0058] In this step, the large language model is fine-tuned under LoRA supervision. Specifically, it takes natural language as input and learns to output a problem file with a stable structure that is consistent with the domain rules. The key constraints are the faithful encoding of object declarations, type consistency, predicate signatures, and initial target facts.

[0059] It's important to note that LoRA fine-tuning allows for adaptation to specific planning tasks by training only a very small number of parameters, significantly reducing training overhead. LoRA fine-tuning achieves this through efficient parameter updates, adding low-rank increments only to a portion of the weight matrix of a large model while simultaneously freezing the original model's parameters. This significantly reduces the number of training parameters and memory usage, avoiding overfitting and excessively high training costs caused by full fine-tuning. Its technical principle is achieved through low-rank decomposition, where the weights of large language models...W The mathematical expression is shown in the following formula:

[0060] ;

[0061] in, These are the base weights for the large language model (same as the large model), and they are kept frozen during training. The low-rank increment introduced for LoRA is a trainable parameter; and It is a pair of low-rank matrices; It is a scaling factor used to control the low-rank increment. For basic weights The update magnitude only requires updating A and B during training; the basic weights are not updated. By keeping it constant, the model's generalization ability is guaranteed, and training efficiency is greatly improved.

[0062] In one embodiment, according to the task planning method based on large model feedback optimization provided by the present invention, calculating the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task includes: calculating the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task based on the negative log-likelihood loss function; the negative log-likelihood loss function is expressed by the following formula:

[0063] ;

[0064] in, It is the negative log-likelihood loss function. The length of the target PDDL symbol sequence. For the first t One target token; Is the model in parameters Below, based on input x With input context information Generate target The probability of.

[0065] In this embodiment, the negative log-likelihood loss is used to measure the semantic matching degree between the PDDL symbol sequence generated by the model and the target PDDL symbol sequence, that is, to minimize the probability of the model generating incorrect tokens.

[0066] Specifically, the model's parameter update process only applies to the low-rank matrix of LoRA. and (i.e., LoRA parameters) During the forward propagation process, the large language model traverses each sample in the current training batch. The output probability P of the model is calculated using the following formula:

[0067] ;

[0068] Next, substitute the P value into... The corresponding loss value is calculated and added to the total batch loss; during the backpropagation of the large language model, the gradients of LoRA parameters A and B are calculated using the chain rule, the LoRA parameters are updated and maintained. Without changing the parameters, the updated parameters are used as the initial parameters for the next batch of training. The forward and backward propagation processes are repeated until the training reaches the preset number of iterations or the loss converges, thus training a large language model with planning and generation capabilities.

[0069] This embodiment helps the model learn the syntactic structure of PDDL (such as bracket matching and keyword spelling) and domain-specific logical constraints (such as predicate parameter order) by minimizing the negative log-likelihood loss. At the same time, LoRA's efficient update mechanism avoids the overfitting risk and high computational cost caused by full fine-tuning.

[0070] Step 130: Perform executability verification on the candidate PDDL planning schemes output by the large language model with planning generation capability. If the verification fails, optimize the input context information of the large language model with planning generation capability based on the error logs corresponding to the candidate PDDL planning schemes, and obtain an executable planning scheme through an iterative verification process. The error logs include at least one of the following: inconsistent predicate signatures, undeclared objects, type mismatch, missing key facts in the initial state, and unreachable target conditions.

[0071] In this step, external tools, such as a syntax checker, the planning and validation tool VAL, and the planner (FastDownward), are used to check whether the PDDL generated by the model is usable. If it is not usable, the logs generated by the planner or validator are parsed and mapped to error logs, including error types (such as inconsistent predicate signatures, missing key facts, etc.).

[0072] Specifically, the feasibility verification of candidate PDDL planning schemes output by large language models with planning generation capabilities includes: using a Fast Downward planner to parse the candidate PDDL planning schemes to obtain the parsed task state and the parsed task objective; if the parsed task state matches the initial state corresponding to the target task, and the parsed task objective is consistent with the target task, the candidate PDDL planning scheme is determined to be an executable scheme.

[0073] In this step, based on the error types mentioned above, new context information containing explicit repair instructions is constructed and re-input into the large language model. The model parameters are not modified, but the model's performance on the current task is optimized through context optimization, enabling it to output the corrected results.

[0074] Because errors in PDDL format often have clear locality, such as inconsistent predicate signatures, undeclared objects, type mismatches, missing key facts in the initial state, and unreachable target conditions, the validator and planner will output localizable information when they fail, such as the reason why the action cannot be applied, the missing preconditions, the unbound object, and the key bottleneck that prevents the target from being reached.

[0075] This embodiment constructs a closed-loop system involving the planner, structures the failure diagnosis of the verifier and planner, and transforms it into localized, executable repair instructions, thereby driving the PDDL format toward executability with minimal modifications.

[0076] The task planning method based on large model feedback optimization provided in this invention converts the natural language description of the problem into a PDDL symbol sequence, calculates the output probability corresponding to the PDDL symbol sequence according to the LoRA parameter, and then calculates the loss value between the output probability and the target PDDL symbol sequence to update the LoRA parameter. Finally, the feasibility of the candidate PDDL planning scheme output by the model is verified. If the verification fails, the input context information of the large language model with planning generation capability is optimized according to the error log, and an executable planning scheme is obtained through an iterative verification process. This method maps the locationable information output when the verification fails to a reusable error category and defines a repair strategy for each type of error. By limiting the scope of modification to only repair the smallest set that leads to unsolvability, new errors caused by full rewriting are avoided. In each iteration, the large language model uses the planner's solvability and successful verification as objective stopping conditions, making the results evaluable and reproducible.

[0077] In some embodiments, after training a large language model with planning generation capabilities, the method further includes: sampling multiple candidate PDDL planning schemes output by the large language model with planning generation capabilities to obtain a set of candidate PDDL planning schemes; performing executability verification on each candidate PDDL planning scheme in the set of candidate PDDL planning schemes, using the verified candidate PDDL planning schemes as positive samples and the unverified candidate PDDL planning schemes as negative samples, and constructing a reward function; calculating the relative advantage of each candidate PDDL planning scheme based on the reward function, constructing a joint loss using the relative advantage, KL constraint, and negative log-likelihood loss function, and updating the LoRA parameters of the large language model with planning generation capabilities based on the joint loss; and obtaining a new large language model when the large language model with planning generation capabilities reaches a preset number of iterations or the loss converges.

[0078] In this embodiment, for the same input PDDL symbol sequence, the large language model samples and generates multiple candidate PDDL planning schemes at once. The planner's solution is mapped to a reward, and then the relative advantage is calculated among the candidates in the same group. The model is updated using the log-likelihood gradient weighted by the relative advantage, making it more biased towards the better candidate within the group. Since the relative advantage naturally removes the mean or standardizes, it can significantly reduce the problem of inconsistent reward scales between instances of different domains and different sizes, and reduce gradient variance, thus making it more stable during multi-domain training. At the same time, by adding the KL constraint of the reference model, the output format and distribution can be kept stable, avoiding syntactic degradation or pattern collapse in the later stages of training.

[0079] Specifically, for each x (The corresponding natural language task description is a structured task representation.) A set of candidate outputs is sampled from the output schemes of current large language models with planning and generation capabilities. y 1 to y K And for each candidate output, perform validation and planning to obtain a reward. r 1 to r K Then, the within-group relative advantage is calculated to reduce scale differences and highlight within-group comparisons. One way to achieve relative advantage is to remove the mean and standardize. Specifically, it is calculated using the following formula:

[0080] ;

[0081] in, For the first i One reward, A numerical stabilization term is introduced to prevent the denominator from being zero. Next, the model is updated using a relative advantage-weighted log-likelihood, and KL constraints are added to maintain closeness to the reference model. Finally, the joint loss, i.e., the GRPO loss, is constructed by combining the current model's negative log-likelihood loss function with the following formula. , is represented as:

[0082] ;

[0083] in, for x The corresponding expectation, For large language models with planning and generation capabilities. The reference model is the selected, supervised, fine-tuned frozen version or base model. The coefficient is used to determine the probability of a candidate when its relative advantage is better than the average in the same group. When the relative advantage is positive, training will increase its probability. When the candidate is worse, the relative advantage is negative, and training will decrease its probability. Since the advantage is normalized within the group, the difference in reward scale between planning domains will be significantly weakened, making it more suitable for multi-domain joint training.

[0084] It should be noted that for the same natural language input, a large language model with planning and generation capabilities may sample multiple candidate problem files. Candidate 1 is unsolvable due to lack of connectivity and has a low reward; Candidate 2 is correctly connected and solvable, with a plan length of three steps and a higher reward; Candidate 3 is also solvable but adds redundant locations or objects, resulting in a longer plan or higher cost, and its reward is between that of Candidate 2 and the unsolvable candidate.

[0085] This embodiment utilizes the GRPO strategy to calculate the relative advantage among candidates in the same group. This will encourage the model to favor the structure of candidate two, which is both solvable and better, rather than simply learning to output a barely solvable version. As a result, the model will be more inclined to complete the necessary connected facts, maintain the consistency of predicate signatures, and avoid introducing irrelevant objects during generation, thereby reducing the number of rounds and costs of loop closure repair from the source.

[0086] In this embodiment, the reward value corresponding to each solution can be composed of multiple components, such as parsable score, solvable score, and plan quality score. When setting the reward in this embodiment, solvability is taken as the primary item and quality is taken as the secondary item under the condition of solvability, so that unsolvable candidates are significantly lower in reward than solvable candidates, and solvable candidates are further sorted according to plan length or cost.

[0087] In this embodiment, during the backpropagation process of the model, the gradient of the LoRA parameters is calculated using the chain rule, and the LoRA parameters are updated using the joint loss mentioned above while keeping the original model parameters unchanged. The updated parameters are used as the initial parameters for the next batch of training. The forward and backward propagation processes are repeated until the training reaches the preset number of iterations or the loss converges, thus obtaining a new large language model.

[0088] In this embodiment, the new large language model outputs a scheme that is objectively verified by the planner and can be reproduced as a specification document. If an error occurs, the system can explain which type of error it is, which initial facts or predicate usages were repaired, and provide whether it is solvable and the final plan after each iteration. This makes the scheme very suitable for deployment in scenarios with strict requirements for reliability and executability, such as automated logistics scheduling, robot task orchestration, and task planning assistants for non-expert users, thus meeting the reliability and interpretability requirements of engineering systems.

[0089] The task planning method based on large model feedback optimization provided in this invention sample multiple candidate PDDL planning schemes output by the model, and perform executability verification on each candidate PDDL planning scheme in the candidate PDDL planning scheme set. A reward function is constructed using verified candidate PDDL planning schemes as positive samples and unverified candidate PDDL planning schemes as negative samples. The relative advantage of each scheme is then calculated. Finally, a joint loss is constructed using relative advantage, KL constraint, and negative log-likelihood loss function to update the LoRA parameters of the model, ultimately obtaining a new large language model. By introducing a GRPO training objective oriented towards planning signals, the large language model not only tends to output solvable specifications but also stably aligns higher-quality plans within the solvable set. Furthermore, the relative advantage within groups reduces training instability caused by differences in reward scales.

[0090] In some embodiments, optimizing the input context information of a large language model with planning generation capability based on the error logs corresponding to the candidate PDDL planning scheme, and obtaining an executable planning scheme through an iterative verification process includes: extracting error features from the error logs and converting them into repair instructions, the repair instructions being used to instruct the large language model with planning generation capability to adjust the error content in the candidate PDDL planning scheme; inputting the repair instructions and PDDL symbol sequences into the large language model with planning generation capability to obtain a new candidate PDDL planning scheme; and obtaining an executable planning scheme when the new candidate PDDL planning scheme passes the executability verification or the large language model with planning generation capability reaches the maximum number of training iterations.

[0091] In this embodiment, during the closed-loop repair part of the inference stage, the candidate PDDL planning schemes output by the large language model with planning generation capability are sequentially checked for syntax, verified and planned. If the planning fails, error features (the most critical evidence of failure) are extracted from the log and structured into repair instructions.

[0092] In this embodiment, the patch instruction emphasizes the principle of minimal modification, allowing only local areas such as object declarations, type annotations, missing fact completion, target condition correction, and obvious predicate parameter order corrections to be modified, while keeping the rest of the structure unchanged.

[0093] In this embodiment, the repair instructions are re-inputted into a large language model with planning generation capabilities along with the original output. The model is required to complete the repair without changing the correct parts. This process can be iterated in multiple rounds until the planner returns a solution and passes verification, or until a preset number of rounds is reached. Since each round has a clear externally verifiable stopping condition, the final output can be independently reproduced and evaluated.

[0094] Specifically, suppose the first candidate PDDL plan (problem file) output by a large language model with planning generation capabilities contains a common error: omitting connectivity relationships or writing the wrong connectivity predicate, causing the planner to fail to find a driving path; for example, the model outputs:

[0095] (:init

[0096] (at truck1 wh1)

[0097] (at pkg1 wh1));

[0098] While the grammar may still be parsable at this point, the planner will fail because the `drive` action requires a `connected` precondition, and there are no `connected` facts in the initial state. Therefore, the planner cannot drive the truck from `wh1` to `shop1`. Planner logs typically indicate that certain actions are unapplicable or cannot be extended to the target during the search. The closed-loop system categorizes this failure as a missing critical initial fact and generates a patch instruction, requiring the addition of location connectivity relationships without altering the object set or existing `at` facts. A large language model with planning generation capabilities makes minimal modifications under this patch instruction constraint, such as adding the following to the original solution:

[0099] (connected wh1 shop1)

[0100] (connected shop1 wh1);

[0101] After solving the problem again, the planner can obtain the plan, for example:

[0102] load pkg1 truck1 wh1

[0103] drive truck1 wh1 shop1

[0104] unload pkg1 truck1 shop1;

[0105] Another, more subtle but common type of error is the disordered order of predicate parameters; for example, if the model writes the parameters of "at" as "at location" or "at object", the output will be:

[0106] (at wh1 truck1)

[0107] (at wh1 pkg1);

[0108] Since the above errors will lead to complete semantic errors and the planner cannot match the action preconditions, the validator or type checker will usually report a predicate mismatch or that no facts satisfying the conditions can be found when the action is instantiated. The closed-loop system will classify the error as a predicate signature inconsistency or parameter order error and give a repair instruction, requiring all at facts to be changed to at objects and locations, while keeping the object name unchanged. Since the repair is a local replacement, the risk of modification is small and it can quickly converge to an executable version.

[0109] In this embodiment, after the above iterative verification process, the optimized large model outputs the final planning scheme, which is the executable scheme.

[0110] The task planning method based on large model feedback optimization provided in this invention extracts error features from error logs and converts them into repair instructions. The repair instructions and PDDL symbol sequences are then input into a large language model with planning generation capabilities to obtain new candidate PDDL planning schemes. When the new candidate PDDL planning scheme passes the executability verification or the large language model with planning generation capabilities reaches the maximum number of training iterations, an executable planning scheme is obtained through an iterative verification process. Local replacement improves the model convergence efficiency and enhances the stability of model operation.

[0111] In some embodiments, after obtaining an executable planning scheme through an iterative verification process, the method further includes: calculating the parsing rate and the solvability rate based on the executability verification success rate of the PDDL planning scheme output by the large language model with planning generation capabilities; calculating the semantic similarity between the PDDL planning scheme output by the large language model with planning generation capabilities and the real PDDL planning scheme; parsing the PDDL planning scheme output by the large language model with planning generation capabilities and the corresponding real PDDL planning scheme respectively, and comparing the consistency of the two sets of parsing results to obtain the consistency rate; evaluating the large language model with planning generation capabilities based on at least one of the parsing rate, solvability rate, semantic similarity, and consistency rate to obtain the model evaluation result.

[0112] In this embodiment, the parsability rate is used to evaluate the grammatical compliance of PDDL files generated by large models. It measures whether the generated files can be parsed normally by classic planners (such as Fast Downward) and planning verification tools (VAL), i.e., to detect whether there are grammatical problems such as undefined variables, incorrect logical operators, and mismatched brackets. The parsability rate metric directly reflects the model's mastery of PDDL grammatical rules.

[0113] In this embodiment, the resolvability rate is expressed as: in the test set of NIn each sample, the proportion of PDDL files that can be successfully parsed by the planner out of the total sample size is calculated using the following formula:

[0114] ;

[0115] in, N The total number of samples in the test set; For the first i PDDL files generated from each test sample; This is an indicator function, representing a value of 1 if parsing is successful, and 0 otherwise; the range of the parsing success rate is... The closer the value is to 1, the fewer syntax errors the generated PDDL file has, and the stronger the modeling ability of the PDDL syntax rules.

[0116] In this embodiment, the solvability rate is a key indicator for further evaluating the logical feasibility of the PDDL file, based on the resolution rate. It measures whether the planning problem corresponding to the generated file can be successfully solved by the planner (i.e., the planner can generate a sequence of actionable steps from the initial state to the target state based on the file), ensuring that the generated symbol file is not only syntactically correct, but also conforms to the logical constraints of the planning task.

[0117] In this embodiment, the solvability rate is expressed as: in the test set of N In each sample, the proportion of PDDL files that can be successfully solved by the planner out of the total number of samples is calculated using the following formula:

[0118] ;

[0119] Here, planner(·) represents the planner, whose input is a PDDL file. If a feasible sequence of actions can be generated (i.e., the planning problem is solvable), return "SUCCESS"; otherwise, return "FAIL". SUCCESS is an indicator function that takes the value 1 if the corresponding planning problem is solvable, and takes the value 0 otherwise.

[0120] In this embodiment, semantic similarity is used to quantitatively evaluate the semantic overlap between the model-generated PDDL file and the real PDDL file in the dataset. It can measure the accuracy of the model's understanding of the core logic of the planning task and avoid the model generating PDDL files that are grammatically correct but semantically deviate.

[0121] In this embodiment, semantic similarity can be expressed as: in the test set N In each sample, a PDDL file is generated by calculating the text similarity of each sample using a text similarity algorithm. With the document The semantic similarity score is then taken as the average of the scores of all samples, specifically calculated using the following formula:

[0122] ;

[0123] Among them, the document For the first i The actual PDDL file corresponding to each test sample in the dataset; It is a function for calculating semantic similarity, and the output is a similarity score in the interval [0,1] (the higher the score, the higher the semantic overlap).

[0124] In this embodiment, the effectiveness of the modeling results is verified by comparing the solution results. That is, the generated PDDL file and the real PDDL file of the same test sample are respectively input into the same planner for solving, and the proportion of the two solution results that are consistent (such as whether action sequences can be generated and whether the core logic of the action sequences is consistent) is counted to obtain the corresponding consistency rate. This avoids the model generating PDDL files that are syntactically correct and semantically similar but cannot meet the requirements of real tasks.

[0125] In this embodiment, the consistency rate can be expressed as: in the test set N In a sample, the proportion of samples in which the generated file and the real file have the same solution result is calculated using the following formula:

[0126] ;

[0127] in, For the first i PDDL files generated from each test sample For the first i A real PDDL file for each test sample. This represents the solution results generated by the planner based on the PDDL file; the consistency rate ranges from [0,1]. The closer the value is to 1, the more likely the PDDL file generated by the model not only meets the requirements in terms of syntax and semantics, but also matches the solution results required by the real task.

[0128] The task planning method based on large model feedback optimization provided in this invention evaluates large language models with planning generation capabilities by calculating parsing rate, solvability rate, semantic similarity, and consistency rate. This multi-dimensional evaluation system can comprehensively verify the comprehensive capabilities of large models in NLP (Natural Language Processing)-PDDL mapping tasks, ensuring that the model can establish an accurate mapping from natural language to PDDL symbols, and guaranteeing the quality of generated symbol files through layer-by-layer verification, thus providing reliable symbolic input for subsequent planning and reasoning stages (such as symbol verification and action sequence generation).

[0129] In some embodiments, the natural language description of the problem is obtained through the following steps: the natural language description information of the target task is formatted using a structural template to obtain the natural language description of the problem; wherein, the structural template includes object declarations, initial state facts, and a set of target conditions, and the problem files corresponding to different target tasks adopt the same writing specifications, including object naming rules, type annotation style, consistency of predicate parameter order, and completeness checks of initial facts.

[0130] In this embodiment, during the specification construction phase, the predicate set, action set, type hierarchy and constraint form of the domain file are predetermined for each planning domain, and a structural template is defined for the generation of natural language into problem file. The structural template may include object declarations, initial state facts and target condition set.

[0131] It should be noted that, in order to improve cross-domain consistency, all issue files adopt a unified writing standard, such as object naming rules, type annotation style, consistency of predicate parameter order, and integrity checks of initial facts; and for common-sense constraints implicit in natural language descriptions, such as the initial location of goods, vehicle capacity limitations, or the requirement that an item must be loaded for transportation, this solution explicitly encodes them as predicates required for initial facts or action preconditions, thereby reducing ambiguity in the planner's search space.

[0132] In this embodiment, the natural language description of the problem (training data) is upgraded from containing only surface text alignment to including executable semantic alignment. The dataset not only provides natural language description and target PDDL, but also provides solvability labels, failure type distribution, and quality statistics such as plan length or cost that can be verified by the planner, so that the model can be exposed to signals consistent with the final evaluation during the training phase.

[0133] In this embodiment, by setting difficulty stratification, such as grouping by the number of objects or task complexity, the model can learn a gradual pattern from simple to complex, reducing mutation failures on larger-scale instances. This dataset and statistical system also provide a stable source of rewards and a baseline for subsequent preference optimization and closed-loop repair. In this way, a large-scale dataset and evaluation system for natural language processing to PDDL is constructed, covering multiple fields and supporting controlled expansion by instance size. Each sample has executable labels and planning statistics verified by the planner.

[0134] The task planning method based on large model feedback optimization provided in this invention provides a consistent signal source from training to evaluation by constructing a dataset and evaluation system covering multiple domains and difficulties with planner validation labels. This significantly improves the model's learning effect on executable semantics and enhances cross-domain generalization ability. Data hierarchies and planning statistics make it easier for the model to learn scalable structural rules, reducing the problem of only learning the format while ignoring semantic constraints. Externally, the data and evaluation system provided by the above method can also serve as a unified benchmark, supporting the comparability evaluation of different models and methods.

[0135] Figure 2 This is the second flowchart of the task planning method based on large model feedback optimization provided by this invention. Figure 2 In the illustrated embodiment, the operational mechanism includes: Channel 1: Closed-loop executability alignment during inference (main process); Channel 2: Dataset construction (NL-PDDL-Bench construction process); Channel 3: Training + Evaluation (two-stage optimization + metrics); In the main process, the natural language description (x) (corresponding to the natural language task description) is first converted using an LLM converter to generate a PDDL problem, represented as... The LLM configuration is as follows: Predicted Objects Initial state ,Target Next, symbolic verification and solving of the PDDL problem are performed. Specifically, syntax / signature checks are conducted using the planning and verification tool VAL, and the Fast Downward planner is used to solve and verify the solution, ultimately outputting an executable solution. Each of them This represents a specific planning action and provides symbolic feedback (FB) based on the error log of the output solution (including parsing errors / constraint non-compliance / missing key facts). (This process uses a minimum modification iteration = 0...) T max(e.g., 5) to update the input context semantics of the LLM converter; in the NL-PDDL-Bench construction process, 23 IPC-style domains and scalable instances (including objects 1–40, represented hierarchically by L1, L2, L3, and L4) are first generated through an IPC domain + problem generator. Symbolic instances are then generated using this domain and instance information to generate PDDL (including domains and problems). AutoPlanBench natural language alignment (including template / rule-based natural language rewriting and NL-PDDL pairing samples) is then performed. Subsequently, VAL parsing filters and FastDownward solvability filters (to address time / memory limitations) are used to perform multi-stage filtering on the aligned data to obtain an executable subset (including 13 domains and approximately 460,000 executable instances). The process involves executing pairs of plans, using these pairs as training samples for subsequent model training. The two-stage optimization + metric process includes Stage 1: LoRA-supervised fine-tuning (SFT) and Stage 2: DPO with signed preferences. In Stage 1, LoRA fine-tuning is performed on the NL–PDDL to learn a stable NL→PDDL mapping. In Stage 2, DPO is performed using preferences validated by the planner, assigning rewards to each plan, such as setting "y + executable / low cost; y - unsatisfiable or high cost". Finally, the trained LLM model is evaluated using the same VAL+FD process, with evaluation metrics including SVR (syntactic validity rate), PSR (planner success rate), TSR (atomic Jaccard similarity to the reference plan), and CR (cost difference between the planner and the reference plan).

[0136] The task planning apparatus based on large model feedback optimization provided by the present invention is described below. The task planning apparatus based on large model feedback optimization described below and the task planning method based on large model feedback optimization described above can be referred to in correspondence.

[0137] Figure 3 This is a schematic diagram of the structure of the task planning device based on large model feedback optimization provided by the present invention, including: a calculation module 310, a training module 320 and an optimization module 330.

[0138] The calculation module 310 is used to convert the natural language description of the problem into a sequence of symbols in the planning domain definition language (PDDL), and to calculate the output probability corresponding to the PDDL symbol sequence based on the LoRA parameters of the large language model.

[0139] The training module 320 is used to calculate the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task, and update the LoRA parameters according to the loss value. When the training of the large language model reaches the preset number of iterations or the loss converges, a large language model with planning and generation capabilities is trained.

[0140] The optimization module 330 is used to perform executability verification on the candidate PDDL planning scheme output by the large language model with planning generation capability. If the verification fails, the input context information of the large language model with planning generation capability is optimized according to the error log corresponding to the candidate PDDL planning scheme, and an executable planning scheme is obtained through an iterative verification process. The error log includes at least one of the following: predicate signature inconsistency, object not declared, type mismatch, missing key facts in the initial state, and unreachable target conditions.

[0141] The task planning device based on large model feedback optimization provided in this invention converts the natural language description of the problem into a PDDL symbol sequence, calculates the output probability corresponding to the PDDL symbol sequence according to the LoRA parameter, and then calculates the loss value between the output probability and the target PDDL symbol sequence to update the LoRA parameter. Finally, the feasibility of the candidate PDDL planning scheme output by the model is verified. If the verification fails, the input context information of the large language model with planning generation capability is optimized according to the error log, and an executable planning scheme is obtained through an iterative verification process. This method maps the locationable information output when the verification fails to a reusable error category and defines a repair strategy for each type of error. By limiting the scope of modification to only repair the smallest set that leads to unsolvability, new errors caused by full rewriting are avoided. In each iteration, the large language model uses the planner's solvability and verification as objective stopping conditions, making the results evaluable and reproducible.

[0142] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can call logical instructions in the memory 430 to execute a task planning method based on large model feedback optimization. This method includes: converting the natural language description of the problem into a PDDL symbol sequence (a domain-defined language for planning), and calculating the output probability corresponding to the PDDL symbol sequence based on the LoRA parameters of the large language model; calculating the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task, and updating the LoRA parameters based on the loss value; training a large language model with planning generation capabilities when the large language model training reaches a preset number of iterations or the loss converges; performing executability verification on the candidate PDDL planning schemes output by the large language model with planning generation capabilities; if the verification fails, optimizing the input context information of the large language model with planning generation capabilities based on the error logs corresponding to the candidate PDDL planning schemes, and obtaining an executable planning scheme through an iterative verification process; the error logs include at least one of the following: predicate signature inconsistency, undeclared object, type mismatch, missing key facts in the initial state, and unreachable target conditions.

[0143] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the task planning method based on large model feedback optimization provided by the above methods. The method includes: converting the natural language description of the problem into a PDDL symbol sequence of the planning domain definition language, and calculating the output probability corresponding to the PDDL symbol sequence according to the LoRA parameters of the large language model; calculating the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task, and updating the LoRA parameters according to the loss value; training a large language model with planning generation capability when the large language model training reaches a preset number of iterations or the loss converges; performing executability verification on the candidate PDDL planning scheme output by the large language model with planning generation capability; if the verification fails, optimizing the input context information of the large language model with planning generation capability according to the error log corresponding to the candidate PDDL planning scheme, and obtaining an executable planning scheme through an iterative verification process; the error log includes at least one of the following: predicate signature inconsistency, object undeclared, type mismatch, missing key facts in the initial state, and unreachable target conditions.

[0145] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task planning method based on large model feedback optimization, characterized in that, include: The natural language description of the problem is converted into a sequence of symbols in the planning domain definition language (PDDL), and the output probability corresponding to the PDDL symbol sequence is calculated based on the LoRA parameters of the large language model. Calculate the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task, and update the LoRA parameter according to the loss value. If the large language model training reaches a preset number of iterations or the loss converges, a large language model with planning and generation capabilities is trained. The feasibility of the candidate PDDL planning schemes output by the large language model with planning generation capability is verified. If the verification fails, the input context information of the large language model with planning generation capability is optimized according to the error logs corresponding to the candidate PDDL planning schemes, and an executable planning scheme is obtained through an iterative verification process. The error logs include at least one of the following: predicate signature inconsistency, object not declared, type mismatch, missing key facts in the initial state, and unreachable target conditions.

2. The task planning method based on large model feedback optimization according to claim 1, characterized in that, The calculation of the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task includes: The loss value between the output probability and the target PDDL symbol sequence corresponding to the target task is calculated based on the negative log-likelihood loss function. The negative log-likelihood loss function is expressed by the following formula: ； in, Let be the negative log-likelihood loss function. The length of the target PDDL symbol sequence. For the first t One target token; Is the model in parameters Below, based on x With input context information Generate target The probability, x This describes a natural language task.

3. The task planning method based on large model feedback optimization according to claim 1, characterized in that, After obtaining a large language model with planning and generation capabilities through training, the method further includes: The multiple candidate PDDL planning schemes output by the large language model with planning generation capability are sampled to obtain a set of candidate PDDL planning schemes. Each candidate PDDL planning scheme in the candidate PDDL planning scheme set is tested for feasibility. The candidate PDDL planning schemes that pass the test are used as positive samples, and the candidate PDDL planning schemes that fail the test are used as negative samples. A reward function is then constructed. The relative advantage of each candidate PDDL planning scheme is calculated based on the reward function. A joint loss is constructed using the relative advantage, KL constraint, and negative log-likelihood loss function. The LoRA parameters of the large language model with planning generation capability are updated based on the joint loss. A new large language model is obtained when the large language model with planning generation capability reaches a preset number of iterations or the loss converges.

4. The task planning method based on large model feedback optimization according to claim 1, characterized in that, The step of optimizing the input context information of the large language model with planning generation capability based on the error logs corresponding to the candidate PDDL planning schemes, and obtaining an executable planning scheme through an iterative verification process, includes: Error features are extracted from the error log and converted into repair instructions. The repair instructions are used to instruct the large language model with planning generation capabilities to adjust the error content in the candidate PDDL planning scheme. The repair instructions and the PDDL symbol sequence are input into the large language model with planning generation capability to obtain a new candidate PDDL planning scheme. If the new candidate PDDL planning scheme passes the executability verification or the large language model with planning generation capability reaches the maximum number of training iterations, an executable planning scheme is obtained.

5. The task planning method based on large model feedback optimization according to claim 1, wherein the feasibility verification of the candidate PDDL planning scheme output by the large language model with planning generation capability includes: The candidate PDDL planning scheme is parsed using a Fast Downward planner to obtain the parsed task state and the parsed task objective; If the parsed task state matches the initial state corresponding to the target task, and the parsed task objective is consistent with the target task, then the candidate PDDL planning scheme is determined to be an executable scheme.

6. The task planning method based on large model feedback optimization according to claim 1, after obtaining an executable planning scheme through the iterative verification process, the method further includes: The parsing rate and solvability rate are calculated based on the executability verification success rate of the PDDL planning scheme output by the large language model with planning generation capability. Calculate the semantic similarity between the PDDL planning scheme output by the large language model with planning generation capability and the real PDDL planning scheme; The PDDL planning scheme output by the large language model with planning generation capability is analyzed and the corresponding real PDDL planning scheme is analyzed respectively. The consistency of the two sets of analysis results is compared to obtain the consistency rate. The large language model with planning generation capability is evaluated based on at least one of the parsable rate, the solvable rate, the semantic similarity, and the consistency rate to obtain the model evaluation result.

7. The task planning method based on large model feedback optimization according to claim 1, wherein the natural language description of the problem is obtained through the following steps: The natural language description information of the target task is formatted using a structural template to obtain a natural language description of the problem; wherein, The structural template includes object declarations, initial state facts, and a set of target conditions. Problem files corresponding to different target tasks adopt the same writing standards, which include object naming rules, type annotation style, consistency of predicate parameter order, and completeness checks of initial facts.

8. A task planning device based on large model feedback optimization, characterized in that, include: The computation module is used to convert the natural language description of the problem into a sequence of symbols in the planning domain definition language (PDDL), and to calculate the output probability corresponding to the PDDL symbol sequence based on the LoRA parameters of the large language model. The training module is used to calculate the loss value between the output probability and the target PDDL symbol sequence corresponding to the target task, and update the LoRA parameter according to the loss value. When the large language model training reaches a preset number of iterations or the loss converges, a large language model with planning and generation capabilities is trained. The optimization module is used to perform executability verification on the candidate PDDL planning schemes output by the large language model with planning generation capability. If the verification fails, the module optimizes the input context information of the large language model with planning generation capability based on the error logs corresponding to the candidate PDDL planning schemes, and obtains an executable planning scheme through an iterative verification process. The error logs include at least one of the following: predicate signature inconsistency, object not declared, type mismatch, missing key facts in the initial state, and unreachable target conditions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the task planning method based on large model feedback optimization as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the task planning method based on large model feedback optimization as described in any one of claims 1 to 7.

Images