Continuous task planning system and method based on asynchronous causal inference reflection

Abstract

The present invention belongs to the field of information processing technology, and discloses a continuous task planning system and method based on asynchronous causal inference reflection. Based on a large language model, through an actor module, task objectives and environmental observations are used as inputs, and the next action is predicted as output; through an association reasoning module, the association information between things is extracted from the environmental observation of the current trajectory, and the first level of hierarchical causal inference is performed to determine the association between things; through an intervention plan generation module, the second level of hierarchical causal inference is performed according to the new trajectory obtained by counterfactual reasoning and the previously extracted association information, and a plan for the next intervention implementation is generated to verify the cause of task failure; through a counterfactual reasoning module, the task trajectory is regenerated under the conditions of the task objectives and task trajectory, and the cause of the failure of the previous attempt is determined. The success rate of long-term task planning is significantly improved.

Description

Technical Field

[0001] The present invention belongs to the field of information processing technology, and in particular relates to a continuous task planning system and method based on asynchronous causal inference reflection. Background Art

[0002] Task planning refers to generating a feasible sequence of actions to complete a task given the task goal and environmental information. Task planning has a wide range of applications in the field of artificial intelligence, including robot control, autonomous driving, logistics planning, and daily housework. In recent years, large language models have been enhanced into autonomous decision-making language agents with reflective capabilities due to their excellent commonsense reasoning and question-answering capabilities. Reflective feedback is an improvement plan obtained by reflecting on previous failed behaviors. This method is used as a "semantic" gradient signal to optimize future behaviors. The method of using reflective optimization instructions can achieve task goals through continuous instruction adjustment optimization without re-adjusting the parameters of the large language model. However, current reflective methods mainly focus on single-step or multi-step decision tasks, and there is still a lack of effective solutions for asynchronous causal attribution analysis of long-term task planning. The reasons for failure in long-term task planning are often asynchronous, and there may be a time difference between the actual cause of task failure and the step that failed to execute. For example, the reason why an item is not placed correctly may be because the item was not found or picked up correctly before. Therefore, relying solely on problem feedback and optimization plans from task execution failures may not accurately locate the cause of failure, and may even lead to greater errors and catastrophic results. Summary of the invention

[0003] The purpose of the present invention is to provide a continuous task planning system and method based on asynchronous causal inference reflection to solve the above-mentioned technical problems.

[0004] In order to solve the above technical problems, the specific technical solutions of a continuous task planning system and method based on asynchronous causal inference and reflection of the present invention are as follows:

[0005] A continuous task planning system based on asynchronous causal inference reflection, including an actor module, an association reasoning module, an intervention plan generation module and a counterfactual reasoning module;

[0006] The actor module outputs the result of task planning based on the current task goal and state information based on the context-based autoregressive large language model;

[0007] The association reasoning module is composed of a large language model, extracts association information from the current environment observation, and uses the first level of association reasoning through hierarchical causal inference to determine the association between items in the failed trajectory of the previous attempt;

[0008] The intervention plan generation module is composed of a large language model, and generates a plan for the next intervention implementation based on the previously extracted association information and counterfactual reasoning information through hierarchical causal inference at the second level of reasoning, that is, to verify the cause of task failure;

[0009] The counterfactual reasoning module is composed of a large language model. According to the current task trajectory, the third level of counterfactual reasoning is performed through hierarchical causal inference. By imagining the results of actions in different scenarios, counterfactual reasoning feedback is generated to determine the cause of task failure and optimize the subsequent intervention plan.

[0010] Furthermore, the counterfactual reasoning module is the highest level inference in causal inference. This module performs counterfactual inference on the last attempted action. When performing causal inference, the highest level counterfactual inference of the last attempt is first performed, and then the low level inference of this attempt is completed.

[0011] The present invention also discloses a continuous task planning method of a continuous task planning system based on asynchronous causal inference reflection, comprising the following steps:

[0012] S1, the actor module outputs the optimal action for the agent based on the current task information and environment information;

[0013] S2, input the action or decision output to the agent for execution, interact with the environment, and observe the environmental observation information after the action is executed ;

[0014] S3, determine whether the current environmental status information meets the task objectives If the task goal is not met, return to step 1 and continue to perform the next task. If the maximum number of steps is not reached, repeat steps 1-3. If the maximum number of steps is reached, the task fails and steps 4-7 are executed before the next attempt.

[0015] S4, the counterfactual inference module integrates the information of planning failure through the third level of hierarchical causal inference and outputs the reasons for planning failure;

[0016] S5, the association reasoning module integrates the task planning trajectory information through the first level of association reasoning of hierarchical causal inference, and outputs the association information between the object and the container;

[0017] S6, the intervention plan generation module integrates the information generated by the counterfactual reasoning module and the associative reasoning module through the second level of reasoning of hierarchical causal inference, and outputs the intervention plan for the next task completion;

[0018] S7, Access to intervention plans After that, go back to step 1 and add it to the asynchronous causal inference reflection information of the state information and then proceed to the next attempt.

[0019] Furthermore, the specific steps of S1 are as follows:

[0020] S101, obtain a large language model M Actor Module as Subject , the actor module The input parameters include the demonstration of mission planning And the following two parts of input information;

[0021] S102, obtaining text input, the text input includes the current task target And including historical action tracks , Environmental Observation and asynchronous causal inference and reflection information Status information ,Will and Add to input parameters;

[0022] S103, input the current input parameters into the actor module , by following the demonstration , get the corresponding action or decision output ,Right now .

[0023] Furthermore, the specific steps of S4 are as follows:

[0024] S401, obtaining a large language model M Counterfactual Reasoning Module as Agent , the counterfactual reasoning module The input parameters include a demonstration of determining the cause of planning failure And the following two parts of input information;

[0025] S402, get execution m Failed task trace after step 、Mission Objectives , Execution Results and counterfactual scenarios for achieving mission objectives Add to input parameters, input to counterfactual reasoning module , by following the demonstration , identify and modify the key steps of the original trajectory that have errors, and rely on the counterfactual reasoning ability of the large language model to obtain the potential results of different steps without actual execution, analyze the root causes of past failures, and obtain a new task trajectory that can complete the task goal based on the minimal modification of the original trajectory ,Right now .

[0026] Furthermore, the specific steps of S5 are as follows:

[0027] S501, obtaining a large language model M Associative reasoning module as the main body , the association reasoning module The input parameters are derived from the demonstration of the associated information in the inferred task trajectory And the following input information consists of two parts;

[0028] S502, obtaining failed task traces And the mission objectives Add to input parameters and input to the association reasoning module , by following the demonstration , using the association reasoning ability of the large language model to identify environmental observations in the trajectory, and infer the association between objects and containers, and obtain the association information between objects and containers in the trajectory ,Right now .

[0029] Furthermore, the specific steps of S6 are as follows:

[0030] S601, obtaining a large language model M Intervention plan generation module as the main body , the intervention plan generation module The input parameters include combining the information generated by the counterfactual reasoning module and the association reasoning module to generate a demonstration of the intervention plan And the following two parts of input information;

[0031] S602: The task trajectory generated by the counterfactual reasoning module and the associative reasoning module to meet the task goal And the association information between the object and the container Added to input parameters, input to intervention plan generation module , by following the demonstration , according to the task trajectory and the container where the object required in the task goal is located, formulate an intervention plan for the next action ,Right now .

[0032] Furthermore, the large language modelM Get text output through text input. The specific steps are as follows:

[0033] S801, get the input text ,in Indicates i words or tokens, the length of the text sequence is n , first, each word is converted into a vector representation of fixed dimension through an embedding layer:

[0034] ,

[0035] in, It is a word The embedding vector of d is the embedding dimension;

[0036] S802, the embedded vector sequence is passed to the multi-layer self-attention mechanism Transformer. The self-attention mechanism is applied to each layer to capture context information. The self-attention calculation formula of the layer is:

[0037] ,

[0038] in, , , is the weight matrix of query, key, value, and attention score:

[0039] ,

[0040] The output of each layer is passed to the next layer through residual connection and layer normalization until the last layer obtains the context representation. ;

[0041] S803, the final representation after Transformer encoding , will be passed to the output layer to generate the output probability of each word unit. Assuming that the target word unit is , the model predicts its probability distribution as follows:

[0042] ,

[0043] in, and is the weight matrix and bias term of the output layer;

[0044] S804, finally, through the greedy algorithm The word with the highest probability at each step is selected as the output of the current step until the end symbol is generated or the maximum sequence length is reached.

[0045] The continuous task planning system and method based on asynchronous causal inference reflection of the present invention has the following advantages:

[0046] The present invention provides a reflection framework based on asynchronous causal inference to improve the decision-making effect of autonomous decision-making language agents with reflection ability in long-term task planning. The present invention uses an actor module based on a large language model to take task objectives and environmental observations as inputs and predict the next action as output. The present invention uses an association reasoning module based on a large language model to extract association information between things from environmental observations of the current trajectory, and performs causal inference at the first level of hierarchical causal inference to determine the correlation between events. The present invention uses an intervention plan generation module based on a large language model to perform causal inference at the second level of hierarchical causal inference based on the new trajectory obtained by counterfactual reasoning and the previously extracted association information, and generates a plan for the next intervention implementation to verify the cause of task failure. The present invention uses a counterfactual reasoning module based on a large language model to think about the cause of failure of the previous attempt, and regenerates the task trajectory under the conditions of task objectives and task trajectory. The present invention combines three levels of hierarchical causal inference to perform causal inference, comprehensively analyzes the cause of task failure, and generates an optimized action plan according to the intervention plan, which can significantly improve the success rate of long-term task planning. In addition, the association reasoning module, intervention plan generation module and counterfactual reasoning module of the present invention do not rely on the inherent structure of a specific framework, can be combined with any reflection framework and can be easily paired with various basic planning methods. This means that there is no need to retrain or fine-tune the language model, thereby saving time and computing resources. This makes the present invention easy to apply in practical applications and reduces the complexity of deployment and integration. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the application framework of the method in the present invention.

[0048] Figure 2 Schematic diagram of an actor module in an embodiment of the present invention.

[0049] Figure 3 Schematic diagram of a counterfactual reasoning module in an embodiment of the present invention.

[0050] Figure 4 Schematic diagram of an association reasoning module in an embodiment of the present invention.

[0051] Figure 5 Schematic diagram of an intervention plan generation module in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a continuous task planning system and method based on asynchronous causal inference and reflection of the present invention in conjunction with the accompanying drawings.

[0053] like Figure 1 As shown, a continuous task planning system based on asynchronous causal inference and reflection of the present invention includes an actor module, an association reasoning module, an intervention plan generation module and a counterfactual reasoning module.

[0054] The actor module, based on the context-based autoregressive large language model, outputs the result of task planning according to the current task goal and state information. The task goal and state information are usually text in natural language form. The result of task planning is usually the next action or decision.

[0055] The association reasoning module is composed of a large language model, which can extract association information from the current environmental observation and determine the association between items in the failed trajectory of the previous attempt through hierarchical causal inference at the first level of association reasoning;

[0056] The intervention plan generation module is composed of a large language model, which can generate a plan for the next intervention implementation through hierarchical causal inference based on the previously extracted association information and counterfactual reasoning information to verify the cause of the task failure;

[0057] The counterfactual reasoning module is composed of a large language model, which can generate counterfactual reasoning feedback by imagining the results of actions in different scenarios through hierarchical causal inference based on the current task trajectory to determine the cause of task failure and optimize subsequent intervention plans. The counterfactual reasoning module is the highest level of inference in causal inference. This module performs counterfactual inference on the last attempted action. Therefore, when performing causal inference, the highest level of counterfactual inference of the last attempt is performed first, and then the low-level inference of this attempt is completed.

[0058] A continuous task planning method based on asynchronous causal inference reflection of the present invention comprises the following steps:

[0059] S1, the actor module outputs the optimal action for the agent based on the current task information and environment information. The specific steps are as follows:

[0060] S101, obtain a large language model M Actor Module as Subject Among them, the actor module The input parameters include the demonstration of mission planning And the following two parts of input information;

[0061] S102, obtaining text input, the text input includes the current task target And including historical action tracks , Environmental Observation and asynchronous causal inference and reflection information Status information ,Will and Add to input parameters;

[0062] S103, input the current input parameters into the actor module , by following the demonstration , get the corresponding action or decision output ,Right now ;

[0063] S2, input the action or decision output to the agent for execution, interact with the environment, and observe the environmental observation information after the action is executed ;

[0064] S3, determine whether the current environmental status information meets the task objectives If the task goal is not met, return to step 1 and continue to perform the next task. If the maximum number of steps is not reached, repeat steps 1-3. If the maximum number of steps is reached, the task fails and steps 4-7 are executed before the next attempt.

[0065] S4, the counterfactual inference module integrates the information of planning failure through the third level of hierarchical causal inference and outputs the reasons for planning failure. The specific steps are as follows:

[0066] S401, obtaining a large language model M Counterfactual Reasoning Module as Agent Among them, the counterfactual reasoning module The input parameters include a demonstration of determining the cause of planning failure And the following two parts of input information;

[0067] S402, get execution m Failed task trace after step 、Mission Objectives , Execution Results and counterfactual scenarios for achieving mission objectives Add to input parameters, input to counterfactual reasoning module , by following the demonstration , identify and modify the key steps of the original trajectory that have errors, and rely on the counterfactual reasoning ability of the large language model to obtain the potential results of different steps without actual execution, analyze the root causes of past failures, and obtain a new task trajectory that can complete the task goal based on the minimal modification of the original trajectory ,Right now ;

[0068] S5, the association reasoning module integrates the task planning trajectory information through the first level of hierarchical causal reasoning, and outputs the association information between the object and the container. The specific steps are as follows:

[0069] S501, obtaining a large language model M Associative reasoning module as the main body Among them, the association reasoning module The input parameters are derived from the demonstration of the associated information in the inferred task trajectory And the following input information consists of two parts;

[0070] S502, obtaining failed task traces And the mission objectives Add to input parameters and input to the association reasoning module , by following the demonstration , using the association reasoning ability of the large language model to identify environmental observations in the trajectory, and infer the association between objects and containers, and obtain the association information between objects and containers in the trajectory ,Right now ;

[0071] S6, the intervention plan generation module integrates the information generated by the counterfactual reasoning module and the association reasoning module through the second level of hierarchical causal inference, and outputs the intervention plan for the next task completion. The specific steps are as follows:

[0072] S601, obtaining a large language model M Intervention plan generation module as the main body Among them, the intervention plan generation module The input parameters include combining the information generated by the counterfactual reasoning module and the association reasoning module to generate a demonstration of the intervention plan And the following two parts of input information;

[0073] S602: The task trajectory generated by the counterfactual reasoning module and the associative reasoning module to meet the task goal And the association information between the object and the container Added to input parameters, input to intervention plan generation module , by following the demonstration , according to the task trajectory and the container where the object required in the task goal is located, formulate an intervention plan for the next action ,Right now ;

[0074] S7, Access to intervention plans After that, go back to step 1 and add it to the asynchronous causal inference reflection information of the state information and then proceed to the next attempt.

[0075] In the above steps, input the large language model M The specific steps to obtain text output through text input are as follows:

[0076] S801, get the input text ,in Indicates i words or tokens, the length of the text sequence is n First, each word is converted into a vector representation of fixed dimension through an embedding layer:

[0077]

[0078] in, It is a word The embedding vector of d is the embedding dimension.

[0079] S802, the embedded vector sequence is passed to the multi-layer self-attention mechanism Transformer, and the self-attention mechanism is applied to each layer to capture context information. The self-attention calculation formula of the layer is:

[0080]

[0081] in, , , is the query ( Query ),key( Key ),value( Value ). The attention score is:

[0082]

[0083] The output of each layer is passed to the next layer through residual connection and layer normalization until the last layer obtains the context representation. .

[0084] S803, the final representation after Transformer encoding , will be passed to the output layer to generate the output probability of each word. Assume that the target word is , the model predicts its probability distribution as follows:

[0085]

[0086] in, and is the weight matrix and bias term of the output layer.

[0087] S804, finally, through the greedy algorithm The word with the highest probability at each step is selected as the output of the current step until the end symbol is generated or the maximum sequence length is reached. Example

[0088] like Figure 2 As shown in FIG. 1 , the actor module based on the large language model receives the input task goal and state information and outputs the corresponding next action. For example, receiving the task goal of “checking the pen with a desk lamp”, the next action of “going to drawer 1” is given.

[0089] like Figure 3 As shown in the figure, the counterfactual reasoning module based on the large language model takes the task information and the set execution result information as input, and generates a new task trajectory that can complete the task goal based on the minimal modification of the original trajectory by setting the action trajectory in the scenario of successful task results, so as to determine the cause of task failure and optimize the subsequent intervention plan. Among them, the newly generated trajectory is compressed into a trajectory described by natural language in order to reduce the length. For example: after counterfactual reasoning is performed on the trajectory of the unfinished task goal of "putting two pillows on the sofa", after replacing the key steps that made the error, we get a new trajectory that after placing the first pillow, we should find and place the second pillow.

[0090] like Figure 4 As shown in the figure, the association reasoning module based on the large language model takes the task goal and trajectory as input, infers the association between objects and containers through observations in the trajectory, and thus gives the association information between objects and containers. For example, in the trajectory observation of completing the task of "putting two pillows on the sofa", the association information between objects and containers such as "I once saw pillow 1 on the armchair" is inferred.

[0091] like Figure 5As shown, the task trajectory that meets the task goal and the association information between objects and containers generated by the counterfactual reasoning module and the association reasoning module are input into the intervention plan generation module based on the large language model, and we can get the intervention plan for the next attempt. For example: In the intervention plan, we should first find "pillow 1". By combining the object association information, we can know "pillow 1 on armchair 1", so the first step is "go to armchair 1". Then, according to the new trajectory obtained by counterfactual reasoning, we should find and place "pillow 2", and combine the location information of "pillow 2" in the object association information to get the intervention plan, so that the task goal can be successfully completed in the next attempt.

[0092] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A method for continuous task planning of a continuous task planning system based on asynchronous causal inference and reflection, the system comprising an actor module, an association reasoning module, an intervention plan generation module and a counterfactual reasoning module; The actor module outputs the result of task planning based on the current task goal and state information based on the context-based autoregressive large language model; The association reasoning module is composed of a large language model, extracts association information from the current environment observation, and uses the first level of association reasoning through hierarchical causal inference to determine the association between items in the failed trajectory of the previous attempt; The intervention plan generation module is composed of a large language model, and generates a plan for the next intervention implementation based on the previously extracted association information and counterfactual reasoning information through hierarchical causal inference at the second level of reasoning, that is, to verify the cause of task failure; The counterfactual reasoning module is composed of a large language model, and generates counterfactual reasoning feedback by imagining the results of actions in different scenarios based on the current task trajectory through hierarchical causal inference to determine the cause of task failure and optimize the subsequent intervention plan; It is characterized in that The method comprises the following steps: S1, the actor module outputs the optimal action for the agent based on the current task information and environment information; S2, input the action or decision output to the agent for execution, interact with the environment, and observe the environmental observation information after the action is executed ; S3, determine whether the current environmental status information meets the task objectives If the task goal is not met, return to step 1 and continue to perform the next task. If the maximum number of steps is not reached, repeat steps 1-3. If the maximum number of steps is reached, the task fails and steps 4-7 are executed before the next attempt. S4, the counterfactual inference module integrates the information of planning failure through the third level of hierarchical causal inference and outputs the reasons for planning failure; S5, the association reasoning module integrates the task planning trajectory information through the first level of association reasoning of hierarchical causal inference, and outputs the association information between the object and the container; S6, the intervention plan generation module integrates the information generated by the counterfactual reasoning module and the associative reasoning module through the second level of reasoning of hierarchical causal inference, and outputs the intervention plan for the next task completion; S7, Access to intervention plans After that, go back to step 1 and add it to the asynchronous causal inference reflection information of the state information and then proceed to the next attempt.

2. The method according to claim 1, characterized in that: The counterfactual reasoning module is the highest level inference in causal inference. This module performs counterfactual inference on the last attempted action. When performing causal inference, the highest level counterfactual inference of the last attempt is first performed, and then the low level inference of this attempt is completed.

3. The method according to claim 1, characterized in that The specific steps of S1 are as follows: S101, obtain a large language model M Actor Module as Subject , the actor module The input parameters include the demonstration of mission planning And the following two parts of input information; S102, obtaining text input, the text input includes the current task target And including historical action tracks , Environmental Observation and asynchronous causal inference and reflection information Status information ,Will and Add to input parameters; S103, input the current input parameters into the actor module , by following the demonstration , get the corresponding action or decision output ,Right now .

4. The method according to claim 1, characterized in that: The specific steps of S4 are as follows: S401, obtaining a large language model M Counterfactual Reasoning Module as Agent , the counterfactual reasoning module The input parameters include a demonstration of determining the cause of planning failure And the following two parts of input information; S402, get execution m Failed task trace after step 、Mission Objectives , Execution Results and counterfactual scenarios for achieving mission objectives Add to input parameters, input to counterfactual reasoning module , by following the demonstration , identify and modify the key steps of the original trajectory that have errors, and rely on the counterfactual reasoning ability of the large language model to obtain the potential results of different steps without actual execution, analyze the root causes of past failures, and obtain a new task trajectory that can complete the task goal based on the minimal modification of the original trajectory ,Right now .

5. The method according to claim 1, characterized in that: The specific steps of S5 are as follows: S501, obtaining a large language model M Associative reasoning module as the main body , the association reasoning module The input parameters are derived from the demonstration of the associated information in the inferred task trajectory And the following input information consists of two parts; S502, obtaining failed task traces And the mission objectives Add to input parameters and input to the association reasoning module , by following the demonstration , using the associative reasoning ability of the large language model to identify environmental observations in the trajectory, and infer the association between objects and containers, and obtain the association information between objects and containers in the trajectory ,Right now .

6. The method according to claim 1, characterized in that The specific steps of S6 are as follows: S601, obtaining a large language model M Intervention plan generation module as the main body , the intervention plan generation module The input parameters include combining the information generated by the counterfactual reasoning module and the association reasoning module to generate a demonstration of the intervention plan And the following two parts of input information; S602: The task trajectory generated by the counterfactual reasoning module and the associative reasoning module to meet the task goal And the association information between the object and the container Added to input parameters, input to intervention plan generation module , by following the demonstration , according to the task trajectory and the container where the object required in the task goal is located, formulate an intervention plan for the next action ,Right now .

7. The method according to any one of claims 3 to 6, characterized in that: The large language model M Get text output through text input. The specific steps are as follows: S801, get the input text ,in Indicates i words or tokens, the length of the text sequence is n , first, each word is converted into a vector representation of fixed dimension through an embedding layer: ， in, It is a word The embedding vector of d is the embedding dimension; S802, the embedded vector sequence is passed to the multi-layer self-attention mechanism Transformer. The self-attention mechanism is applied to each layer to capture context information. The self-attention calculation formula of the layer is: ， in, , , is the weight matrix of query, key, value, and attention score: ， The output of each layer is passed to the next layer through residual connection and layer normalization until the last layer obtains the context representation. ; S803, the final representation after Transformer encoding , will be passed to the output layer to generate the output probability of each word unit. Assuming that the target word unit is , the model predicts its probability distribution as follows: ， in, and is the weight matrix and bias term of the output layer; S804, finally, through the greedy algorithm The word with the highest probability at each step is selected as the output of the current step until the end symbol is generated or the maximum sequence length is reached.

Images