Reactive planning method and system based on large language model

Abstract

This invention provides a reactive planning method based on a large language model, comprising: designing replanning logic, including multi-step logic and three-hop logic, analyzing the impact of various factors on the completion of the task objective step by step, and updating the current plan in a chain; each step employs reasoning logic including cause analysis logic, conclusion derivation logic, and plan adjustment logic; designing replanning prompts; receiving user commands, generating an initial plan based on user instructions, storing the initial plan, and recording the current plan's progress information; detecting and capturing environmental change information through the robot, and iteratively updating the plan based on environmental change information, plan progress information, user commands, replanning prompts, and replanning logic. This invention also provides a reactive planning system, storage medium, and electronic device based on a large language model. Therefore, this invention enables low-cost, high-accuracy, and high-generalization robot reactive planning.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of artificial intelligence and robotics task planning, and in particular to a reactive planning method, system, storage medium and electronic device based on a large language model. Background Technology

[0002] Robot task planning mainly studies how to enable robots to generate and execute action sequences in a relatively static environment to complete the task objectives corresponding to user commands. Currently, many classic planning methods are used to solve this problem. However, in real life, the environment is often highly variable due to the influence of humans, equipment, or intelligent agents. This variability includes: (1) changes in the position of objects. For example, a water glass is moved from the dining table to the refrigerator. (2) changes in the properties of objects, such as the start and stop of a microwave oven. These changes will have different effects on the task execution of the robot in the current environment. They may have no effect, such as when the robot opens the refrigerator door while opening the microwave oven, the robot will normally execute the next action of the plan. They may have a facilitating effect, such as when the robot finds that the food is already hot when it intends to heat it, the robot needs to remove the action of heating the food from the plan. They may also have a hindering effect, such as when the robot closes the refrigerator door when it is about to put an object in the refrigerator, in this complex situation, the robot needs to remove or add actions to obtain a new plan. In summary, robots need to be able to assess the impact of environmental changes on task execution and then adjust their current plans to adapt to these changes, aligning with the task objectives. Therefore, solving the problem of robots completing user tasks in changing environments (the reactive planning problem) becomes crucial.

[0003] Modeling the replanning logic is the core of solving reactive programming problems. Current research methods can be broadly categorized into two types: the first is script-based methods, the most classic being reactive behavior trees. These methods explicitly model the logical rules of replanning, such as the preconditions and postconditions of actions, and then use deterministic algorithms or processes to reason based on these rules for replanning. However, these methods require a large amount of predefined rules and lack flexibility in adapting to various environmental changes in complex situations. Later, with the rapid development of machine learning technology, a second type of learning-based methods emerged. These methods implicitly model the replanning logic using machine learning, especially deep learning techniques (such as reinforcement learning). While these methods do not require a large number of finely defined rules, they are mostly data-driven and heavily rely on large, evenly distributed training datasets and carefully designed learning strategies to ensure replanning performance. Such high-quality datasets are often difficult to obtain, and suitable learning strategies require continuous human experimentation to discover. Furthermore, learning-based methods cannot smoothly transfer to new environmental changes unseen during training.

[0004] In recent years, with the emergence and development of LLMs (Large Language Models), new approaches have been found to solve reactive planning problems. Since LLMs have already learned massive amounts of world semantic knowledge during the pre-training phase, including common sense and subject-specific knowledge, they can be naturally used to solve everyday planning problems without requiring extensive pre-definition or additional training. In the past two years, a small number of works have been proposed based on LLMs to address reactive planning sub-problems such as action execution failure and uncertain environmental states. The Inner Monologue approach, proposed by Google's robotics team, is inspired by the human concept of "inner monologue." In this approach, LLMs communicate with other information feedback modules via text to obtain information about task progress and environmental states, while simultaneously predicting the next action to be performed. Other modules mentioned in the paper include: a Success Detector that provides the action execution status; a Passive Scene Description module that provides a list of interactive objects in visual observation; and an Active Scene Description module that provides answers according to the LLMs' needs. If the success detector determines that the current action has failed, the method will re-execute the action. If there are changes in the position or state of objects in the environment, the method can obtain the latest state information of the environment from the active environment description module by asking questions of the LLMs. However, the communication process between the LLMs and other modules in this method is difficult to control, and there are cases where the LLMs do not actively obtain the environment state. This causes the LLMs to ignore important feedback information and make incorrect decisions.

[0005] Prog-prompt is a procedural planning method proposed by the NVIDIA team. It leverages the powerful coding capabilities of LLMs, treating the process of a robot planning a sequence of actions as writing a task completion function in a Python-like language, with assertions within the function body. The assertion mechanism allows prog-prompts to generate tree-like plans with multiple branches, rather than a linear plan. Each branch corresponds to a consideration of uncertainty in a certain environmental state. For example, when a robot is about to put an apple in the refrigerator, it first uses an assertion to determine if the refrigerator door is open. If it is open, it puts the apple in; otherwise, it needs to open the door first. However, the plans generated by this method are fixed and cannot be dynamically adjusted later, causing the robot to be unable to cope with environmental changes not covered by the assertions.

[0006] CAPE (Contrastive Parameter Ensembling) is a replanning method proposed by Brown University based on the discrimination of action execution conditions. It posits that each action has preconditions that must be met before execution; the action can only be executed if the environmental state meets these conditions, otherwise, the next action is predicted. It first defines a table of action precondition error types, listing the most likely scenarios where the robot will encounter unmet preconditions. These error types are then described in text as guiding statements and passed to LLMs (Limited Learning Machines), which then plan the new action. However, this method only uses the preconditions of the current action as the sole basis for controlling replanning, ignoring other important environmental information unrelated to the current action that could lead to task failure. For example, if a robot is instructed to put a clean apple into a bowl, but the apple becomes dirty again when it tries to put it in, CAPE will ignore the cleanliness of the apple because the "put it in" action can be performed normally, leading to task failure.

[0007] Furthermore, the tendency of LLMs to "fantasize" has been confirmed in articles from multiple fields, including natural language processing and robotic task planning. This "fantasizing" manifests in several ways: ignoring strong constraints on robot configuration and environmental information, leading to plans that do not reflect reality; overlooking the impact of certain factors on task execution, resulting in insufficient plan adjustments; assuming non-existent elements during reasoning, leading to unreasonable plan adjustment strategies; and neglecting the standardization of plan adjustments, resulting in newly generated plans that are ultimately unexecutable. Therefore, overcoming the "fantasizing" tendency of LLMs while fully leveraging their powerful language understanding and logical reasoning capabilities is a key challenge in solving reactive planning problems using LLMs.

[0008] In conclusion, the existing technology obviously has inconveniences and defects in practical use, so it is necessary to improve it. Summary of the Invention

[0009] To address the aforementioned shortcomings, the present invention aims to provide a reactive planning method, system, storage medium, and electronic device based on a large language model, which can achieve low-cost, high-accuracy, and high-generalization robot reactive planning.

[0010] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0011] In a first aspect, embodiments of the present invention provide a reactive programming method based on a large language model, comprising:

[0012] The design steps of the replanning logic include multi-step logic and three-hop logic. The multi-step logic includes analyzing the impact of various factors on the completion of the task objective step by step and updating the current plan in a chain. The three-hop logic includes reasoning logic that uses cause analysis logic, conclusion derivation logic and plan adjustment logic in each step.

[0013] The design steps for redesigning prompts include problem statements, constraint statements, and case demonstrations;

[0014] The initialization plan includes receiving user commands, generating an initialization plan based on the user instructions, storing the initialization plan, and recording the current plan's progress information.

[0015] The iterative update step involves the robot detecting and capturing current environmental change information, and then iteratively updating the plan based on the environmental change information, the plan progress information, the user instructions, the replanning prompts, and the replanning logic. This step is repeated until the task objective is achieved.

[0016] According to the reactive planning method of the present invention, the cause analysis logic includes: evaluating whether the environmental change has an impact on the current plan through a large language model, and analyzing the reasons for the plan adjustment that the current plan needs to be adjusted.

[0017] The derivation logic of the conclusion includes: deriving the plan adjustment measures based on the reasons for the plan adjustment given in the causal analysis;

[0018] The plan adjustment logic includes: deriving plan adjustment measures based on the conclusions, adjusting the current plan, and obtaining a new plan.

[0019] According to the reactive programming method of the present invention, the causal analysis logic further includes:

[0020] The large language model checks whether the latest plan obtained from the previous adjustment can be used as the final new plan. If not, it finds out the reason and identifies which feedback information has not been considered.

[0021] The plan adjustment logic further includes:

[0022] If all types of feedback have been fully considered, then this revised new plan will be output as the final plan.

[0023] According to the reactive planning method of the present invention, the user instructions are in natural language form, and the initialization plan is in text format;

[0024] The progress information of the plan includes: information on the completed and uncompleted parts of the current plan.

[0025] According to the reactive planning method of the present invention, the step of detecting and capturing environmental change information by a robot includes:

[0026] During the robot's interaction with the environment phase, the robot detects and captures environmental state information and environmental change information, and converts the environmental state information and environmental change information into text data using natural language description.

[0027] According to the reactive programming method of the present invention, the step of iteratively updating the plan is followed by:

[0028] Extract the action sequence from the new plan and execute it.

[0029] According to the reactive planning method of the present invention, the problem statement describes the task to be accomplished by the large language model in indirect language, including how the task should be accomplished and what the following content of the replanning prompt is respectively;

[0030] The constraint statement describes all interactive objects in the environment, as well as the robot's executable operations on those objects, using programming-like declaration statements.

[0031] The case demonstrations explicitly list several cases and output the analysis process and the adjusted new plan in a predetermined output format.

[0032] Secondly, embodiments of the present invention provide a reactive planning system based on a large language model, characterized in that it includes:

[0033] The replanning logic design module is used to design the replanning logic, which includes multi-step logic and three-hop logic. The multi-step logic includes analyzing the impact of various factors on the completion of the task objective step by step and updating the current plan in a chain. The three-hop logic includes reasoning logic that uses cause analysis logic, conclusion deduction logic and plan adjustment logic in each step.

[0034] The replanning prompt design module is used to design replanning prompts, which include problem statements, constraint statements, and case demonstrations.

[0035] The plan initialization module is used to receive user commands, generate an initialization plan according to the user instructions, store the initialization plan, and record the current plan progress information;

[0036] The iterative update module is used to detect and capture current environmental change information through the robot, and perform iterative updates to the plan based on the environmental change information, the plan progress information, the user instructions, the replanning prompts, and the replanning logic. This step is repeated until the task objective is achieved.

[0037] Thirdly, the present invention also provides a storage medium, characterized in that it is used to store a computer program for executing the reactive programming method based on a large language model as described in any of the above claims.

[0038] Fourthly, the present invention also provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the reactive programming method based on a large language model as described above.

[0039] In this embodiment of the invention, a robot reactive planning method based on LLMs is proposed. First, a replanning logic is designed, including multi-step logic and three-hop logic. The multi-step logic involves analyzing the impact of various factors on the task objective step by step and updating the current plan in a chain. The three-hop logic includes reasoning logic employing cause analysis, conclusion derivation, and plan adjustment logic in each step. Next, planning prompts are designed, including problem statements, constraint statements, and case demonstrations. Then, user commands are received, an initialization plan is generated based on user instructions, the initialization plan is stored, and the current plan's progress information is recorded. Subsequently, the robot detects and captures current environmental change information, and iteratively updates the plan based on environmental change information, plan progress information, user instructions, replanning prompts, and replanning logic. This step is repeated until the task objective is achieved. Therefore, this invention solves the problem of reactive planning where robots dynamically adjust plans to adapt to various environmental changes and ensure task completion in a dynamic environment, achieving high accuracy in replanning and a high success rate in task objective completion at low cost. Furthermore, this invention has strong generalization ability to new scenarios and tasks. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the reactive programming method based on a large language model provided in Embodiment 1 of the present invention.

[0041] Figure 2 This is a flowchart illustrating the reactive planning method based on a large language model provided in Embodiment 2 of the present invention.

[0042] Figure 3 This is a schematic diagram of the structure of the reactive planning system based on a large language model provided in Embodiment 1 of the present invention;

[0043] Figure 4 This is a schematic diagram of the structure of the reactive planning system based on a large language model provided in Embodiment 2 of the present invention;

[0044] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0046] It should be noted that references to "an embodiment," "embodiment," "example embodiment," etc., in this specification refer to the described embodiment including specific features, structures, or characteristics, but not every embodiment must include these specific features, structures, or characteristics. Furthermore, such expressions do not refer to the same embodiment. Moreover, when describing specific features, structures, or characteristics in conjunction with embodiments, whether or not explicitly described, it is indicated that incorporating such features, structures, or characteristics into other embodiments is within the knowledge of those skilled in the art.

[0047] Furthermore, certain terms are used in the specification and subsequent claims to refer to specific components or parts. Those skilled in the art will understand that manufacturers may use different names or terms to refer to the same component or part. This specification and subsequent claims do not distinguish components or parts by differences in name, but rather by differences in function. The terms "comprising" and "including" used throughout the specification and subsequent claims are open-ended and should be interpreted as "including but not limited to." Additionally, the term "connection" here includes any direct and indirect electrical connection means. Indirect electrical connection means include connections made through other means.

[0048] The reactive planning method based on a large language model provided by the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0049] This invention belongs to the interdisciplinary field of artificial intelligence and robot task planning, and in particular relates to technologies such as LLMs inference based on few-shot context learning, semantic transformation of multimodal feedback information, and robot reactive planning logic that drive robots to modify their plans in real time in changing environments to adapt to environmental changes and ensure the completion of user commands.

[0050] This invention takes into account the significant positive impact of Large Language Models (LLMs) on robot task planning research, and therefore employs LLMs to solve reactive planning problems. Considering the limitations of existing methods, the inventors found a considerable challenge in researching "achieving flexible robot plan adjustments to cope with various environmental changes while ensuring successful task completion based on LLMs." The method must overcome the inherent tendency of LLMs to overthink, guiding it to comprehensively analyze the impact of various factors on task execution under strong constraints of robot configuration and environmental information, and deduce appropriate plan adjustment strategies. Furthermore, it must standardize the LLMs plan adjustment process, ultimately generating a new plan that is action-executable, environmentally adaptable, and task-complete.

[0051] The purpose of this invention is to enable robots to generate and dynamically adjust plans based on user commands in constantly changing daily environments, adapting to environmental changes and ensuring successful task completion. Existing script-based and learning-based methods rely on large amounts of finely predefined, high-quality datasets and carefully designed learning strategies to achieve this goal. Furthermore, LLM-based methods either generate fixed plans or overlook key factors during plan adjustments, leading to errors. Therefore, this invention proposes a novel LLM-based replanning framework (Text2Reaction). This method uses a "multi-step-three-hop" reasoning mechanism as its core for replanning. Designed replanning prompts guide LLMs to perform a chain-like analysis of the impact of all factors on plan execution, gradually adjusting the original plan to form a new one. To facilitate the iterative update process of the task plan, this method introduces a plan initialization phase. The initial plan for the task is generated solely based on user commands in natural language form during this phase, followed by dynamic updates during the robot's interaction with the environment. This invention achieves low-cost, high-accuracy, and highly generalizable reactive robot planning.

[0052] Figure 1 The diagram shown is a flowchart of a reactive programming method based on LLMs provided in Embodiment 1 of the present invention. The method includes the following steps:

[0053] Step S101: Design steps for replanning logic. Replanning logic includes multi-step logic and three-hop logic. Multi-step logic involves analyzing the impact of various factors on achieving the task objective step by step and updating the current plan in a chain. Three-hop logic includes reasoning logic that employs cause-effect analysis, conclusion derivation, and plan adjustment logic in each step.

[0054] Preferably, the cause analysis logic includes: assessing whether environmental changes have an impact on the current plan using LLMs, and analyzing the reasons for the need to adjust the current plan. More preferably, the cause analysis logic further includes: checking whether the plan obtained from the latest adjustment can serve as the final new plan using LLMs; if not, identifying the reasons and recognizing which feedback information has not yet been considered.

[0055] Preferably, the conclusion derivation logic includes: deriving the plan adjustment measures based on the reasons for plan adjustment given in the causal analysis.

[0056] Preferably, the plan adjustment logic includes: deriving plan adjustment measures based on the conclusions, adjusting the current plan, and obtaining a new plan. More preferably, the plan adjustment logic further includes: if all types of feedback information have been fully considered, then outputting this adjusted new plan as the final plan.

[0057] Step S102 involves redesigning the cue words. Redesigning the cue words includes the problem statement, constraint statement, and case demonstration.

[0058] Preferably, the problem statement describes the task that LLMs need to accomplish using indirect language, including how the task should be accomplished and what the following content of the replanning prompt is.

[0059] Preferably, the constraint statement describes all interactive objects in the environment, as well as the robot's executable operations on those objects, using programming-like declaration statements.

[0060] Preferably, the case demonstration explicitly lists several cases and outputs the analysis process and the adjusted new plan in a predetermined output format.

[0061] Step S103, Plan Initialization Step. Receive user commands, generate an initialization plan based on user instructions, store the initialization plan, and record the current plan's progress information.

[0062] Preferably, the user instructions are in natural language form, and the initialization plan is in text format. The plan progress information includes: information on the completed and uncompleted portions of the current plan.

[0063] Step S104, Iterative Update Step. The robot detects and captures current environmental change information. Based on the environmental change information, plan progress information, user instructions, replanning prompts, and replanning logic, the plan is iteratively updated. This step is repeated until the task objective is achieved.

[0064] Preferably, during the robot-environment interaction phase, the robot detects and captures environmental state information and environmental change information, and converts the environmental state information and environmental change information into text data using natural language description.

[0065] Preferably, the step of iteratively updating the plan includes:

[0066] Extract the action sequence from the new plan and execute it.

[0067] This invention provides a novel LLM-based reactive planning framework for robots. The performance verification of this reactive planning framework includes the following key aspects:

[0068] I. Representation of Replanning Forms. GPT-3.5, GPT-4.0, and other GPT (Generative Pre-trained Transformer) models are used as the core inference module for solving reactive planning problems. Since the data received and output by these LLMs is text-based, the replanning form is represented as a cyclical text data processing process: First, the robot perceives environmental changes and describes the environmental state, task objectives, and plan execution progress using text data; then, this is combined with prompt text used to guide the LLMs' replanning and input into the LLMs; the LLMs perform inference based on the received text information and generate text-based replanning results; the text generated by the LLMs is obtained and the new action sequence is parsed; the above process is repeated until the task objective is achieved.

[0069] II. Designing the Logic of Replanning. Based on the characteristic that LLMs excel at using chain reasoning to complete language processing tasks, a "multi-step-three-hop" interpretable reasoning mechanism was designed for the replanning process. This mechanism requires LLMs to first analyze the complex impact of various factors on task completion step-by-step in multiple steps, and update the current plan in a chain. Then, in each step, the reasoning logic of "cause analysis - conclusion deduction - plan adjustment" is used: analyzing the reasons why the plan needs further adjustment, providing specific measures for plan adjustment, and outputting the results of the adjusted plan.

[0070] III. Design of Replanning Cue Words. To enable LLMs to process the input text data as required and complete the replanning process based on "multi-step-three-hop", replanning cue words were constructed to guide LLMs, including three main parts: problem statement, constraint statement, and case demonstration. (1) Problem Statement: The task to be completed by the LLMs is described in indirect language, how to complete the task, and the following content of the cue words. (2) Constraint Statement: All interactive objects in the environment and the robot's executable operations on these objects are concisely described using programming-like declaration statements. (3) Case Demonstration: Several cases are explicitly listed so that LLMs can learn more intuitively how to replan and output the analysis process and adjusted plan in a formatted manner.

[0071] IV. Constructing a Reactive Planning Framework. The Text2Reaction reactive planning framework is constructed by integrating the previous three key points. It includes: an offline planner, an online planner, a memory storage system, and an environment representation system. When the framework receives a user command in natural language, the offline planner first generates an initial plan based on the user command and stores it in the memory storage system to record the plan's execution progress. Then, the reactive planning framework enters the iterative update phase of the plan. In each iteration, the environment representation system detects and captures environmental changes, then converts the environmental state and change information into text data. Simultaneously, the memory storage system converts the task completion progress into text data. This text data, combined with replanning prompts, is input into the online planner, which parses the new plan from the textual output generated by the online planner for execution.

[0072] The beneficial effects of this invention include:

[0073] (1) Achieving high accuracy in replanning and high success rate in task completion at low cost. This invention proposes a novel reactive planning method for robots based on LLMs, which solves the problem of reactive planning in which robots dynamically adjust their plans to adapt to various environmental changes and ensure task completion in a dynamic environment. It achieves high accuracy in replanning and high success rate in task completion at low cost. The low cost is due to three aspects: 1. The replanning prompts contain concise problem and constraint descriptions, reducing the amount of manually predefined information by at least 60% compared to script-based methods. 2. The case demonstrations in the replanning prompts provide LLMs with a small amount of case data for learning, which is at least 99% less than the amount of training data required by learning-based methods. 3. Once the replanning prompts are designed, they can be directly used to guide LLMs in replanning, without the need for carefully designed and controlled training strategies required by learning-based methods. Comparative experiments on robot simulation platforms and real-world scenarios show that the new plans generated by this invention after replanning using different types of LLMs have an accuracy rate of up to 90%, and a success rate of at least 85% in completing the task objectives corresponding to user commands. Its overall performance is better than that of learning-based methods, script-based methods, and other LLM-based planning methods compared with this invention.

[0074] (2) Strong generalization ability to new scenarios and tasks. This invention has good generalization ability to new scenarios and tasks, which is not found in script-based methods and learning-based methods. ① New scenario: refers to the environment settings and robot operation settings that LLMs have not seen during few-sample context learning, including new object types such as bananas, new object attributes such as temperature, and new robot operation types such as slicing. ② New task: refers to the task types and environmental change types that LLMs have not seen during learning. Unseen task types are the task types corresponding to user commands that have not appeared in the replanning prompts, such as heating objects, cleaning objects, etc. Unseen environmental change types are the object changes that have not appeared in the replanning prompts, including changes in unseen objects and changes in unseen attributes of seen objects. Experiments on robot simulation platforms and real-world scenario demonstrations have demonstrated that this invention achieved a task success rate of 86% in the application of new scenarios and tasks, showing strong generalization ability.

[0075] This invention employs a few-shot context learning method to teach LLMs how to solve reactive programming problems. All algorithms and learning processes are implemented in Python. In applications with varying complexity, the number of tokens (standard text length) required for replanning prompts ranges from 1000 to 3000, and the number of case studies ranges from 5 to 30. Replanning is performed using GPT-series generative pre-trained language models such as gpt-3.5-turbo-16k, employing a non-streaming data transmission mode, with the parameter `temperature` controlling the diversity of generated text set to 0.5. To verify the performance of this invention, a test task set was constructed based on the AI2THOR simulation platform and the ALFRED (Action Learning From Realisic Environments and Directives) benchmark. Two real-world scenarios were constructed: one deploying a desktop UR (Ultra-Robo) robotic arm to complete the operation task, and the other completing navigation and operation tasks on a mobile device equipped with a JAVA robotic arm. Both robotic arms are controlled by a ROS (Robot Operating System) system. Experiments demonstrate that this invention has better performance than other methods.

[0076] Figure 2 The diagram shown is a flowchart of a reactive planning method based on LLMs provided in Embodiment 2 of the present invention. The method includes the following steps:

[0077] Step S201: Design the replanning logic. The replanning logic includes multi-step logic and three-hop logic. The multi-step logic involves analyzing the impact of various factors on achieving the task objective step by step and updating the current plan in a chain. The three-hop logic includes reasoning logic that uses cause analysis logic, conclusion derivation logic, and plan adjustment logic in each step.

[0078] Preferably, the cause analysis logic includes: assessing whether environmental changes have an impact on the current plan using LLMs, and analyzing the reasons for the need to adjust the current plan. More preferably, the cause analysis logic further includes: checking whether the plan obtained from the latest adjustment can serve as the final new plan using LLMs; if not, identifying the reasons and recognizing which feedback information has not yet been considered.

[0079] Preferably, the conclusion derivation logic includes: deriving the plan adjustment measures based on the reasons for plan adjustment given in the causal analysis.

[0080] Preferably, the plan adjustment logic includes: deriving plan adjustment measures based on the conclusions, adjusting the current plan, and obtaining a new plan. More preferably, the plan adjustment logic further includes: if all types of feedback information have been fully considered, then outputting this adjusted new plan as the final plan.

[0081] Step S202: Design replanning prompts. Replanning prompts include problem statements, constraint statements, and case demonstrations.

[0082] Preferably, the problem statement describes the task that LLMs need to accomplish using indirect language, including how the task should be accomplished and what the following content of the replanning prompt is.

[0083] Preferably, the constraint statement describes all interactive objects in the environment, as well as the robot's executable operations on those objects, using programming-like declaration statements.

[0084] Preferably, the case demonstration explicitly lists several cases and outputs the analysis process and the adjusted new plan in a predetermined output format.

[0085] Step S203: In the plan initialization phase, receive user commands, generate an initialization plan according to user instructions, and store the initialization plan.

[0086] Preferably, the user instructions are in natural language form, and the initialization plan is in text format.

[0087] Step S204: In the plan initialization phase, record the current plan progress information.

[0088] The progress information of the plan includes: information on the completed and uncompleted parts of the current plan.

[0089] Step S205: In the iterative update phase, the robot detects and captures current environmental change information.

[0090] Preferably, during the robot-environment interaction phase, the robot detects and captures environmental state information and environmental change information, and converts the environmental state information and environmental change information into text data using natural language description.

[0091] Step S206: In the iterative update phase, the plan is iteratively updated based on environmental change information, plan progress information, user instructions, replanning prompts, and replanning logic.

[0092] Step S207: In the iterative update phase, extract the action sequence from the new plan and execute it.

[0093] The above steps S205 to 207 are executed repeatedly until the task objective is achieved.

[0094] In another specific embodiment, the present invention implements a novel LLM-based reactive planning method for robots, comprising:

[0095] Step 1: Representing the Form of Replanning. Generative pre-trained language models such as GPT-3.5 and GPT-4.0 are used as the core inference module for solving reactive planning problems. Since the data received and output by these LLMs is text-based, the replanning form is represented as a cyclical text data processing process: First, the robot perceives environmental changes and describes multi-source information such as environmental state, task objectives, and plan execution progress using text data; then, this is combined with prompt text used to guide the LLMs' replanning and input into the LLMs; the LLMs perform inference based on the received text information and generate a text-based replanning result; the text generated by the LLMs is obtained and the new action sequence is parsed; the above process is repeated until the task objective is achieved.

[0096] Step 2: Design the replanning logic. Based on the characteristic that LLMs excel at using chained reasoning to complete language processing tasks, a "multi-step-three-hop" interpretable reasoning mechanism was designed as the core of the replanning process, such as... Figure 1 The mechanism first requires LLMs to analyze the complex impact of various factors on task completion in multiple steps and update the current plan in a chain. Specifically, this includes the following basic principles: In steps 1 and 2, assess whether environmental changes affect the overall task and the current plan; in step 3, determine the actions required for the new plan based on environmental changes; in subsequent steps, deeply analyze the impact of various combinations of all types of feedback on the task and iteratively adjust the action sequence of the previous step; if obvious errors are found in the original plan, they should be corrected immediately. Secondly, the "multi-step-three-hop" reasoning mechanism requires LLMs to use procedural reasoning at each step. This includes causal analysis in the first two hops and action sequence adjustment in the third hop. The first hop is causal analysis: LLMs check whether the plan obtained from the latest adjustment in the previous step can serve as the final new plan; if not, identify the reasons and identify which feedback information has not yet been considered. The second hop is conclusion derivation: LLMs deduce the measures for plan adjustment based on the reasons given in the first hop. The third jump is plan adjustment: Based on the conclusions inferred from the second jump, LLMs adjust the plan obtained in the previous step by adding or deleting actions to obtain a new plan. If all types of feedback information have been fully considered, this adjusted plan is output as the final plan.

[0097] Step 3: Design Replanning Cue Words. To enable LLMs to process the input text data as required and complete the replanning process based on a "multi-step, three-hop" approach, replanning cue words were constructed to guide LLMs. These cue words consist of three main parts: problem statement, constraint statement, and case demonstration. Figure 2 As shown. The replanning prompts convey the principles of replanning to LLMs through a summary of the problem statement and constraint statement, along with detailed descriptions of case demonstrations. The problem statement concisely conveys three key points: First, LLMs' task is to learn how to handle reactive planning; second, when dealing with different replanning problems, multi-step thinking is required, and in each step, LLMs need to reason based on the three-hop principle of cause analysis, conclusion derivation, and plan adjustment; finally, LLMs are obligated to generate replanning content according to the specified format. The constraint statement uses two types of formula declarations to demonstrate all interactive objects in the environment and the operations that the robot can perform. Object declarations (Z = {A, B, ...}) first group objects with similar affiliations, such as A and B, that support the same robot operations into a group, and then assign a new name Z to this group. Operation declarations (Z(O1, O2, ...)) associate the object group Z with a set of shared operations, such as O1. For example, the object declaration Fruit = {Apple, Pear} groups apples and pears into the Fruit group because both apples and pears are fruits. The `Fruit(Pick, Slice)` method indicates that each object in the `Fruit` class can be picked up and sliced ​​by the robot. The case demonstration clarifies the input and output details of the replanning process. After detecting an environmental change, the online planner receives four types of feedback, ordered by user commands, sequence of executed actions, semantic description of the environmental change, and sequence of incomplete actions. Furthermore, the online planner must adhere to the reasoning principles and output format demonstrated in the demonstration during replanning.

[0098] Step 4: Constructing the reactive programming framework. The reactive programming framework is constructed by combining the previous three steps.

[0099] —Text2Reaction, framework structure as follows Figure 2As shown, the system comprises three main modules: an offline planner, an online planner, and a memory module, as well as an environment representation module (Env). The offline planner uses LLMs as its planner, mapping received natural language user commands into a textualized task plan under the guidance of prompts, and then extracting the action sequence from the task plan text. The online planner uses LLMs as its replanner, employing a "multi-step-three-hop" reasoning mechanism under the guidance of replanning prompts designed in this invention to analyze the impact of four types of multi-source feedback information (including: user commands, completed parts of the plan, incomplete parts of the plan, and language descriptions of environmental changes) on the current plan, progressively updating the plan and generating a textualized reasoning process and results, then extracting a new plan from the generated content. The memory module stores the latest plan and provides the online planner with the completed and incomplete parts of the current plan during the replanning process. The environmental characterizer acquires the robot's visual observations every 0.5 seconds as a frame and maps them into symbolic representations of the environmental state through a neural network (e.g., "door - open" is a symbol). Then, it compares the latest symbolic representation with the symbolic representation of the previous frame. Symbols with different information indicate environmental changes (e.g., "door - open" in the previous frame and "door - closed" in the next frame, indicating that the door's open / closed state has changed). Finally, the environmental changes are described in natural language and provided to the online planner.

[0100] The reactive planning framework of this invention consists of two stages from receiving user commands to completing the task objective: In the first stage, the offline planner receives user commands in natural language and generates an initial plan, which is then stored in memory. In the second stage, the environmental representative continuously acquires the environmental state and, upon detecting environmental changes, describes the changes in natural language and transmits them to the online planner. Simultaneously, the memory describes the completed and incomplete parts of the current plan in natural language and transmits them to the online planner. After receiving these three feedback messages, the user command, and the replanning prompt, the online planner performs reasoning and generates a new plan. The new plan overwrites the original plan in memory and is executed. This process is repeated in the second stage until the task objective is achieved.

[0101] The reactive planning method based on a large language model provided in this invention first designs a replanning logic, including multi-step logic and three-hop logic. The multi-step logic involves analyzing the impact of various factors on the completion of the task objective step by step and updating the current plan in a chain. The three-hop logic includes reasoning logic that employs cause analysis logic, conclusion derivation logic, and plan adjustment logic in each step. Next, planning prompts are designed, including problem statements, constraint statements, and case demonstrations. Then, user commands are received, an initialization plan is generated based on user instructions, the initialization plan is stored, and the current plan's progress information is recorded. Subsequently, the robot detects and captures current environmental change information, and iteratively updates the plan based on environmental change information, plan progress information, user instructions, replanning prompts, and replanning logic. This step is repeated until the task objective is achieved. Therefore, this invention solves the problem of reactive planning where robots dynamically adjust plans in changing environments to adapt to various environmental changes and ensure task completion, achieving high accuracy in replanning and a high success rate in task objective completion at low cost. Furthermore, this invention has strong generalization ability for new scenarios and tasks.

[0102] It should be noted that the reactive programming method based on a large language model provided in this embodiment of the invention can be executed by an electronic device, a reactive programming system based on a large language model, or a control module within the reactive programming system based on a large language model for executing the reactive programming method based on a large language model. This embodiment of the invention uses the execution of the reactive programming method based on a large language model by a reactive programming system based on a large language model as an example to illustrate the reactive programming system based on a large language model provided in this embodiment of the invention.

[0103] Figure 3 This is a schematic diagram of the structure of a reactive planning system based on a large language model provided in Embodiment 1 of the present invention. The reactive planning system 100 based on a large language model includes at least a replanning logic design module 10, a replanning prompt word design module 20, a plan initialization module 30, and an iterative update module 40, wherein:

[0104] The design module 10 for the replanning logic is used to design the replanning logic, which includes multi-step logic and three-hop logic. The multi-step logic involves analyzing the impact of various factors on the completion of the task objective step by step and updating the current plan in a chain. The three-hop logic includes reasoning logic that uses cause analysis logic, conclusion derivation logic, and plan adjustment logic in each step.

[0105] Preferably, the cause analysis logic includes: assessing whether environmental changes affect the current plan through a large language model, and analyzing the reasons for the need to adjust the current plan. More preferably, the cause analysis logic further includes: the large language model checking whether the plan obtained from the latest adjustment can serve as the final new plan; if not, identifying the reasons and recognizing which feedback information has not yet been considered.

[0106] Preferably, the conclusion derivation logic includes: deriving the plan adjustment measures based on the reasons for plan adjustment given in the causal analysis.

[0107] Preferably, the plan adjustment logic includes: deriving plan adjustment measures based on the conclusions, adjusting the current plan, and obtaining a new plan. More preferably, the plan adjustment logic further includes: if all types of feedback information have been fully considered, then outputting this adjusted new plan as the final plan.

[0108] The replanning prompt design module 20 is used to design replanning prompts, which include problem statements, constraint statements, and case demonstrations.

[0109] Preferably, the problem statement describes the task that the large language model needs to accomplish using indirect language, including how the task should be accomplished and what the following content of the replanning prompt words should be.

[0110] Preferably, the constraint statement describes all interactive objects in the environment, as well as the robot's executable operations on those objects, using programming-like declaration statements.

[0111] Preferably, the case demonstration explicitly lists several cases and outputs the analysis process and the adjusted new plan in a predetermined output format.

[0112] The plan initialization module 30 is used to receive user commands, generate an initialization plan according to user instructions, store the initialization plan, and record the current plan progress information.

[0113] Preferably, the user instructions are in natural language form, and the initialization plan is in text format. The plan progress information includes: information on the completed and uncompleted portions of the current plan.

[0114] The iterative update module 40 is used to detect and capture current environmental change information through the robot, and to iteratively update the plan based on the environmental change information, plan progress information, user instructions, replanning prompts and replanning logic. This step is repeated until the task objective is achieved.

[0115] Preferably, during the robot-environment interaction phase, the robot detects and captures environmental state information and environmental change information, and converts the environmental state information and environmental change information into text data using natural language description.

[0116] Preferably, the iterative update module 40 is also used to extract and execute the action sequence from the new plan after iteratively updating the plan.

[0117] Figure 4 This is a schematic diagram of the structure of a reactive planning system based on a large language model provided in Embodiment 2 of the present invention. The reactive planning system 100 based on a large language model includes at least a replanning logic design module 10, a replanning prompt word design module 20, a plan initialization module 30, and an iterative update module 40, wherein:

[0118] The design module 10 for the replanning logic is used to design the replanning logic, which includes multi-step logic and three-hop logic. The multi-step logic involves analyzing the impact of various factors on the completion of the task objective step by step and updating the current plan in a chain. The three-hop logic includes reasoning logic that uses cause analysis logic, conclusion derivation logic, and plan adjustment logic in each step.

[0119] Preferably, the cause analysis logic includes: assessing whether environmental changes affect the current plan through a large language model, and analyzing the reasons for the need to adjust the current plan. More preferably, the cause analysis logic further includes: the large language model checking whether the plan obtained from the latest adjustment can serve as the final new plan; if not, identifying the reasons and recognizing which feedback information has not yet been considered.

[0120] Preferably, the conclusion derivation logic includes: deriving the plan adjustment measures based on the reasons for plan adjustment given in the causal analysis.

[0121] Preferably, the plan adjustment logic includes: deriving plan adjustment measures based on the conclusions, adjusting the current plan, and obtaining a new plan. More preferably, the plan adjustment logic further includes: if all types of feedback information have been fully considered, then outputting this adjusted new plan as the final plan.

[0122] The replanning prompt design module 20 is used to design replanning prompts, which include problem statements, constraint statements, and case demonstrations.

[0123] Preferably, the problem statement describes the task that the large language model needs to accomplish using indirect language, including how the task should be accomplished and what the following content of the replanning prompt words should be.

[0124] Preferably, the constraint statement describes all interactive objects in the environment, as well as the robot's executable operations on those objects, using programming-like declaration statements.

[0125] Preferably, the case demonstration explicitly lists several cases and outputs the analysis process and the adjusted new plan in a predetermined output format.

[0126] The plan initialization module 30 further includes: an initialization plan submodule 31 and a progress recording submodule 32; the initialization plan submodule 31 is used to receive user commands, generate an initialization plan according to user instructions, and store the initialization plan. Preferably, the user instructions are in natural language form, and the initialization plan is in text format. The progress recording submodule 32 is used to record the plan progress information of the current plan. Preferably, the plan progress information includes: information on the completed part and information on the uncompleted part of the current plan.

[0127] The iterative update module 40 further includes: an environment detection submodule 41, an iterative update submodule 42, and a plan execution submodule 43. The environment detection submodule 41 is used to detect and capture current environmental change information through the robot. The iterative update submodule 42 is used to iteratively update the plan based on environmental change information, plan progress information, user instructions, replanning prompts, and replanning logic. The plan execution submodule 43 is used to extract the action sequence from the new plan and execute it after the iterative update. The environment detection submodule 41, iterative update submodule 42, and plan execution submodule 43 cycle until the task objective is achieved.

[0128] The reactive programming system based on a large language model provided in this embodiment of the invention can realize all the processes implemented in the embodiment of the reactive programming method based on a large language model. To avoid repetition, these will not be described again here.

[0129] The reactive planning system based on a large language model provided in this invention includes at least a replanning logic design module 10, a replanning prompt word design module 20, a plan initialization module 30, and an iterative update module 40. The replanning logic design module 10 designs the replanning logic, including multi-step logic and three-hop logic. The multi-step logic involves analyzing the impact of various factors on achieving the task objective step by step and updating the current plan in a chain. The three-hop logic includes reasoning logic employing cause analysis logic, conclusion derivation logic, and plan adjustment logic in each step. The replanning prompt word design module 20 designs planning prompt words, including problem statements, constraint statements, and case demonstrations. The plan initialization module 30 receives user commands, generates an initialization plan based on user instructions, stores the initialization plan, and records the current plan's progress information. The iterative update module 40 detects and captures current environmental change information through the robot, and iteratively updates the plan based on environmental change information, plan progress information, user instructions, replanning prompt words, and replanning logic, repeating this process until the task objective is achieved. Therefore, this invention solves the reactive planning problem of robots dynamically adjusting their plans in changing environments to adapt to various environmental changes and ensure task completion, achieving high accuracy in replanning and a high success rate in achieving task objectives at low cost. Secondly, this invention has strong generalization ability to new scenarios and tasks.

[0130] The present invention also provides a storage medium for storing, for example, Figures 1-2 The computer program is any of the reactive programming methods based on a large language model. For example, computer program instructions, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the present invention through the operation of the computer, achieving the same technical effect. To avoid repetition, these will not be elaborated further here. The program instructions that invoke the methods of the present invention may be stored in a fixed or removable storage medium, and / or transmitted via data streams in broadcast or other signal carrying media, and / or stored in the storage medium of a computer device operating according to the program instructions.

[0131] According to one embodiment of the present invention, the present invention also provides such a Figure 5The illustrated electronic device 400 may optionally include a storage medium 200 for storing computer programs and a processor 300 for executing the computer programs. When the computer program is executed by the processor 300, it implements any of the aforementioned reactive programming methods based on large language models, triggering the electronic device 400 to execute methods and / or technical solutions based on the foregoing embodiments, achieving the same technical effect. To avoid repetition, these will not be elaborated further here. It should be noted that the electronic devices in this embodiment include mobile electronic devices and non-mobile electronic devices. For example, mobile electronic devices may be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, super mobile personal computers, netbooks, or personal digital assistants, etc., while non-mobile electronic devices may be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This embodiment does not specifically limit the scope of the invention.

[0132] It should be noted that the present invention can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present invention (including associated data structures) can be stored in a computer-readable recording medium, such as RAM memory, a magnetic or optical drive, a floppy disk, or similar devices. Furthermore, some steps or functions of the present invention can be implemented in hardware, for example, as circuitry that works with a processor to perform the various steps or functions.

[0133] This invention can be implemented on a computer as a computer-based method, or in dedicated hardware, or a combination of both. Executable code or portions thereof for the method according to the invention can be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Optionally, the computer program product includes non-transitory program code components stored on a computer-readable medium so as to execute the method according to the invention when the program product is executed on a computer.

[0134] In an optional embodiment, the computer program includes computer program code components adapted to perform all the steps of the method according to the invention when the computer program is run on a computer. Optionally, the computer program is embodied on a computer-readable medium.

[0135] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0136] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A reactive programming method based on a large language model, characterized in that, include: The design steps of the replanning logic include multi-step logic and three-hop logic. The multi-step logic includes analyzing the impact of various factors on the completion of the task objective step by step and updating the current plan in a chain. The three-hop logic includes reasoning logic that uses cause analysis logic, conclusion derivation logic and plan adjustment logic in each step. The design steps for redesigning prompts include problem statements, constraint statements, and case demonstrations; The initialization plan includes receiving user instructions, generating an initialization plan based on the user instructions, storing the initialization plan, and recording the current plan's progress information. The iterative update step involves the robot detecting and capturing current environmental change information, and then iteratively updating the plan based on the environmental change information, the plan progress information, the user instructions, the replanning prompts, and the replanning logic. This step is repeated until the task objective is achieved.

2. The reactive programming method according to claim 1, characterized in that, The reason analysis logic includes: using a large language model to assess whether the environmental changes have an impact on the current plan, and analyzing the reasons for the plan adjustment that requires further adjustment to the current plan; The derivation logic of the conclusion includes: deriving the plan adjustment measures based on the reasons for the plan adjustment given in the causal analysis; The plan adjustment logic includes: deriving plan adjustment measures based on the conclusions, adjusting the current plan, and obtaining a new plan.

3. The reactive programming method according to claim 2, characterized in that, The causal analysis logic further includes: The large language model checks whether the latest plan obtained from the previous adjustment can be used as the final new plan. If not, it finds out the reason and identifies which feedback information has not been considered. The plan adjustment logic further includes: If all types of feedback have been fully considered, then this revised new plan will be output as the final plan.

4. The reactive programming method according to claim 2, characterized in that, The user instructions are in natural language form, and the initialization plan is in text format; The progress information of the plan includes: information on the completed and uncompleted parts of the current plan.

5. The reactive programming method according to claim 1, characterized in that, The steps of detecting and capturing environmental change information using a robot include: During the robot's interaction with the environment phase, the robot detects and captures environmental state information and environmental change information, and converts the environmental state information and environmental change information into text data using natural language description.

6. The reactive programming method according to claim 2, characterized in that, The step of iteratively updating the plan is followed by: Extract the action sequence from the new plan and execute it.

7. The reactive programming method according to claim 2, characterized in that, The problem statement describes the task that the large language model is to accomplish in indirect language, including how to accomplish the task and what the following content of the replanning prompt is. The constraint statement describes all interactive objects in the environment, as well as the robot's executable operations on those objects, using programming-like declaration statements. The case demonstrations explicitly list several cases and output the analysis process and the adjusted new plan in a predetermined output format.

8. A reactive programming system based on a large language model constructed according to the method described in any one of claims 1 to 7, characterized in that, include: The replanning logic design module is used to design the replanning logic, which includes multi-step logic and three-hop logic. The multi-step logic includes analyzing the impact of various factors on the completion of the task objective step by step and updating the current plan in a chain. The three-hop logic includes reasoning logic that uses cause analysis logic, conclusion deduction logic and plan adjustment logic in each step. The replanning prompt design module is used to design replanning prompts, which include problem statements, constraint statements, and case demonstrations. The plan initialization module is used to receive user instructions, generate an initialization plan according to the user instructions, store the initialization plan, and record the current plan progress information; The iterative update module is used to detect and capture current environmental change information through the robot, and perform iterative updates to the plan based on the environmental change information, the plan progress information, the user instructions, the replanning prompts, and the replanning logic. This step is repeated until the task objective is achieved.

9. A storage medium, characterized in that, Used to store a computer program for executing a reactive programming method based on a large language model as described in any one of claims 1 to 7.

10. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the reactive programming method based on a large language model as described in any one of claims 1 to 7.

Images