Memory driving and closed-loop feedback-based tool planning method, storage medium and equipment

Abstract

The invention discloses a body planning method based on memory driving and closed-loop feedback, a storage medium and equipment, belongs to the field of body intelligence of a computer and robot interdiscipline, and particularly relates to the body planning method based on memory driving and closed-loop feedback, the storage medium and the equipment. The objective of the invention is to solve the problem that an intelligent agent cannot effectively accumulate experience to dynamically correct a planning strategy in long-range interaction due to lack of an updating mechanism of adaptive learning in the prior art, so that the adaptability is often insufficient when facing environment change or new knowledge. According to the invention, whenever the environment or the object position changes, the task progress record and the object position record in the memory mechanism can be updated in time, and the latest reference is provided for subsequent planning. The intelligent agent can effectively accumulate experience in long-range interaction to dynamically correct a planning strategy, the adaptability is high when facing environment changes or new knowledge, and the higher requirements of real application for expansibility and robustness are met.

Description

Technical Field

[0001] The present invention belongs to the field of embodied intelligence, which is the interdisciplinary field of computers and robotics, and specifically relates to an embodied planning method, storage medium, and device based on memory drive and closed-loop feedback. Background Art

[0002] With the rapid development of intelligent robotics and artificial intelligence technologies, an increasing number of application scenarios require autonomous agents to complete complex, multi-step tasks in real or near-real environments. Existing research on embodied planning methods generally faces the following shortcomings: First, most current benchmarks have limited requirements for the complexity of the planning phase. Long-term task planning involves a small number of interactive objects, limited actions, and simple interaction logic. Furthermore, most current embodied planning methods are based on fixed knowledge and cannot cope with diverse scenarios or adapt to dynamic environments in real time. This results in planning algorithms often lacking the desired scalability and robustness for long-term temporal decision-making.

[0003] Secondly, most methods are unable to achieve effective collaborative management when faced with multi-source knowledge (such as common sense, domain knowledge, empirical knowledge, physical rules, etc.). The single or fragmented use of this knowledge in the planning process will make the system inefficient in vertical domain tasks and may ignore the physical constraints that must be followed. In addition, in response to the reality that the status of objects and tools constantly changes during task execution, there is still a lack of sufficiently flexible planning mechanisms to update the model and make corresponding adjustments in a timely manner, resulting in action conflicts or repeated operations during the execution process, affecting the overall success rate and efficiency.

[0004] Furthermore, many existing embodied planning frameworks focus on a single component: perception, action execution, or task planning. They lack a complete closed loop from perception to planning and execution, making it difficult for the system to fully integrate perceptual feedback to adjust decision-making plans. Furthermore, due to the lack of an adaptive learning update mechanism, the agent cannot effectively accumulate experience over long-term interactions to dynamically adjust its planning strategy, resulting in insufficient adaptability in the face of environmental changes or new knowledge. Furthermore, the action sequences generated by the planning model can also encounter challenges such as difficulty in execution. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem that due to the lack of an adaptive learning update mechanism, intelligent agents are unable to effectively accumulate experience to dynamically correct planning strategies in long-term interactions, resulting in insufficient adaptability when facing environmental changes or new knowledge. An embodied planning method, storage medium, and device based on memory drive and closed-loop feedback are proposed.

[0006] The specific process of an embodied planning method based on memory drive and closed-loop feedback is as follows:

[0007] Step 1: Obtain the semantic information and location information of the 3D point cloud of objects in the environment;

[0008] Synchronize the semantic information and position information of the 3D point cloud of objects in the environment into the memory mechanism;

[0009] Step 2: The user gives the target instruction;

[0010] Step 3: Input the domain knowledge and target instructions in the memory mechanism into the large language model, and the large language model outputs the intent of the target instruction;

[0011] Step 4: The task decomposer decomposes the intention into a sequence of subtasks;

[0012] Step 5: Input the subtask sequence, the content of the memory mechanism, the set of supported actions in the current state, and related item and tool knowledge into the large language model. The large language model decomposes each subtask into a sequence of atomic action names.

[0013] Step 6: Pass the name sequence of the atomic actions decomposed in step 5 through the action API mapper to obtain the action API;

[0014] Step 7: The multimodal model is used to collect information about changes in the environment when the agent executes the action API in step 6. The multimodal model determines whether the agent successfully executes the action API in step 6.

[0015] If successful, the environment changes when the agent executes the action API in step six are collected through the multimodal model and stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the environment changes when the agent executes the action API in step six, and the output of the multimodal model is the environment changes when the agent executes the action API in step six, thereby obtaining a trained multimodal model;

[0016] If the action fails, the action is adjusted in a timely manner, and the cause of the failure and the environmental changes when the agent executes the action API in step six after the adjustment are stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six, thus obtaining a trained multimodal model;

[0017] Step 8. The trained multimodal model collects the environmental changes when the agent executes the action API in step 6 and passes it back to the memory mechanism.

[0018] Preferably, in step 1, semantic information and position information of a 3D point cloud of objects in the environment are obtained;

[0019] Synchronize the semantic information and position information of the 3D point cloud of objects in the environment into the memory mechanism;

[0020] The specific process is:

[0021] Get the environment RGB image;

[0022] Process the environment RGB image using the DepthAnything algorithm to obtain a depth map;

[0023] The environmental RGB image is processed by the SFM algorithm to obtain the internal and external parameters of the camera;

[0024] The SLAM algorithm is used to process the environmental RGB image, depth map, and internal and external parameters of the camera to obtain the 3D point cloud of each object in the environmental RGB image;

[0025] The 3D point cloud of each object is input into the multimodal model, and the multimodal model outputs the semantic information and position information of the 3D point cloud;

[0026] The multimodal model is GPT-4o, Janus or LlaVA.

[0027] Preferably, in step 3, the domain knowledge and target instructions in the memory mechanism are input into the large language model, and the large language model outputs the intention in the target instruction; the specific process is:

[0028] Large language models include ChatGPT, DeepSeek, Gemini, Claude, or diffusion model.

[0029] Preferably, in step 4, the task decomposer decomposes the intention into a subtask sequence; the specific process is:

[0030] The domain knowledge in the memory mechanism and the intent output by the three language models are input into the large language model, and the large language model decomposes the intent into a sequence of subtasks.

[0031] Preferably, in step 5, the content of the memory mechanism, the supported action set in the current state, the relevant item knowledge, and the tool knowledge are input into the large language model, and the large language model decomposes each subtask into a sequence of atomic action names; the specific process is:

[0032] There are four types of atomic action names:

[0033] 1) Navigation action;

[0034] 2) Perceived action;

[0035] 3) Operations related to tools;

[0036] 4) Interactive actions;

[0037] The memory mechanism includes the following contents: domain knowledge, common sense knowledge, experiential knowledge, current task progress, item location, and item interaction history;

[0038] The process of obtaining the supported action set in the current state is as follows:

[0039] The attributes of objects in the current environment, agent information, and common-sense action rules are input into the large language model, which then outputs the set of supported actions in the current state.

[0040] The action rules that conform to common sense are rules written in advance by humans;

[0041] The relevant item knowledge refers to items and materials used in the current environment scenario;

[0042] The tool knowledge refers to the tools used in the current scenario environment.

[0043] Preferably, in step 6, the name sequence of the atomic actions decomposed in step 5 is passed through an action API mapper to obtain the action API; the specific process is:

[0044] The action API mapper is a VLA model;

[0045] The VLA model is open VLA or pi0.

[0046] Preferably, in step seven, the environment changes when the agent executes the action API in step six are collected through a multimodal model, and the multimodal model determines whether the agent successfully executes the action API in step six;

[0047] If successful, the environment changes when the agent executes the action API in step six are collected through the multimodal model and stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the environment changes when the agent executes the action API in step six, and the output of the multimodal model is the environment changes when the agent executes the action API in step six, thereby obtaining a trained multimodal model;

[0048] If the action fails, the action is adjusted in a timely manner, and the cause of the failure and the environmental changes when the agent executes the action API in step six after the adjustment are stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six, thus obtaining a trained multimodal model;

[0049] The specific process is:

[0050] The multimodal model is used to collect information about changes in the environment when the agent executes the action API in step six. The multimodal model determines whether the agent successfully executes the action API in step six.

[0051] If successful, the environment changes when the agent executes the action API in step six are collected through the multimodal model and stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the environment changes when the agent executes the action API in step six, and the output of the multimodal model is the environment changes when the agent executes the action API in step six, thereby obtaining a trained multimodal model;

[0052] If the action fails, the action is adjusted in time, and the cause of the failure and the environmental changes when the agent executes the step 6 action API after the adjustment are stored in the memory mechanism; when the amount of information in the memory mechanism reaches the threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the agent executing the step 6 action API, and the output of the multimodal model is the environmental changes when the agent executes the step 6 action API, thus obtaining a trained multimodal model; the specific process is as follows:

[0053] When the multimodal model determines that the reason for the failure of the agent to execute the action API in step six is an operation failure, the operation in this step is undone and steps five, six, and seven are re-executed; and the reason for the failure and the environmental changes when the agent executes the action API in step six after adjustment are stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism, where the input of the multimodal model is the execution of the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six, thereby obtaining a trained multimodal model;

[0054] When the multimodal model determines that the reason for the failure of the intelligent agent to execute the action API in step six is a subtask division error, steps four, five, six, and seven are re-executed; and the reason for the failure and the environmental changes when the adjusted intelligent agent executes the action API in step six are stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism, the input of the multimodal model is the intelligent agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the intelligent agent executes the action API in step six, thereby obtaining a trained multimodal model.

[0055] Preferably, the environmental changes when the agent executes the action API in step 6 include:

[0056] The new status of the item, the location of the item, the current subtask completion progress, the name sequence of the atomic action corresponding to each subtask, the name sequence of the historical atomic action corresponding to each subtask, and whether there is any unexpected situation.

[0057] A storage medium stores at least one instruction, which is loaded and executed by a processor to implement an embodied planning method based on memory drive and closed-loop feedback.

[0058] An embodied planning device based on memory drive and closed-loop feedback, the device comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement an embodied planning method based on memory drive and closed-loop feedback.

[0059] The beneficial effects of the present invention are:

[0060] The first step of the present invention is to obtain the semantic information and position information of the 3D point cloud of the objects in the environment; synchronize the semantic information and position information of the 3D point cloud of the objects in the environment to the memory mechanism; the second step of the present invention is that the user gives the target instruction; the third step of the present invention is to input the domain knowledge and target instruction in the memory mechanism into the large language model, and the large language model outputs the intention in the target instruction; the fourth step of the present invention is to decompose the intention into a subtask sequence by the task decomposer; the fifth step of the present invention is to input the content contained in the memory mechanism, the supported action set in the current state, the relevant object knowledge, and the tool knowledge into the large language model, and the large language model decomposes each subtask into a name sequence of atomic actions; the sixth step of the present invention is to obtain the action API through the action API mapper through the name sequence of the atomic action decomposed in step five; the seventh step of the present invention is to collect the action API of the intelligent agent execution step six through the multimodal model The environment changes when the agent executes the action API in step six, and passes it back to the memory mechanism. When the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the environment changes when the agent executes the action API in step six, and the output of the multimodal model is the environment changes when the agent executes the action API in step six, so as to obtain a trained multimodal model. The trained multimodal model (existing multimodal models, such as GPT-4o, Janus, LlaVA and other existing models) collects the environment changes when the agent executes the action API in step six, and passes it back to the memory mechanism to form a closed loop. After each execution of step seven, the system will re-perceive and record the environment changes, and update the action sequence to be executed accordingly. The memory mechanism will record the successful experiences and failed lessons, and the next execution will refer to this new information to continuously improve the efficiency and accuracy in the multi-step long-range interaction scenario. Whenever the environment or the position of the object changes, the present invention can timely update the task progress record and the item location record in the memory mechanism, providing the latest reference for subsequent planning. The intelligent agent of the present invention can effectively accumulate experience in long-range interactions to dynamically modify planning strategies. It is highly adaptable when facing environmental changes or new knowledge, and meets the higher requirements of real applications for scalability and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0062] Specific implementation method 1: This implementation method is an embodied planning method based on memory drive and closed-loop feedback. The specific process is as follows:

[0063] Step 1: Obtain the semantic information and location information of the 3D point cloud of objects in the environment;

[0064] Synchronize the semantic information and position information of the 3D point cloud of objects in the environment into the memory mechanism;

[0065] Step 2: The user gives a target instruction in natural language (e.g., "Can you make me a lemon chicken breast?");

[0066] Step 3: Input the domain knowledge and target instructions in the memory mechanism into the large language model, and the large language model outputs the intent of the target instruction (such as "target: lemon chicken breast");

[0067] Step 4: The task decomposer decomposes the intention into a relatively clear subtask sequence;

[0068] Step 5: Input the subtask sequence, the content of the memory mechanism, the set of supported actions in the current state, related object knowledge, and tool knowledge into the large language model. The large language model decomposes each subtask into a sequence of atomic action names (grab the bottle, no matter how to grab it); for example, "open the refrigerator → take the chicken → cut the chicken → light the fire → put the chicken in the pot and add lemon juice";

[0069] Step 6: Pass the name sequence of the atomic action decomposed in step 5 through the action API mapper to obtain the action API (action instructions that the actuator can execute);

[0070] Step 7: Use a multimodal model (existing multimodal models, such as GPT-4o, Janus, LlaVA, etc.) to collect environmental changes when the agent executes the action API in step 6. The multimodal model determines whether the agent successfully executes the action API in step 6.

[0071] If successful, the environment changes when the agent executes the step six action API are collected through a multimodal model (existing multimodal models, such as GPT-4o, Janus, LlaVA, etc.) and stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the environment changes when the agent executes the step six action API, and the output of the multimodal model is the environment changes when the agent executes the step six action API, thereby obtaining a trained multimodal model;

[0072] If the action fails, the action is adjusted in a timely manner, and the cause of the failure and the environmental changes when the agent executes the step 6 action API after the adjustment are stored in the memory mechanism; this provides richer experience support for subsequent tasks and improves planning efficiency; when the amount of information in the memory mechanism reaches a threshold, the information in the memory mechanism is used to train a multimodal model. The input of the multimodal model is the agent executing the step 6 action API, and the output of the multimodal model is the environmental changes when the agent executes the step 6 action API, thus obtaining a trained multimodal model;

[0073] Step 8. The trained multimodal model (existing multimodal models, such as GPT-4o, Janus, LlaVA, etc.) collects the environmental changes when the agent executes the action API in step 6 and passes it back to the memory mechanism to form a closed loop.

[0074] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that: in step 1, the semantic information and position information of the 3D point cloud of the object in the environment are obtained;

[0075] Synchronize the semantic information and position information of the 3D point cloud of objects in the environment into the memory mechanism;

[0076] The specific process is:

[0077] Get the environment RGB image;

[0078] Process the environment RGB image using the DepthAnything algorithm to obtain a depth map;

[0079] The environmental RGB image is processed by the SFM algorithm to obtain the internal and external parameters of the camera;

[0080] The SLAM algorithm is used to process the environmental RGB image, depth map, and internal and external parameters of the camera to obtain the 3D point cloud of each object in the environmental RGB image;

[0081] The 3D point cloud of each object is input into a multimodal model (existing multimodal models, such as GPT-4o, Janus, LlaVA, etc.), and the multimodal model outputs the semantic information and location information of the 3D point cloud;

[0082] The multimodal model is GPT-4o, Janus or LlaVA.

[0083] Other steps and parameters are the same as those in the first embodiment.

[0084] Specific embodiment three: This embodiment differs from specific embodiments one or two in that: in step three, the domain knowledge and target instructions in the memory mechanism are input into the large language model, and the large language model outputs the intent in the target instruction (such as "target: lemon chicken breast"); the specific process is:

[0085] Domain knowledge includes the types of tools and materials used in the field, tool knowledge and action knowledge on how to use these tools or materials, and the basic principles or fixed logic that have been summarized in the field (such as recipes for various dishes in the kitchen, operating manuals for assembling different computer models, the specific structure and parts models of different cars when repairing cars, and the use of alkaline substances to clean oil stains and acidic substances to clean dirt in cleaning scenarios).

[0086] Large language models include ChatGPT, DeepSeek, Gemini, Claude, or diffusion model.

[0087] Other steps and parameters are the same as those in the first or second embodiment.

[0088] Specific embodiment 4: This embodiment differs from any one of specific embodiments 1 to 3 in that: in step 4, the task decomposer decomposes the intention into a relatively clear subtask sequence; the specific process is:

[0089] The domain knowledge in the memory mechanism and the intent output by the three language models of the steps are input into the large language model. The large language model decomposes the intent into a sequence of subtasks (such as "take out the chicken", "add seasoning", "start the fire and cook", etc.).

[0090] The other steps and parameters are the same as those in the first to third embodiments.

[0091] Specific embodiment 5: This embodiment differs from any one of specific embodiments 1 to 4 in that in step 5, the content of the memory mechanism, the set of supported actions in the current state, the relevant object knowledge, and the tool knowledge are input into the large language model. The large language model decomposes each subtask into a sequence of atomic action names (grab the bottle, no matter how to grab it); for example, "open the refrigerator → take the chicken → cut the chicken → light the fire → put the chicken in the pot and add lemon juice"; the specific process is as follows:

[0092] There are four types of atomic action names:

[0093] 1) Navigation actions (moving, turning, etc.);

[0094] 2) Perceiving actions (moving the camera up or down, etc.);

[0095] 3) Operations related to tools (cutting with a knife, turning on a blender, etc.);

[0096] 4) Interactive actions (pick up, place, etc.);

[0097] The memory mechanism includes the following contents: domain knowledge, common sense knowledge, experiential knowledge, current task progress, item location, and item interaction history;

[0098] The process of obtaining the supported action set in the current state is as follows:

[0099] The attributes of objects in the current environment, agent information, and common-sense action rules are input into the large language model, which then outputs the set of supported actions in the current state.

[0100] The action rules that conform to common sense are rules written in advance by humans, such as opening the refrigerator first before taking something out of it;

[0101] The relevant item knowledge refers to items and materials used in the current environment, such as eggs, soy sauce, and seasonings in a kitchen scenario.

[0102] The tool knowledge refers to the tools used in the current scene environment, such as kitchen utensils in a kitchen scene;

[0103] Item attributes in the current environment: This can be understood as the item's status information, location information, weight or number, temperature, etc. at this moment. Different types of items and tools have different attributes. For example, a liquid bottle has attributes such as capacity, remaining amount, and status (whether it is broken). Food ingredients have attributes such as weight, size, shelf life, heating status, and form (whole piece, cut into pieces, juiced).

[0104] Agent information refers to: the state and position of the robot, whether it is interacting with other objects, etc.

[0105] Common sense knowledge: Physical rules or common sense that are consistent with the current scenario, such as information about the robot itself, whether it has single or dual robotic arms, whether it is a wheeled or bipedal robot, and that you cannot directly handle heated objects, that plates will break if dropped, and that you need to open the microwave door before placing items in it.

[0106] Experiential knowledge: including successful experiences and lessons from failure;

[0107] Current task progress: records the relationship between completed tasks and the overall process, which is equivalent to a to-do list, letting the robot know what stage the task has reached;

[0108] Object interaction history: Interactions may change the state or location of an object. Recording this information prevents repeated interactions with the same object. If the task requires the robot to interact with multiple objects, it is necessary to record the object interaction history, including which objects have been interacted with and which have not.

[0109] The other steps and parameters are the same as those in the first to fourth embodiments.

[0110] Specific embodiment 6: This embodiment differs from any one of specific embodiments 1 to 5 in that: in step 6, the name sequence of the atomic action decomposed in step 5 is passed through the action API mapper to obtain the action API (action instructions that the execution mechanism can execute); the specific process is as follows:

[0111] The action API mapper is a VLA model;

[0112] The VLA model is open VLA or pi0.

[0113] The action API mapper maps the name sequence of atomic actions, such as pick up and put down, into action instructions that the robot can execute;

[0114] The other steps and parameters are the same as those in the first to fifth embodiments.

[0115] Specific embodiment seven: This embodiment differs from any one of specific embodiments one to six in that: in step seven, a multimodal model (existing multimodal models, such as GPT-4o, Janus, LlaVA, etc.) is used to collect environmental changes when the agent executes the action API in step six, and the multimodal model determines whether the agent successfully executes the action API in step six;

[0116] If successful, the environment changes when the agent executes the step six action API are collected through a multimodal model (existing multimodal models, such as GPT-4o, Janus, LlaVA, etc.) and stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the environment changes when the agent executes the step six action API, and the output of the multimodal model is the environment changes when the agent executes the step six action API, thereby obtaining a trained multimodal model;

[0117] If the action fails, the action is adjusted in a timely manner, and the cause of the failure and the environmental changes when the agent executes the step 6 action API after the adjustment are stored in the memory mechanism; this provides richer experience support for subsequent tasks and improves planning efficiency; when the amount of information in the memory mechanism reaches a threshold, the information in the memory mechanism is used to train a multimodal model. The input of the multimodal model is the agent executing the step 6 action API, and the output of the multimodal model is the environmental changes when the agent executes the step 6 action API, thus obtaining a trained multimodal model;

[0118] The specific process is:

[0119] A multimodal model (existing multimodal models, such as GPT-4o, Janus, LlaVA, etc.) is used to collect environmental changes when the agent executes the action API in step six. The multimodal model determines whether the agent successfully executes the action API in step six.

[0120] If successful, the environment changes when the agent executes the step six action API are collected through a multimodal model (existing multimodal models, such as GPT-4o, Janus, LlaVA, etc.) and stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the environment changes when the agent executes the step six action API, and the output of the multimodal model is the environment changes when the agent executes the step six action API, thereby obtaining a trained multimodal model;

[0121] If the action fails, the action is adjusted in a timely manner, and the cause of the failure and the environmental changes when the agent executes the step 6 action API after the adjustment are stored in the memory mechanism; this provides richer experience support for subsequent tasks and improves planning efficiency; when the amount of information in the memory mechanism reaches a threshold, the information in the memory mechanism is used to train a multimodal model. The input of the multimodal model is the agent executing the step 6 action API, and the output of the multimodal model is the environmental changes when the agent executes the step 6 action API, thus obtaining a trained multimodal model; the specific process is as follows:

[0122] When the multimodal model determines that the reason why the agent failed to execute the action API in step six is an operation failure, the backtracking mechanism is used to undo the current operation (for example, the corresponding undo operation for moving forward is moving backward, the undo operation for picking up is putting down, and so on) and re-execute steps five, six, and seven; and the reason for the failure and the environmental changes when the agent executes the action API in step six after adjustment are stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism, and the input of the multimodal model is the agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six, thereby obtaining a trained multimodal model;

[0123] When the multimodal model determines that the reason for the failure of the intelligent agent to execute the action API in step six is a subtask division error, steps four, five, six, and seven are re-executed; and the reason for the failure and the environmental changes when the adjusted intelligent agent executes the action API in step six are stored in the memory mechanism; when the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism, the input of the multimodal model is the intelligent agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the intelligent agent executes the action API in step six, thereby obtaining a trained multimodal model.

[0124] The other steps and parameters are the same as those in the first to sixth embodiments.

[0125] Specific embodiment eight: This embodiment differs from any one of specific embodiments one to seven in that the environment changes when the agent executes the action API in step six include:

[0126] The new status of the item, the location of the item, the completion progress of the current subtask (the name sequence of the atomic action entering the subtask), the name sequence of the atomic action corresponding to each subtask, the name sequence of the historical atomic action corresponding to each subtask, and whether any unexpected situations occurred.

[0127] Items: tools, materials, and all other objects that can be interacted with in the environment. Items include food ingredients, containers, kitchen utensils, etc.

[0128] New state of an object: In this patent, state is a term used to describe objects and operators. For example, a vinegar bottle being intact and broken would be considered two states. Picking up the vinegar and putting it down would also be considered two states. The interaction between the operator and the object changes the object's state. Objects include food, containers, tools, and more. State refers to the properties of these objects, which influence the decision about the next action.

[0129] The other steps and parameters are the same as those in the first to seventh embodiments.

[0130] Specific embodiment nine: This embodiment is a storage medium, which stores at least one instruction. The at least one instruction is loaded and executed by a processor to implement the embodied planning method based on memory drive and closed-loop feedback.

[0131] It should be understood that any method described herein may be provided as a computer program product, software, or computerized method, which may include a non-transitory machine-readable medium having instructions stored thereon, the instructions being used to program a computer system or other electronic device. The storage medium may include, but is not limited to, magnetic storage media, optical storage media; magneto-optical storage media including: read-only memory (ROM), random access memory (RAM), erasable programmable memory (e.g., EPROM and EEPROM), and flash memory layers; or other types of media suitable for storing electronic instructions.

[0132] Specific embodiment 10: This embodiment is an embodied planning device based on memory drive and closed-loop feedback, the device including a processor and a memory. It should be understood that, including any device including a processor and a memory described in the present invention, the device may also include other units and modules that perform display, interaction, processing, control, and other functions through signals or instructions;

[0133] At least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the embodied planning method based on memory drive and closed-loop feedback.

[0134] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. An embodied planning method based on memory drive and closed-loop feedback, characterized by: The specific process of the method is: Step 1: Obtain the semantic information and location information of the 3D point cloud of objects in the environment; Synchronize the semantic information and position information of the 3D point cloud of objects in the environment into the memory mechanism; Step 2: The user gives the target instruction; Step 3: Input the domain knowledge and target instructions in the memory mechanism into the large language model, and the large language model outputs the intent of the target instruction; Step 4: The task decomposer decomposes the intention into a sequence of subtasks; Step 5: Input the subtask sequence, the content of the memory mechanism, the set of supported actions in the current state, and related item and tool knowledge into the large language model. The large language model decomposes each subtask into a sequence of atomic action names. Step 6: Pass the name sequence of the atomic actions decomposed in step 5 through the action API mapper to obtain the action API; Step 7: The multimodal model is used to collect information about changes in the environment when the agent executes the action API in step 6. The multimodal model determines whether the agent successfully executes the action API in step 6. If successful, the environment changes when the agent executes the action API in step 6 are collected through the multimodal model and stored in the memory mechanism; When the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six, thus obtaining a trained multimodal model. If it fails, the action is adjusted in time, and the reason for the failure and the environmental changes when the agent executes the action API in step 6 after the adjustment are stored in the memory mechanism; When the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six, thus obtaining a trained multimodal model. Step 8. The trained multimodal model collects the environmental changes when the agent executes the action API in step 6 and passes it back to the memory mechanism.

2. The embodied planning method based on memory drive and closed-loop feedback according to claim 1, characterized in that: In the step 1, the semantic information and position information of the 3D point cloud of the object in the environment are obtained; Synchronize the semantic information and position information of the 3D point cloud of objects in the environment into the memory mechanism; The specific process is: Get the environment RGB image; Process the environment RGB image using the DepthAnything algorithm to obtain a depth map; The environmental RGB image is processed by the SFM algorithm to obtain the internal and external parameters of the camera; The SLAM algorithm is used to process the environmental RGB image, depth map, and internal and external parameters of the camera to obtain the 3D point cloud of each object in the environmental RGB image; The 3D point cloud of each object is input into the multimodal model, and the multimodal model outputs the semantic information and position information of the 3D point cloud; The multimodal model is GPT-4o, Janus or LlaVA.

3. The embodied planning method based on memory drive and closed-loop feedback according to claim 2, characterized in that: In step 3, the domain knowledge and target instructions in the memory mechanism are input into the large language model, and the large language model outputs the intent in the target instruction; The specific process is: Large language models include ChatGPT, DeepSeek, Gemini, Claude, or diffusion model.

4. The embodied planning method based on memory drive and closed-loop feedback according to claim 3, characterized in that: In step 4, the task decomposer decomposes the intention into a subtask sequence; the specific process is: The domain knowledge in the memory mechanism and the intent output by the three language models are input into the large language model, and the large language model decomposes the intent into a sequence of subtasks.

5. The embodied planning method based on memory drive and closed-loop feedback according to claim 4, characterized in that: In step 5, the content of the memory mechanism, the supported action set in the current state, the relevant item knowledge, and the tool knowledge are input into the large language model. The large language model decomposes each subtask into a sequence of atomic action names. The specific process is as follows: There are four types of atomic action names: 1) Navigation action; 2) Perceived action; 3) Operations related to tools; 4) Interactive actions; The memory mechanism includes the following contents: domain knowledge, common sense knowledge, experiential knowledge, current task progress, item location, and item interaction history; The process of obtaining the supported action set in the current state is as follows: The attributes of objects in the current environment, agent information, and common-sense action rules are input into the large language model, which then outputs the set of supported actions in the current state. The action rules that conform to common sense are artificially written rules; The relevant item knowledge refers to items and materials used in the current environment scenario; The tool knowledge refers to the tools used in the current scenario environment.

6. The embodied planning method based on memory drive and closed-loop feedback according to claim 5, characterized in that: In step 6, the name sequence of the atomic actions decomposed in step 5 is passed through the action API mapper to obtain the action API; the specific process is as follows: The action API mapper is a VLA model; The VLA model is open VLA or pi0.

7. The embodied planning method based on memory drive and closed-loop feedback according to claim 6, characterized in that: In step seven, the multimodal model is used to collect the environmental changes when the agent executes the action API in step six, and the multimodal model determines whether the agent successfully executes the action API in step six; If successful, the environment changes when the agent executes the action API in step 6 are collected through the multimodal model and stored in the memory mechanism; When the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six, thus obtaining a trained multimodal model. If it fails, the action is adjusted in time, and the reason for the failure and the environmental changes when the agent executes the action API in step 6 after the adjustment are stored in the memory mechanism; When the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six, thus obtaining a trained multimodal model. The specific process is: The multimodal model is used to collect information about changes in the environment when the agent executes the action API in step six. The multimodal model determines whether the agent successfully executes the action API in step six. If successful, the environment changes when the agent executes the action API in step 6 are collected through the multimodal model and stored in the memory mechanism; When the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six, thus obtaining a trained multimodal model. If it fails, the action is adjusted in time, and the reason for the failure and the environmental changes when the agent executes the action API in step 6 after the adjustment are stored in the memory mechanism; When the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six. The trained multimodal model is obtained. The specific process is as follows: When the multimodal model determines that the reason for the agent's failure to execute the action API in step 6 is an operation failure, the current operation is undone and steps 5, 6, and 7 are re-executed; the reason for the failure and the environmental changes when the agent executes the action API in step 6 after adjustment are stored in the memory mechanism; When the amount of information in the memory mechanism reaches a threshold, the multimodal model is trained using the information in the memory mechanism. The input of the multimodal model is the agent executing the action API in step six, and the output of the multimodal model is the environmental changes when the agent executes the action API in step six, thus obtaining a trained multimodal model. When the multimodal model determines that the reason for the agent's failure to execute the action API in step 6 is a subtask division error, steps 4, 5, 6, and 7 are re-executed; and the reason for the failure and the environmental changes when the agent executes the action API in step 6 after adjustment are stored in the memory mechanism; When the amount of information in the memory mechanism reaches a threshold, the information in the memory mechanism is used to train the multimodal model. The input of the multimodal model is the agent executing the step six action API, and the output of the multimodal model is the environmental changes when the agent executes the step six action API, thereby obtaining a trained multimodal model.

8. The embodied planning method based on memory drive and closed-loop feedback according to claim 7, characterized in that: The environmental changes when the agent executes the action API in step 6 include: The new status of the item, the location of the item, the current subtask completion progress, the name sequence of the atomic action corresponding to each subtask, the name sequence of the historical atomic action corresponding to each subtask, and whether there is any unexpected situation.

9. A storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the embodied planning method based on memory drive and closed-loop feedback as described in any one of claims 1 to 8.

10. An embodied planning device based on memory drive and closed-loop feedback, characterized by: The device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement an embodied planning method based on memory drive and closed-loop feedback as described in any one of claims 1 to 8.

Images