Intelligent agent interaction method and device, equipment, medium and product

Abstract

The invention relates to the technical field of artificial intelligence, and provides an agent interaction method and device, equipment, a medium and a product, and the agent interaction method comprises the steps: obtaining task input information; generating a task link sub-instruction according to the task input information, the short-term memory information and the long-term memory information; calling an execution tool to obtain execution result information; updating the short-term memory information based on the reflection evaluation processing; circularly generating a task link sub-instruction, acquiring execution result information and updating short-term memory information until the task is confirmed to be completed; and updating the long-term memory information according to the short-term memory information. Based on a dual-memory architecture, the time sensitivity to memory information is improved, based on reflection evaluation processing, short-term memory is updated in real time according to execution result feedback, the error correction capability in the interaction process is improved, the next generated task link sub-instruction is optimized, the task completion accuracy is improved, and meanwhile, the reliability and the stability are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an intelligent agent interaction method, device, equipment, medium and product. Background Technology

[0002] Significant progress has been made in agent technology, which is based on large language models (LLM). These agents can understand human task instructions and complete complex tasks through planning, calling tools, and other means.

[0003] However, existing intelligent agents suffer from insufficient reliability and stability during interaction. Specifically, when performing complex multi-step tasks, an agent may deviate from the expected or be interrupted due to a misjudgment in one step, indicating insufficient error correction capabilities. Furthermore, existing agents have limited understanding and memory of long-term contexts, potentially losing early historical information when relying on context and processing data over extended periods. While some agents have adopted Retrieval-Augmented Generation (RAG) techniques to preserve long-term information, this approach exhibits the same sensitivity to information generated at different times, failing to capture deeper connections between memories from different periods and impacting task accuracy.

[0004] Therefore, enhancing error correction capabilities, improving the reliability and stability of interactions, and increasing the accuracy of task completion are urgent problems to be solved in the process of intelligent agent interaction. Summary of the Invention

[0005] This application provides an intelligent agent interaction method, device, equipment, medium, and product to address the shortcomings of existing intelligent agent interaction processes, such as insufficient error correction capabilities and the inability to capture memory associations at different times.

[0006] This application provides an intelligent agent interaction method, including: Obtain task input information; Based on the task input information, short-term memory information, and long-term memory information, generate task link sub-instructions; According to the task chain sub-instructions, the execution tool is invoked to obtain the execution result information; Based on the execution result information, the short-term memory information is updated using reflective evaluation processing; The process of repeatedly generating the task link sub-instructions, obtaining the execution result information, and updating the short-term memory information continues until the task is confirmed to be completed. Update the long-term memory information based on the short-term memory information; Wherein, the short-term memory information represents the memory information stored during the task, the long-term memory information includes long-term stored memory information, and the reflection and evaluation process is used to evaluate the execution results and update the short-term memory information to optimize task link decisions.

[0007] According to the intelligent agent interaction method provided in this application, the step of updating the short-term memory information based on the execution result information and reflection evaluation processing includes: Based on the execution result information, reflection and evaluation information and link symbols are obtained. The reflection and evaluation information represents the score, reasons and next steps of the execution result, and the link symbols represent whether the task is completed. Based on the link symbol, the task link ends when the task is completed; Based on the aforementioned reflection and evaluation information, memory retrieval processing is performed to obtain the first intermediate memory information; Based on the first intermediate memory information and the short-term memory information, and using the first merging and updating process, the updated short-term memory information is obtained; The first merge update process is used to merge the first intermediate memory information with the current short-term memory information, as well as to delete duplicate memories and analyze and replace contradictory information.

[0008] According to the intelligent agent interaction method provided in this application, the step of obtaining reflection and evaluation information and link symbols based on the execution result information includes: Based on the execution result information, obtain the task completion score, task link rationality score, tool call rationality score, scoring reasons, and next step suggestions, and generate the reflection and evaluation information; Based on the execution result information, if the task is not completed, a continuation symbol is generated as the link symbol; if the task is completed, a termination symbol is generated as the link symbol.

[0009] According to the intelligent agent interaction method provided in this application, the step of performing memory retrieval processing based on the reflection and evaluation information to obtain first intermediate memory information includes: Based on the aforementioned reflection and evaluation information, factual information, user preference information, and process information are extracted to form the first intermediate memory information.

[0010] According to the intelligent agent interaction method provided in this application, the step of generating task link sub-instructions based on the task input information, short-term memory information, and long-term memory information includes: Based on the task input information, the short-term memory information, and the long-term memory information, the task is decomposed to generate at least one task link sub-instruction. Where there is a contradiction between the short-term memory information and the long-term memory information, the short-term memory information shall prevail.

[0011] According to the intelligent agent interaction method provided in this application, the step of updating the long-term memory information based on the short-term memory information includes: Based on the short-term memory information, long-term factual information, user preference information, and task information are extracted to form a second intermediate memory information; Based on the second intermediate memory information and the long-term memory information, and using the second merging and updating process, the updated long-term memory information is obtained. The second merge update process is used to merge the second intermediate memory information with the current long-term memory information, as well as to delete duplicate memories and analyze and replace contradictory information.

[0012] According to the intelligent agent interaction method provided in this application, after updating the long-term memory information based on the short-term memory information, the method further includes: Release the memory storing the short-term memory information.

[0013] According to the intelligent agent interaction method provided in this application, the step of obtaining task input information includes: In response to voice input information, based on the voice input information and speech recognition processing, first text information is generated as the task input information; In response to image input information, based on the image input information and image recognition processing, second text information is generated as the task input information; In response to text input information, the text input information is used as the task input information.

[0014] This application also provides an intelligent agent interaction device, including: The multimodal conversion module is used to acquire user input information in different modalities and convert it to form task input information. The planning module is used to generate task link sub-instructions based on the task input information, short-term memory information, and long-term memory information. The tool invocation module is used to invoke the execution tool and obtain execution result information according to the task chain sub-instructions; The reflection module is used to perform reflection and evaluation processing based on the execution result information and obtain reflection and evaluation information; The inference module is used to update the short-term memory information based on the reflection and evaluation information, and to update the long-term memory information based on the short-term memory information.

[0015] A short-term memory module is used to store the short-term memory information; A long-term memory module is used to store the long-term memory information; The planning module, the tool invocation module, the reflection module, the inference module, the short-term memory module, and the long-term memory module work together to implement the above-mentioned intelligent agent interaction method.

[0016] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an intelligent agent interaction method as described above.

[0017] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an intelligent agent interaction method as described above.

[0018] This application also provides a computer program product, including a computer program that, when executed by a processor, implements an intelligent agent interaction method as described above.

[0019] This application provides an intelligent agent interaction method, device, equipment, medium, and product, which has at least the following beneficial effects: By dividing memory information into short-term memory information and long-term memory information, i.e., adopting a dual-memory architecture, it is easier to capture the deep correlation between short-term and long-term memory. When the intelligent agent obtains the user's task input information, it combines short-term and long-term memory information to plan the task and generate the first task chain sub-instruction; according to the task chain sub-instruction, it calls the corresponding execution tool to obtain execution result information to obtain real-time execution feedback; then, based on the execution result information, it updates the short-term memory information based on reflection and evaluation processing, so that the short-term memory includes the results of the previous execution process and the evaluation of the execution process; furthermore, when generating the next task chain sub-instruction, it can optimize in real time based on the results of the execution based on the short-term memory information, making the next planned task chain sub-instruction more reasonable and accurate, and can also correct errors based on real-time feedback in the event of execution errors. The process of planning sub-instructions, calling execution tools, reflecting and evaluating execution results, and updating short-term memory information is repeated until the task is completed, and then the long-term memory information is updated based on the short-term memory information. Therefore, based on a dual-memory architecture—that is, memory is divided into short-term and long-term memory—the temporal sensitivity of memory information is improved. When generating task chain sub-instructions, the planning integrates task input information, short-term memory information, and long-term memory information, which can capture the correlation between memories at different times, making the planned task chain sub-instructions more accurate. Simultaneously, based on reflective evaluation processing, after each invocation of the execution tool according to the task chain sub-instruction, the short-term memory is updated in real time based on the execution result feedback. This effectively improves the error correction capability during agent interaction and optimizes the next generated task chain sub-instruction based on real-time execution result feedback until the task is completed, which helps improve the accuracy of task completion and also enhances reliability and stability. Furthermore, after confirming task completion, the long-term memory information is updated based on the short-term memory information to extract and store information with long-term memory value from the short-term memory. This facilitates the provision of long-term information such as user preferences for subsequent tasks, enabling continuous learning and evolution of agent interaction and improving the intelligence and personalization of agent interaction. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating an intelligent agent interaction method provided in this application.

[0022] Figure 2 This is a schematic diagram of information processing for an intelligent agent interaction method provided in this application.

[0023] Figure 3 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The following is combined with Figure 1 and Figure 2 This application describes an intelligent agent interaction method, comprising: S100: Obtain task input information; S200: Generate task link sub-instructions based on the task input information, short-term memory information, and long-term memory information; S300: According to the task chain sub-instruction, call the execution tool to obtain the execution result information; S400: Based on the execution result information, update the short-term memory information according to the reflection and evaluation process; The process of repeatedly generating the task link sub-instructions, obtaining the execution result information, and updating the short-term memory information continues until the task is confirmed to be completed. S500: Update the long-term memory information based on the short-term memory information; Wherein, the short-term memory information represents the memory information stored during the task, the long-term memory information includes long-term stored memory information, and the reflection and evaluation process is used to evaluate the execution results and update the short-term memory information to optimize task link decisions.

[0026] By dividing memory information into short-term and long-term memory—a dual-memory architecture—it is easier to capture the deep correlation between short-term and long-term memory. When the agent receives task input from the user, it combines short-term and long-term memory information to plan the task and generate the first task chain sub-instruction. Based on the task chain sub-instruction, it calls the corresponding execution tool to obtain execution result information for real-time execution feedback. Then, based on the execution result information, and through reflection and evaluation, it updates the short-term memory information to include the results of previous execution processes and their evaluations. Furthermore, when generating the next task chain sub-instruction, based on the short-term memory information, it can optimize the task chain decision in real time based on the already executed results, making the next planned task chain sub-instruction more reasonable and accurate. It can also correct errors based on real-time feedback. This process of planning sub-instructions, calling execution tools, reflecting and evaluating execution results, and updating short-term memory information is repeated until the task is completed, at which point the long-term memory information is updated based on the short-term memory information.

[0027] Therefore, based on the dual memory architecture, which divides memory into short-term memory information and long-term memory information, the time sensitivity of memory information is improved. When generating task link sub-instructions, the task input information, short-term memory information and long-term memory information are integrated for planning. This can capture the correlation of memory at different times and make the planned task link sub-instructions more accurate.

[0028] Meanwhile, based on reflective evaluation processing, after each call to the execution tool according to the task link sub-instruction, the short-term memory information is updated in real time according to the execution result feedback, which effectively improves the error correction capability in the interaction process of the intelligent agent, and can optimize the task link decision based on the real-time execution result feedback, making the next generated task link sub-instruction more accurate until the task is completed. This is conducive to improving the accuracy of task completion, as well as improving reliability and stability.

[0029] In addition, after the task is confirmed to be completed, the long-term memory information is updated based on the short-term memory information in order to extract and store information with long-term memory value from the short-term memory. This facilitates the provision of long-term information such as user preferences for subsequent tasks, which is conducive to the continuous learning and evolution of intelligent agent interaction and improves the intelligence and personalization of intelligent agent interaction.

[0030] It is important to emphasize that, compared to breaking down a task into multiple instructions and then distributing those instructions for execution, this application, after executing a task chain sub-instruction, provides real-time feedback based on the execution result information. Through reflection, evaluation, and updating short-term memory information, it can optimize task chain decisions, that is, optimize and adjust the next generated task chain sub-instruction. This makes subsequent task analysis and planning more reasonable and accurate, enables timely response to external feedback during execution, allows for timely error correction, avoids error accumulation, and facilitates obtaining more accurate and reliable task execution results.

[0031] It should be noted that generating task chain sub-instructions, obtaining execution result information, updating short-term memory information, and updating long-term memory information can be implemented using neural network models, such as Large Language Models (LLMs). Furthermore, when implementing the above processing using a Large Language Model, the output of the Large Language Model can be made closer to the requirements based on the Prompt project. The Prompt project guides and "programs" the Large Language Model by designing and inputting text instructions or questions to it, causing it to output the desired result. For the different steps of generating task chain sub-instructions, obtaining execution result information, updating short-term memory information, and updating long-term memory information, different input information structures can be designed based on the Prompt project.

[0032] In some embodiments of this application, the task input information can be obtained by direct input from the user, such as the user directly inputting text information of the task content; or it can be obtained by converting the user's original input information, such as the user inputting information by voice or providing image input information, and then converting the original input information into text information that is more suitable for processing through voice recognition conversion, image recognition conversion and other processing.

[0033] It should be noted that short-term memory and long-term memory information can be empty. For example, in the initial state, both short-term and long-term memory information are empty. During continuous interaction between the user and the intelligent agent, the short-term and long-term memory information is continuously updated. Furthermore, short-term memory information can be released after the task is completed. Specifically, this can be done by releasing the memory storing the short-term memory information. In some embodiments, it can also be released after completing a related task. The related task can be confirmed based on a preset time interval between tasks. Alternatively, it can be implemented through periodic release, such as daily release or release after a preset time has elapsed following task completion.

[0034] Short-term memory information can include factual information, user preference information, process information, etc. Long-term memory information can include factual information, user preference information, and historical task information, etc. To more intuitively understand the process of the intelligent agent interaction method of this application, an illustrative example is given: Hypothetical scenario: There are three drinks on the table: a cola, a coffee, and a water bottle, as well as other items such as napkins and bread; The user gives a voice command to the robot, which acts as an intelligent agent: "I'm a little thirsty, do you have anything I can drink?" After the robot receives the voice command, it converts it into text information as the task input information. Then, it combines short-term memory information and long-term memory information to perform task parsing and planning, and generates the first task link sub-instruction: "Call the camera tool to obtain surrounding information". The robot uses its camera to capture images, performs image recognition processing, and obtains the execution result information: "There is bread, napkins, a water bottle, and a beverage with its label covered by the water bottle on the table." Based on the execution results, the robot reflects on and evaluates the process, updating its short-term memory information. This short-term memory information may include: Fact: "The camera detected a table 3 meters directly in front of the robot, with drinks on the table." User preferences: "What does the user want to drink?" Process: {"User instruction": "I'm a little thirsty, do you have anything I can drink?";} Sub-command 1: "Invoke the camera tool to obtain surrounding information"; "Execution result of sub-instruction 1": "There is bread, napkins, a water bottle, and a beverage with the label covered by the water bottle on the table"; Evaluation: "Sub-instruction 1 was completed correctly"; "Suggestion": "Adjust the position and take another picture to identify the object to obtain complete information."

[0035] The robot then generates the next task link sub-instruction based on the task input information, combined with short-term and long-term memory information. Specifically, it can combine the information in short-term memory such as "the beverage with the label covered by the water bottle" and "adjust the position and take another picture to identify the item to obtain complete information" for real-time feedback and optimization, and generate the instruction "call the mobile tool to move forward two meters and call the camera tool to obtain surrounding information" as the next task link sub-instruction. After the robot uses its mobility tool to move, it uses its camera tool again to capture images. After image recognition processing, it obtains the execution result information: there is bread, napkins, cola, coffee, and a water bottle on the table. Based on the execution results, the robot reflects on and evaluates the process, updating its short-term memory information. This short-term memory information may include: Fact: "The camera detected a table with drinks on it, located 1 meter directly in front of the robot." User preferences: "What does the user want to drink?" Process: {"User instruction": "I'm a little thirsty, do you have anything I can drink?";} Sub-command 1: "Invoke the camera tool to obtain surrounding information"; "Execution result of sub-instruction 1": "There is bread, napkins, a water bottle, and a beverage with the label covered by the water bottle on the table"; Sub-command 2: "Invoke the movement tool to move forward two meters and invoke the camera tool to obtain surrounding information"; "Execution result of sub-instruction 2": "There is bread, napkins, cola, coffee and water bottle on the table"; Evaluation: "Sub-instruction 2 was completed correctly"; "Suggestion": "Filter beverage item information and call the voice broadcast tool to announce the results"}.

[0036] The robot then generates the next task link sub-instruction based on the task input information, combined with short-term and long-term memory information. Specifically, it can combine the information in the short-term memory such as "there is bread, napkins, cola, coffee and water bottle on the table" and "filter beverage information and call the voice broadcast tool to broadcast the results" for optimization in real time, and generate the instruction "call the processing tool to filter beverages from "there is bread, napkins, cola, coffee and water bottle on the table" and call the voice broadcast tool to broadcast the results" as the next task link sub-instruction. After the robot calls the processing tool to filter beverage items, it obtains the result "There is a Coke, coffee and a water bottle on the table". Then it calls the voice broadcast tool to broadcast the voice content "There is a Coke, coffee and a water bottle on the table", and obtains the execution result information: beverage item filtering successful and voice broadcast successful. Based on the execution results, the robot performs reflection and evaluation to confirm that the task has been completed and updates its short-term memory information. Specific short-term memory information may include: Fact: "The camera detected a table with drinks on it, located 1 meter directly in front of the robot." User preferences: "What does the user want to drink?" Process: {"User instruction": "I'm a little thirsty, do you have anything I can drink?";} Sub-command 1: "Invoke the camera tool to obtain surrounding information"; "Execution result of sub-instruction 1": "There is bread, napkins, a water bottle, and a beverage with the label covered by the water bottle on the table"; Sub-command 2: "Invoke the movement tool to move forward two meters and invoke the camera tool to obtain surrounding information"; "Execution result of sub-instruction 2": "There is bread, napkins, cola, coffee and water bottle on the table"; Sub-instruction 3: "Invoke the processing tool to filter the beverage items 'There is bread, napkins, cola, coffee and water bottle on the table' and invoke the voice broadcast tool to broadcast the results." "Execution result of sub-instruction 3": "Beverage item selection successful, voice broadcast successful"; "Evaluation": "Sub-instruction 3 was completed correctly, task accomplished"}.

[0037] Once the robot confirms task completion, it updates its long-term memory based on short-term memory information. Long-term memory information may include: Fact: There were drinks on the table; User preference: None; Historical Task: {“User Instruction”: “I’m a little thirsty, do you have anything I can drink?”; “Task Link”: “Sub-instruction 1”: “Use the camera tool to get surrounding information”; “Sub-instruction 2”: “Use the movement tool to move forward two meters and use the camera tool to get surrounding information”; “Sub-instruction 3”: “Use the processing tool to filter beverage items “There is bread, napkins, cola, coffee and water bottle on the table” and use the voice broadcast tool to broadcast the results”}.

[0038] Based on the above illustrative example, in the above scenario, a typical intelligent agent interaction might use the initial image recognition result of "bread, napkins, water bottle, and a beverage with its label obscured by the water bottle" as the result and feed it back to the user via a voice broadcast tool. This presents the problem that the user cannot clearly know what exactly the "beverage with its label obscured by the water bottle" is, and the result may contain information about items that are not beverages, such as "bread" and "napkins," which would prevent the user from completing the task accurately and reliably.

[0039] As illustrated in the above illustrative example of the complete interaction process, the intelligent agent interaction method of this application achieves self-correction capabilities by introducing reflection, evaluation, and dynamic adjustment mechanisms after each step of task execution. This includes correcting unclear information such as "the label of a beverage is obscured by a water bottle," and removing non-beverage items like "bread" and "napkins" from the final result. This ensures that the intelligent agent is less likely to deviate from its goal when facing complex, multi-step tasks, making the interaction process more stable and reliable. By constructing a dual-memory architecture with short-term and long-term hierarchies, the integrity of the current task context information is guaranteed, while accurately grasping the user's long-term preferences is achieved. This allows the intelligent agent to make more accurate judgments and decisions based on dynamically changing interaction scenarios. After the task is completed, valuable information is retrieved from short-term memory and transferred to long-term memory, enabling the intelligent agent to continuously learn from interactions, continuously optimize its knowledge base and user profile, and provide increasingly personalized and intelligent services.

[0040] It should be noted that the long-term memory information in the above illustrative example is assumed to be empty. When the user preference in the long-term memory information includes information such as "the user is accustomed to getting drinks from the table", when analyzing and planning to generate task chain sub-instructions, the instruction "call the mobile tool to move to the table" can be generated first to better match the user's personalized preferences.

[0041] It is important to emphasize that, in some embodiments of this application, the entity executing the intelligent agent interaction method is not limited to a single intelligent agent. That is, in some embodiments, it can be implemented by a single intelligent agent; in other embodiments, it can be implemented by an intelligent agent collaborative system formed by multiple intelligent agents. A master intelligent agent can be responsible for planning and reflection, while multiple slave intelligent agents are responsible for calling execution tools, obtaining execution result information, and transmitting it to the master intelligent agent. The master intelligent agent performs comprehensive reflection and evaluation based on all collected execution results, thereby improving the processing efficiency of complex tasks. Furthermore, the task input information is not limited to user input; it can be triggered by preset timed events or environmental sensor signals. For example, when a sensor detects that the indoor temperature exceeds a threshold, it automatically initiates the task chain with the task input information of "turn on the air conditioner and adjust it to a comfortable temperature."

[0042] In some embodiments of the intelligent agent interaction method of this application, step S400 includes: Based on the execution result information, reflection and evaluation information and link symbols are obtained. The reflection and evaluation information represents the score, reasons and next steps of the execution result, and the link symbols represent whether the task is completed. Based on the link symbol, the task link ends when the task is completed; Based on the aforementioned reflection and evaluation information, memory retrieval processing is performed to obtain the first intermediate memory information; Based on the first intermediate memory information and the short-term memory information, and using the first merging and updating process, the updated short-term memory information is obtained; The first merge update process is used to merge the first intermediate memory information with the current short-term memory information, as well as to delete duplicate memories and analyze and replace contradictory information.

[0043] Based on the execution results, structured reflection and evaluation information and link symbols are output. The link symbols provide precise termination signals for the agent's cyclical process, clearly confirming task completion. The reflection and evaluation information guides the next optimization step, ensuring that each task link sub-instruction generated in the task link is under control. It allows for dynamic analysis and planning of the generated task link sub-instructions based on actual execution results, enabling timely error correction and preventing the accumulation of errors that deviate from the task objective.

[0044] Meanwhile, by retrieving and processing the reflection and evaluation information, the first intermediate memory information is obtained and merged with the current short-term memory information to ensure the real-time nature, accuracy and conciseness of the short-term memory information.

[0045] In some embodiments of this application, obtaining reflection and evaluation information may be achieved by processing execution result information together with task input information, short-term memory information, and long-term memory information, so as to enable more comprehensive reflection and evaluation.

[0046] It should be noted that when performing the first merge update process, if there is a contradiction between the short-term memory information and the first intermediate memory information, an analysis and replacement process is required. Specifically, if the first intermediate memory information provides more recent or more accurate new content, then it replaces the contradictory content in the current short-term memory information; if the new content in the first intermediate memory information is vague and lacks detail, then the original content in the short-term memory information is retained, and the contradictory content in the first intermediate memory information is discarded.

[0047] In some embodiments of this application, for memory retrieval processing, when implemented using a large language model based on a Prompt project, the specific construction of the Prompt project can be as follows (illustrating key prompt word information of the Prompt project, not the complete Prompt project): Introduction: You are an expert in short-term memory retrieval.

[0048] Objective: Your objective is to retrieve the memory of a specified user ID.

[0049] Note: Infer facts, user preferences, and process flows from the provided text.

[0050] Constraints: Facts, user preferences, and process links should be concise and informative. Do not remember the details of the provided users or agents; only remember the facts, user preferences, and process links.

[0051] In some embodiments of this application, for the first merge update process, when implemented using a large language model based on a Prompt project, the specific construction of the Prompt project can be as follows (illustrating key prompt word information of the Prompt project, not the complete Prompt project): Introduction: You are an expert in merging, updating, and organizing memories.

[0052] Objective: When given existing memories and new information, your task is to merge and update the memory list to reflect the most accurate and up-to-date information. You will also receive a matching score for each existing memory with the new information, ensuring you use this information to make informed decisions about which memories to update or merge.

[0053] Instructions: Eliminate duplicate memories and merge related memories to ensure the list is concise and up-to-date; if a memory contradicts new information, evaluate both; if a new memory provides a more recent or accurate update, replace the old memory; if a new memory seems inaccurate or lacks detail, retain the original memory and discard the old one; maintain a consistent and clear style across all memories to ensure each entry is concise and informative; if a new memory is a variation or extension of an existing memory, update the existing memory to reflect the new information.

[0054] In some embodiments of the intelligent agent interaction method of this application, the step of obtaining reflection and evaluation information and link symbols based on the execution result information includes: Based on the execution result information, obtain the task completion score, task link rationality score, tool call rationality score, scoring reasons, and next step suggestions, and generate the reflection and evaluation information; Based on the execution result information, if the task is not completed, a continuation symbol is generated as the link symbol; if the task is completed, a termination symbol is generated as the link symbol.

[0055] The task is scored from three perspectives: task completion, task chain rationality, and tool utilization rationality. Reasons for each score and suggestions for future steps are provided, making the evaluation of previous task planning more comprehensive and complete. The scoring reasons help determine the reasonableness of the scores, facilitating error correction, while the suggestions for future steps help dynamically optimize the next task chain sub-instruction. This generates reflective evaluation information, providing a foundation for task analysis, planning, and error correction, helping to pinpoint the root causes of errors and providing comprehensive feedback.

[0056] For example, when the task completion rate is low but the link rationality is high, it can be known that the task planning direction is correct and the task can be completed by continuing execution; while when the task completion rate and the task link rationality are high, but the tool rationality is low, it can be inferred that the execution tool should be changed to correct the situation, rather than changing the entire task planning.

[0057] In some embodiments of this application, the link symbol may include symbols such as continue and terminate.

[0058] In some embodiments of this application, when processing the acquisition of reflection and evaluation information is implemented using a large language model based on a Prompt project, the specific construction of the Prompt project can be as follows (illustrating key prompt word information of the Prompt project, not the complete Prompt project): Description: You are an expert who excels at evaluating and providing suggestions on robot task performance.

[0059] Objective: Your objective is to evaluate the robot's task performance and provide reasons and suggestions for improvement.

[0060] Note: The evaluation includes: task completion, task flow rationality, and tool suitability (out of 5 points), and provides the reasons for the scores and suggestions.

[0061] To more intuitively understand the process of reflection and evaluation, an example is given: Based on the aforementioned example of a user issuing a voice command to a robot acting as an intelligent agent, "I'm a little thirsty, do you have anything I can drink?", the robot executes the first task chain sub-instruction, "call the camera tool to obtain surrounding information," and obtains the execution result information: "There is bread, napkins, a water bottle, and a beverage with the label covered by the water bottle on the table." After the robot performs reflection and evaluation based on the execution results, the resulting reflection and evaluation information may include (with a 5-point rating): Task completion rating: 3 / 5; Task chain rationality score: 4 / 5; Tool rationality rating: 5 / 5; Current link score: 4; Previous link accumulated score: 0; Accumulated score: 4; Number of steps / links: 1; Reasons for the rating: 1. Since the user asked about items that can be drunk, the returned information needs to filter out items that cannot be drunk; 2. Because some beverages have obscured labels, the robot needs to use the mobile module to move its position and then use the camera to identify them to obtain complete information. Next step suggestion: Adjust the position and take another picture to identify the object to obtain complete information.

[0062] Obtained link symbol: Continue.

[0063] The current link score can be determined by the average of the task completion score, the task link rationality score, and the tool rationality score.

[0064] It should be noted that the above reflection and evaluation information is only an illustrative example. It may also include factual information, such as the factual information in the example above, such as "there is bread, napkins, water bottle, and beverage with the label covered by the water bottle on the table", so that it can be included in the short-term information after memory retrieval processing and updating of short-term memory information.

[0065] In some embodiments of the intelligent agent interaction method of this application, the step of performing memory retrieval processing based on the reflection and evaluation information to obtain first intermediate memory information includes: Based on the aforementioned reflection and evaluation information, factual information, user preference information, and process information are extracted to form the first intermediate memory information.

[0066] Based on the large language model, the execution result information is reflected and evaluated. The resulting reflection and evaluation information may include redundant information, which is extracted through memory retrieval. Specifically, factual information, user preference information, and process information are extracted, and unstructured information content is transformed into structured information content. This facilitates the subsequent merging and updating of short-term memory information, making short-term memory information more concise, clear, and organized, and improving the useful information density of short-term memory information.

[0067] It should be noted that during the memory retrieval process, the content can be modified based on the scoring information in the reflection and evaluation information. For example, in some embodiments, when the current link score is high, the retrieved content is appropriately modified to "the sub-instruction is completed correctly and the task planning for the next stage can proceed," and when the current link score is low, the retrieved content is appropriately modified to "the sub-instruction is not completed correctly and the task planning for the current stage needs to be readjusted for error correction."

[0068] It is understandable that the information structure of the first intermediate memory information formed by memory retrieval processing corresponds to the information structure of the short-term memory information. That is, the first intermediate memory information includes an information structure of factual information, user preference information, and process information, and the short-term memory information also includes an information structure of factual information, user preference information, and process information.

[0069] In some embodiments of the intelligent agent interaction method of this application, step S200 includes: Based on the task input information, the short-term memory information, and the long-term memory information, the task is decomposed to generate at least one task link sub-instruction. Where there is a contradiction between the short-term memory information and the long-term memory information, the short-term memory information shall prevail.

[0070] Because of the dual-memory architecture, task analysis and planning integrate short-term and long-term memory information to generate task chain sub-instructions. During this process, conflicts may arise between short-term and long-term memory information. Since short-term memory information, which is closer to the current moment, is generally more accurate than the generalized long-term memory information, it should have a higher decision-making weight in task analysis and planning. Therefore, when there is a conflict between short-term and long-term memory information, prioritizing short-term memory information is beneficial for more accurate task completion and better aligns with behavioral logic and user expectations.

[0071] To illustrate this more intuitively, consider the following example: In a food ordering task, long-term memory information includes the user's preference of "not eating spicy food," while short-term memory information includes the user's preference of "wanting to try spicy food today." Prioritizing short-term memory information for task analysis and planning will more accurately meet the user's task requirements.

[0072] In some embodiments of the intelligent agent interaction method of this application, step S500 includes: Based on the short-term memory information, long-term factual information, user preference information, and task information are extracted to form a second intermediate memory information; Based on the second intermediate memory information and the long-term memory information, and using the second merging and updating process, the updated long-term memory information is obtained. The second merge update process is used to merge the second intermediate memory information with the current long-term memory information, as well as to delete duplicate memories and analyze and replace contradictory information.

[0073] After a task is completed, short-term memory contains task information, which includes factual information, user preference information, and process information corresponding to sub-instructions in each task chain. Similar to updating short-term memory, long-term factual information, user preference information, and task information are extracted from short-term memory to form second intermediate memory. This second intermediate memory is then merged with the current long-term memory to update the long-term memory. In this way, long-term stable facts and newly discovered user preferences—information with long-term reference value—can be updated into long-term memory. This allows the agent to accumulate experience and knowledge from each task interaction, continuously improving user habits and making subsequent task analysis and planning more closely aligned with user needs. This contributes to providing increasingly accurate and personalized agent interaction experiences.

[0074] It should be noted that when performing the second merge update process, if there is a contradiction between short-term memory information and long-term memory information, an analysis and replacement process is required. Specifically, if the short-term memory information provides newer or more accurate memory content, then it replaces the contradictory memory content in the current long-term memory information; if the new memory content in the short-term memory information is vague and lacks detail, then the original memory content in the long-term memory information is retained, and the contradictory memory content in the short-term memory information is discarded.

[0075] For example, in long-term memory information, user preferences include "like drinking sour drinks," while in short-term memory information, user preferences include "like drinking sweet drinks." For this contradictory information, since short-term memory information provides more recent and accurate content, the original "like drinking sour drinks" in long-term memory information can be replaced with "like drinking sweet drinks" to update the information.

[0076] It is understandable that, unlike short-term memory, long-term memory does not have the result and evaluation of each sub-instruction execution; only the parts that involve long-term memory are retained.

[0077] In some embodiments of this application, for the second merge update process, when implemented using a large language model based on a Prompt project, the specific construction of the Prompt project can be as follows (illustrating key prompt word information of the Prompt project, not the complete Prompt project): Introduction: You are an expert in long-term memory retrieval.

[0078] Objective: Your objective is to extract long-term memory data from a specified user ID.

[0079] Note: Infer long-term facts, long-term user preferences, and task information from the provided text.

[0080] Constraints: Long-term facts, long-term user preferences, and task information should be concise and informative, used to store information over long periods of time, typically including information that has been repeatedly reinforced. Do not remember details of the provided users or agents; only remember long-term facts, long-term preferences, and long-term memories.

[0081] In some embodiments of the intelligent agent interaction method of this application, after updating the long-term memory information according to the short-term memory information, the method further includes: Release the memory storing the short-term memory information.

[0082] After completing the task and updating the long-term memory information based on the short-term memory information, the short-term memory is released so that the short-term memory of the previous task does not affect the next task, achieving the effect of task isolation. This helps prevent contextual confusion between tasks and ensures that each agent interaction begins in a clean environment, thereby improving stability and predictability.

[0083] At the same time, releasing memory can prevent expired information from occupying resources for a long time. Especially in long-running scenarios or intelligent agent devices with limited hardware resources, it can avoid problems such as memory overflow and improve the stability and reliability of intelligent agent operation.

[0084] In some embodiments of this application, short-term memory can be released not only after a task is completed, but also after a preset number of tasks are completed, that is, short-term memory can be retained between adjacent preset number of tasks; it can also be implemented by periodically releasing the memory storing short-term memory information; in some embodiments, the memory storing short-term memory information can be released after a preset number of tasks are completed or after a preset time is reached.

[0085] In some embodiments of the intelligent agent interaction method of this application, step S100 includes: In response to voice input information, based on the voice input information and speech recognition processing, first text information is generated as the task input information; In response to image input information, based on the image input information and image recognition processing, second text information is generated as the task input information; In response to text input information, the text input information is used as the task input information.

[0086] For input information of different modalities, appropriate processing is performed to form text information that is easy for the intelligent agent to process, which is then used as task input information to trigger subsequent processing. In this way, support for multimodal input information is achieved, which can adapt to user input methods in different scenarios, allowing users to interact with the intelligent agent in a more natural and convenient way, such as speaking or displaying images, which helps to improve the convenience and flexibility of intelligent agent interaction.

[0087] It should be noted that when both voice and image input are present, the first text information can be combined with the second text information to form the task input information. For example, if a user shows an image to the agent or uses the agent to take a picture and says "What is this?", the agent performs speech recognition processing on the voice input to generate the first text information, and performs image recognition processing on the shown image or the currently captured image to generate the second text information. The first and second text information are then combined to form the task input information. Similarly, when both text and image input are present, the second text information can be combined with the text input information to form the task input information.

[0088] It should be noted that multimodal input information can also include video information, etc., which is processed through video recognition to form third-party text information as task input information.

[0089] The following describes an intelligent agent interaction device provided in this application. The intelligent agent interaction device described below and the intelligent agent interaction method described above can be referred to in correspondence.

[0090] This application also provides an intelligent agent interaction device, including: The multimodal conversion module is used to acquire user input information in different modalities and convert it to form task input information. The planning module is used to generate task link sub-instructions based on the task input information, short-term memory information, and long-term memory information. The tool invocation module is used to invoke the execution tool and obtain execution result information according to the task chain sub-instructions; The reflection module is used to perform reflection and evaluation processing based on the execution result information and obtain reflection and evaluation information; The inference module is used to update the short-term memory information based on the reflection and evaluation information, and to update the long-term memory information based on the short-term memory information.

[0091] A short-term memory module is used to store the short-term memory information; A long-term memory module is used to store the long-term memory information; The planning module, the tool invocation module, the reflection module, the inference module, the short-term memory module, and the long-term memory module work together to implement the above-mentioned intelligent agent interaction method.

[0092] In an intelligent agent, memory information is divided into short-term memory information and long-term memory information through short-term memory module and long-term memory module, that is, a dual memory architecture is adopted to facilitate the capture of the deep correlation between short-term memory and long-term memory.

[0093] In the intelligent agent, the multimodal conversion module converts user input information from different modalities, such as voice input information and image input information, into task input information.

[0094] When the planning module obtains the user's task input information, it combines short-term memory and long-term memory information to perform task planning and generate the first task chain sub-instruction. The tool invocation module invokes the corresponding execution tool according to the task chain sub-instruction to obtain execution result information and obtain real-time execution feedback. Then, the reflection module obtains reflection evaluation information based on the execution result information and reflection evaluation processing. The inference module updates the short-term memory information based on the reflection evaluation information, so that the short-term memory includes the results of the previous execution process and the evaluation of the execution process. Furthermore, when the planning module generates the next task chain sub-instruction, it can optimize it in real time based on the results of the execution based on the short-term memory information, making the next planned task chain sub-instruction more reasonable and accurate. At the same time, it can also correct errors based on real-time feedback in the event of execution errors.

[0095] The process involves the cyclic planning module executing planning sub-instructions, the tool calling module calling and executing tools, the reflection module reflecting on and evaluating the execution results, and the inference module updating short-term memory information. This continues until the task is completed, at which point the inference module updates long-term memory information based on the short-term memory information.

[0096] Therefore, based on the dual memory architecture, which divides memory into short-term memory information and long-term memory information, the time sensitivity of memory information is improved. When generating task link sub-instructions, the task input information, short-term memory information and long-term memory information are integrated for planning. This can capture the correlation of memory at different times and make the planned task link sub-instructions more accurate.

[0097] Meanwhile, based on reflective evaluation processing, after each call to the execution tool according to the task chain sub-instruction, the short-term memory is updated in real time according to the execution result feedback, which effectively improves the error correction capability in the interaction process of the intelligent agent. It can also optimize the next generated task chain sub-instruction according to the real-time execution result feedback until the task is completed, which is conducive to improving the accuracy of task completion, as well as improving reliability and stability.

[0098] In addition, after the task is confirmed to be completed, the long-term memory information is updated based on the short-term memory information in order to extract and store information with long-term memory value from the short-term memory. This facilitates the provision of long-term information such as user preferences for subsequent tasks, which is conducive to the continuous learning and evolution of intelligent agent interaction and improves the intelligence and personalization of intelligent agent interaction.

[0099] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions stored in the memory 830 to execute the aforementioned intelligent agent interaction method.

[0100] The electronic devices provided in this application can specifically be robots, mobile phones, or other devices that can act as intelligent agents.

[0101] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute an intelligent agent interaction method provided by the above methods.

[0103] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform an intelligent agent interaction method provided by the methods described above.

[0104] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0106] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0107] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for intelligent agent interaction, characterized in that, include: Obtain task input information; Based on the task input information, short-term memory information, and long-term memory information, generate task link sub-instructions; According to the task chain sub-instructions, the execution tool is invoked to obtain the execution result information; Based on the execution result information, the short-term memory information is updated using reflective evaluation processing; The process of repeatedly generating the task link sub-instructions, obtaining the execution result information, and updating the short-term memory information continues until the task is confirmed to be completed. Update the long-term memory information based on the short-term memory information; Wherein, the short-term memory information represents the memory information stored during the task, the long-term memory information includes long-term stored memory information, and the reflection and evaluation process is used to evaluate the execution results and update the short-term memory information to optimize task link decisions.

2. The intelligent agent interaction method according to claim 1, characterized in that, The step of updating the short-term memory information based on the execution result information and through reflective evaluation processing includes: Based on the execution result information, reflection and evaluation information and link symbols are obtained. The reflection and evaluation information represents the score, reasons and next steps of the execution result, and the link symbols represent whether the task is completed. Based on the link symbol, the task link ends when the task is completed; Based on the aforementioned reflection and evaluation information, memory retrieval processing is performed to obtain the first intermediate memory information; Based on the first intermediate memory information and the short-term memory information, and using the first merging and updating process, the updated short-term memory information is obtained; The first merge update process is used to merge the first intermediate memory information with the current short-term memory information, as well as to delete duplicate memories and analyze and replace contradictory information.

3. The intelligent agent interaction method according to claim 2, characterized in that, The step of obtaining reflection and evaluation information and link symbols based on the execution result information includes: Based on the execution result information, obtain the task completion score, task link rationality score, tool call rationality score, scoring reasons, and next step suggestions, and generate the reflection and evaluation information; Based on the execution result information, if the task is not completed, a continuation symbol is generated as the link symbol; if the task is completed, a termination symbol is generated as the link symbol.

4. The intelligent agent interaction method according to claim 2, characterized in that, The step of performing memory retrieval processing based on the reflection and evaluation information to obtain first intermediate memory information includes: Based on the aforementioned reflection and evaluation information, factual information, user preference information, and process information are extracted to form the first intermediate memory information.

5. The intelligent agent interaction method according to claim 1, characterized in that, The step of generating task link sub-instructions based on the task input information, short-term memory information, and long-term memory information includes: Based on the task input information, the short-term memory information, and the long-term memory information, the task is decomposed to generate at least one task link sub-instruction. Where there is a contradiction between the short-term memory information and the long-term memory information, the short-term memory information shall prevail.

6. The intelligent agent interaction method according to claim 1, characterized in that, The step of updating the long-term memory information based on the short-term memory information includes: Based on the short-term memory information, long-term factual information, user preference information, and task information are extracted to form a second intermediate memory information; Based on the second intermediate memory information and the long-term memory information, and using the second merging and updating process, the updated long-term memory information is obtained. The second merge update process is used to merge the second intermediate memory information with the current long-term memory information, as well as to delete duplicate memories and analyze and replace contradictory information.

7. The intelligent agent interaction method according to claim 1, characterized in that, After updating the long-term memory information based on the short-term memory information, the method further includes: Release the memory storing the short-term memory information.

8. The intelligent agent interaction method according to claim 1, characterized in that, The acquisition of task input information includes: In response to voice input information, based on the voice input information and speech recognition processing, first text information is generated as the task input information; In response to image input information, based on the image input information and image recognition processing, second text information is generated as the task input information; In response to text input information, the text input information is used as the task input information.

9. An intelligent agent interaction device, characterized in that, include: The multimodal conversion module is used to acquire user input information in different modalities and convert it to form task input information. The planning module is used to generate task link sub-instructions based on the task input information, short-term memory information, and long-term memory information. The tool invocation module is used to invoke the execution tool and obtain execution result information according to the task chain sub-instructions; The reflection module is used to perform reflection and evaluation processing based on the execution result information and obtain reflection and evaluation information; The inference module is used to update the short-term memory information based on the reflection and evaluation information, and to update the long-term memory information based on the short-term memory information. A short-term memory module is used to store the short-term memory information; A long-term memory module is used to store the long-term memory information; The planning module, the tool invocation module, the reflection module, the inference module, the short-term memory module, and the long-term memory module cooperate to implement an intelligent agent interaction method as described in any one of claims 1 to 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements an intelligent agent interaction method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements an intelligent agent interaction method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements an intelligent agent interaction method as described in any one of claims 1 to 8.

Images