Intelligent agent long-term memory management method and device, and storage medium

Abstract

The application discloses an agent long-term memory management method and device and a storage medium, relates to the technical field of agent management, and comprises the following steps: when a current task is received, extracting a current problem representation of the current task; based on the current problem representation, performing multiple screenings in a long-term memory library to obtain a target memory, wherein the long-term memory library stores a plurality of long-term memories, and each long-term memory comprises an intention field, a process field and a benefit field; inputting the current task and the target memory into a large language model to obtain an execution result; and adjusting the target memory according to the execution result. Through the multiple screening mechanism, the utilization efficiency of the long-term memory library is effectively improved, and it is ensured that the selected target memory is highly relevant to the current task. Meanwhile, combined with the processing capacity of the large language model, a more actual demand conforming execution result can be generated, and the content of the target memory is continuously optimized through a feedback mechanism, so that the task execution capacity of the agent is enhanced.

Description

Technical Field

[0001] This application relates to the field of intelligent agent management technology, and in particular to intelligent agent long-term memory management methods, devices and storage media. Background Technology

[0002] Large language model agents frequently need to repeatedly recall historical experiences in multi-turn dialogues, tool calls, code execution, and long-term task processing. From the perspective of current technology, long-term memory solutions can be roughly divided into several categories: The first category is long context concatenation, which preserves multi-turn history, tool results, and user information within the context as much as possible. Its advantage is its simplicity, but its disadvantages are high cost and the tendency for truly crucial information to be diluted over time. The second category is external retrieval based on semantic similarity, which vectorizes historical content and retrieves it based on similarity. This type of solution is more economical than long context concatenation, but it is prone to mis-retrieval of semantically similar but actually useless information. The third category is summary-based or rule-based memory management, such as compressing history into summaries, user profiles, or a small number of long-term slots, or performing fixed rule cleanup based on time and frequency. These methods can control scale, but it is usually difficult to determine whether a particular memory is actually helpful for the current task. Summary of the Invention

[0003] The main purpose of this application is to provide a method, device and storage medium for managing the long-term memory of an intelligent agent, which aims to solve the technical problems of low efficiency and difficulty in accurately matching the current task requirements of long-term memory management.

[0004] To achieve the above objectives, this application proposes a method for managing the long-term memory of an intelligent agent, the method comprising:

[0005] Upon receiving the current task, extract the current problem representation of the current task;

[0006] Based on the current problem representation, multiple filters are performed in the long-term memory to obtain the target memory. The long-term memory stores multiple long-term memories, each of which includes an intent field, a process field, and a benefit score field.

[0007] The current task and the target memory are combined and input into the large language model to obtain the execution result;

[0008] The target memory is adjusted based on the execution result.

[0009] In one embodiment, the step of performing multiple siftings in the long-term memory based on the current problem representation to obtain the target memory includes:

[0010] Based on the semantic matching and filtering of the current problem representation and the intent fields of each long memory in the long memory, the semantic similarity between the current problem representation and the intent fields of each long memory is obtained.

[0011] Long-term memories with semantic similarity greater than a preset similarity threshold are selected as candidate memories to obtain a candidate memory set;

[0012] Calculate the overall score of each candidate memory in the candidate memory set;

[0013] The candidate memories are filtered based on the comprehensive score to obtain the target memories.

[0014] In one embodiment, the step of calculating the comprehensive score of each candidate memory in the candidate memory set includes:

[0015] Obtain the candidate semantic similarity, historical score, and time decay factor for each candidate memory;

[0016] The candidate semantic similarity, the historical benefit score, and the time decay factor are weighted and summed to obtain the comprehensive score for each candidate memory.

[0017] In one embodiment, the step of obtaining the time decay factor for each candidate memory includes:

[0018] Obtain usage data and current time for each candidate memory;

[0019] If the usage data is that the candidate memory has not been effectively used again after being written into the long-term memory bank, the writing time of the candidate memory shall be used as the most recent effective timestamp.

[0020] In the case where the data used is the candidate memory being used effectively after it has been written into the long-term memory bank, the time when the candidate memory was last used effectively is taken as the most recent effective timestamp.

[0021] The time decay factor of each candidate memory is obtained based on the time difference between the current time and the most recent valid timestamp.

[0022] In one embodiment, the step of adjusting the target memory based on the execution result includes:

[0023] Based on the execution results, task execution information, invocation cost, and execution feedback are obtained.

[0024] The scalar reward value is determined based on the task execution information, the invocation cost, and the execution feedback;

[0025] Based on the scalar reward value, the revenue field of the target memory that has been used is dynamically updated.

[0026] In one embodiment, after the step of adjusting the target memory based on the execution result, the method further includes:

[0027] The task execution information is obtained based on the execution results.

[0028] Based on the task execution information, determine whether there are preset valid process conditions during the task execution process using the target memory;

[0029] If the conditions for a valid process are met during the execution of a task using the target memory, the execution process using the target memory is written into the long-term memory bank as a new long-term memory.

[0030] In one embodiment, the method further includes:

[0031] Retrieve the time interval between the last effective use of each long-term memory in the long-term memory bank, the historical versions of each long-term memory, and the historical memory content of each long-term memory;

[0032] When the time interval is greater than a preset time interval threshold, the benefit field of the long-term memory is reduced by a preset decay rate;

[0033] When the content of the historical memory has the same meaning as the content of the new memory to be written, but the historical version is lower than the version of the new memory to be written, the priority of the long-term memory is reduced or the benefit score field of the long-term memory is reduced.

[0034] When the historical memory content is complementary to the content to be written into the new memory, the historical memory content is incorporated into the content to be written into the new memory, and the priority of the long-term memory is reduced.

[0035] In one embodiment, prior to the step of performing multiple siftings in the long-term memory based on the current problem representation to obtain the target memory, the method further includes:

[0036] Retrieve the intent field, process field, revenue field, version tag, and task information from long-term memory;

[0037] Construct triplet information for each long-term memory based on the intent field, the process field, and the revenue distribution field;

[0038] The triplet information is stored in the long-term memory, along with the version tag, task information, and write time of each long-term memory.

[0039] Furthermore, to achieve the above objectives, this application also proposes an intelligent agent long-term memory management device, which includes:

[0040] An extraction module is used to extract the current problem representation of the current task when the current task is received;

[0041] The filtering module is used to perform multiple filtering operations in the long-term memory based on the current problem representation to obtain the target memory. The long-term memory stores multiple long-term memories, and each long-term memory includes an intent field, a process field, and a benefit score field.

[0042] The input module is used to input the current task and the target memory into the large language model to obtain the execution result;

[0043] An adjustment module is used to adjust the target memory based on the execution result.

[0044] In addition, to achieve the above objectives, this application also proposes an intelligent agent long-term memory management device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the intelligent agent long-term memory management method as described above.

[0045] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the intelligent agent long-term memory management method described above.

[0046] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the intelligent agent long-term memory management method described above.

[0047] One or more technical solutions proposed in this application have at least the following technical effects:

[0048] 1) In this embodiment, upon receiving a current task, the current problem representation of the current task is extracted. Based on the current problem representation, multiple filters are performed in the long-term memory to obtain target memories. The long-term memory stores multiple long-term memories, each including an intent field, a process field, and a benefit score field. The current task and the target memories are combined and input into a large language model to obtain an execution result. The target memories are then adjusted based on the execution result. This multiple-filtering mechanism effectively improves the utilization efficiency of the long-term memory, ensuring that the selected target memories are highly relevant to the current task. Simultaneously, combined with the processing capabilities of the large language model, execution results that better meet actual needs can be generated, and the content of the target memories is continuously optimized through a feedback mechanism. This enhances the agent's task execution capabilities, reduces interference from irrelevant memories, and improves the agent's adaptability and accuracy in complex scenarios. Furthermore, the dynamic adjustment of the target memories further strengthens the self-updating and optimization capabilities of the long-term memory.

[0049] 2) By semantically matching the intent field of the current problem with long-term memory, memories with high relevance to the problem can be effectively filtered out, thus improving the accuracy of the filtering. The comprehensive scoring step evaluates candidate memories by combining semantic similarity, historical benefit scores, and time decay factors. It not only considers the semantic similarity of memories but also introduces historical benefits and time decay factors, enabling dynamic adjustment of the weight of each memory for refined selection based on actual needs. Through this multi-dimensional filtering and scoring mechanism, the system can more intelligently select the most valuable memories to respond to the current problem, improving not only the relevance and accuracy of the response but also enhancing the system's adaptability to cope with changing needs in different contexts. The design incorporating the time decay factor considers the timeliness of memories, ensuring that information in the long-term memory bank does not become irrelevant or ineffective over time. This helps the agent maintain high memory quality when facing long-term tasks.

[0050] 3) Adjusting target memories based on execution results not only dynamically adjusts memory priorities but also merges or downgrades memory content according to actual task requirements, thereby avoiding interference from memory redundancy and outdated information. Furthermore, by introducing a time decay factor and a historical version comparison mechanism, the level of precision in memory management is further enhanced, ensuring that the long-term memory bank remains efficient and orderly at all times. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating an embodiment of the intelligent agent long-term memory management method of this application.

[0054] Figure 2 This is a flowchart illustrating Embodiment 2 of the intelligent agent long-term memory management method of this application;

[0055] Figure 3 This is a flowchart illustrating Embodiment 3 of the intelligent agent long-term memory management method of this application;

[0056] Figure 4 A simplified flowchart is provided for one embodiment of the intelligent agent long-term memory management method of this application;

[0057] Figure 5 This is a schematic diagram of the module structure of the intelligent agent long-term memory management device according to an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the intelligent agent long-term memory management method in the embodiments of this application.

[0059] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0060] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0061] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0062] Because existing technologies rely on contextual splicing, but the context window is limited, the cost increases with the length of the history, and truly useful information is easily buried. Secondly, retrieving only based on semantic similarity often retrieves seemingly relevant but actually useless content. For an agent, what truly matters is not a similar text, but rather the steps taken to solve similar problems in the past, the order in which tools were called, and the corrective measures taken after failures. Thirdly, many memory systems simply store and retrieve information without using task results to adjust memories. Theoretically, a success or a failure should change the system's selection of memories for the next time, but current solutions often fail to do so. Furthermore, long-running agents encounter two practical problems. One is that old information becomes outdated, and continuing to use it can mislead the current task; the other is that the same fact may have multiple versions at different times, and without a simple and effective processing mechanism, the system will exhibit inconsistencies. Based on these problems, a more direct long-term memory method is needed: one that can prioritize the selection of useful memories, continuously update memory scores based on usage results, and perform lightweight processing on outdated or conflicting memories.

[0063] This application provides a solution that significantly improves the performance of intelligent agents in complex task scenarios by constructing a long-term memory bank containing intent, process, and reward fields, combined with a multi-stage filtering mechanism and dynamic adjustment strategy. Specifically, each memory is made into a process memory with a score. It is first recalled based on similarity, then filtered a second time based on similarity plus reward score. After the task is completed, the score is written back using the results. Finally, a lightweight time / version handling mechanism is added. This not only accurately filters highly relevant memories based on current task requirements but also optimizes the memory bank content through a dynamic update mechanism, ensuring its continued efficiency and usability. Furthermore, the introduction of a time decay factor further enhances the flexibility of memory management, ensuring that memories that have not been effectively used for a long time do not interfere with the system, while avoiding erroneous decisions due to outdated information.

[0064] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or intelligent agent long-term memory management device capable of the above functions. The following description uses an intelligent agent long-term memory management device as an example to illustrate this embodiment and the subsequent embodiments. All actions involving the acquisition of signals, information, or data in this application are performed in accordance with the relevant data protection regulations of the country where the application is located and with authorization from the owner of the corresponding device.

[0065] Based on this, embodiments of this application provide a method for managing the long-term memory of an intelligent agent, referring to... Figure 1 , Figure 1This is a flowchart illustrating the first embodiment of the intelligent agent long-term memory management method of this application.

[0066] In this embodiment, the agent long-term memory management method includes steps S10 to S40:

[0067] Step S10: Upon receiving the current task, extract the current problem representation of the current task.

[0068] It's important to note that the current task is the task request input by the user into the large language model, which can be any form of query, instruction, or question. The current question representation is the key information extracted from the current task, used to characterize the core requirements of the task. The extraction process can be based on natural language processing techniques, performing semantic parsing, keyword extraction, and intent recognition on the task text to generate a question representation that accurately reflects the task requirements. This representation will serve as the basis for subsequent filtering in the long-term memory, ensuring that the filtering process focuses on content highly relevant to the task.

[0069] Large language models are models capable of handling complex tasks and generating results. Their core lies in optimizing output by combining information from long-term memory. After extracting the current problem representation, the system proceeds to a filtering process to ensure a high degree of match between the selected memory and task requirements. The large language model incorporates an agent that performs multiple filters in the long-term memory based on the current problem representation to obtain the most relevant target memory. During the filtering process, the agent relies not only on semantic similarity matching but also on the memory's benefit score and time decay factor, achieving a multi-dimensional comprehensive evaluation. This method effectively avoids the bias that may result from relying solely on semantic similarity, ensuring that the filtered memories are both relevant and have practical value.

[0070] Step S20: Based on the current problem representation, perform multiple filtering operations in the long-term memory to obtain the target memory. The long-term memory stores multiple long-term memories, each of which includes an intent field, a process field, and a benefit score field.

[0071] It should be noted that the long-term memory bank is a pre-built database that stores multiple long-term memories. Each long-term memory contains an intent field, a process field, and a reward field. Long-term memories are task-related information stored in advance, and their construction method depends on the execution results and feedback mechanisms of historical tasks.

[0072] This embodiment focuses on external memory. The overall idea is to make each memory a process memory with a score and store it in a long-term memory bank.

[0073] The intent field describes the goal or purpose of the memory, indicating the applicable problem type or target representation, and can be composed of text, vector representation, and task labels. The process field consists of reusable execution fragments, such as step sequence, tool call chains, key intermediate states, and failure recovery methods. This process can come directly from the original execution trajectory or be generated by a process extraction module that compresses the execution trajectory. The process extraction module can use rule extraction, template compression, large language model generation, or a combination of these methods. This information provides reusable experience for the agent when facing similar problems. The benefit score field represents the expected effect score of reusing the process under similar intents; it is updated based on the result each time it is used. These fields collectively constitute the core information of the memory, providing a foundation for subsequent filtering and adjustment.

[0074] In the process of multiple screenings, the semantics of the current problem representation and the intent fields of each memory in the long-term memory bank are first semantically matched to initially screen out candidate memories related to the task requirements. Then, the candidate memories are further evaluated to select the most suitable memory as the target memory.

[0075] In one feasible implementation, prior to step S20, the agent long-term memory management method further includes steps S11-S13:

[0076] Step S11: Obtain the intent field, process field, benefit score field, version tag, and task information from long-term memory;

[0077] In practice, multiple long-term memory fields such as intent fields, process fields, revenue distribution fields, version tags, and task information can be obtained. Version tags and task information can be used as metadata, which may include timestamps or time intervals, entity and event tags, source (session or task), confidence level, version number, access frequency, level (short-term or long-term), conflict relationships, etc.

[0078] Step S12: Construct triplet information for each long-term memory based on the intent field, the process field, and the revenue sub-field;

[0079] In practice, the intent field, process field, and reward field can be constructed into a long-term memory IPR triple organization, namely, intent (I), process (P), and reward (R).

[0080] Step S13: Store the triplet information in the long-term memory bank, along with the version tag, task information, and write time of each long-term memory.

[0081] It is understandable that each long-term memory can be: , This indicates the question or intention corresponding to that long-term memory. This indicates the processing procedure, which can be steps, tool call chains, key intermediate results, or corrective measures. This represents the benefit score of this long-term memory. Indicates the write time. Indicates version tag, This indicates task information, which can be a task type or process label, such as tool use, code repair, database query, multi-turn Q&A, etc. Here, it indicates that the process is stored in long-term memory. It's not the original full dialogue, but a compressed, reusable process, such as the order of actions, which tools were used, which steps were prone to failure, and how to fix them. The system reuses the methods for solving problems.

[0082] Step S30: Input the current task and the target memory combination into the large language model to obtain the execution result.

[0083] It's important to note that after inputting the current task and target memory into the large language model, the model generates specific execution results based on the information from both. This process fully utilizes the intent, process, and benefit fields stored in the target memory, combined with the powerful processing capabilities of the large language model, ensuring that the output not only meets task requirements but also reflects the value of past experience. Furthermore, by introducing a dynamic adjustment mechanism, the execution results are further fed back into the long-term memory bank to optimize the priority or content of related memories.

[0084] In implementation, a structured prompt template can be predefined, dividing the input space into four logical regions: System Instruction: Defines the agent's role, capability boundaries, safety rules, and output format requirements, such as "must output a tool call in JSON format" or "directly answer the user's question." Memory Injection Zone: Specifically for placing retrieved target memories; this zone is located after the system instruction and before the current task to ensure the model prioritizes historical experience. Task Description: Contains the current user input, a summary of the dialogue history, and current environmental state information. Thought Guidance: An optional zone used to insert a small number of chain-of-thought examples to guide the model to reason before acting. Before filling the memory injection zone with retrieved target memories, the data in the triples needs to be formatted to better conform to the attention mechanism of large language models.

[0085] Intent Field Transformation: Convert intents into attention labels. For example, if the intent of a memory is to query the weather, generate the label [Related Scenario: Weather Query] and place it at the top of the memory entry to help the large language model quickly determine whether the current input matches the experience.

[0086] Process Field Transformation: Convert processes into few-shot learning examples or pseudocode / step lists. If the original process is a natural language description, rewrite it into a standard dialogue format: User: [Similar Input] → Assistant: [Execute Step 1] → Tool: [Tool Invocation] → Result: [Result] → Assistant: [Final Response]. If the original process contains complex logic, compress it into a step-by-step list with indentation and key verbs, highlighting key parameter extraction logic.

[0087] Profit Score Field Conversion: Convert profit scores into confidence level prompts. If the profit score is higher than a preset high threshold, such as 0.8, add a marker "[High Confidence Recommended Solution]" before the memory. If the profit score is moderate, add a marker "[Reference Solution]". If multiple target memories exist, they are strictly arranged in descending order of comprehensive score, with higher-scoring memories closer to the current input position.

[0088] The transformed target memory can be batch-replaced in the memory injection area placeholder of the template, and then the current task input by the user can be filled into the task description area for fusion, start streaming generation, monitor the output content in real time, and obtain the execution result.

[0089] Step S40: Adjust the target memory based on the execution result.

[0090] In practice, the benefit score field of the target memory can be updated based on the execution results to reflect the actual effect of the memory in the current task. If the execution results show that the application of the target memory is successful, its benefit score is increased; conversely, if the execution results are unsatisfactory, the benefit score is decreased. This dynamic adjustment mechanism ensures that the long-term memory bank can be continuously optimized according to actual usage, making high-value memories more likely to be selected by subsequent tasks. In addition, for memories that may become outdated over time, the system introduces a time decay factor to gradually reduce their priority, thereby avoiding old information from misleading new tasks. To further enhance the flexibility of memory management, the system also performs lightweight processing on conflicting or duplicate memories. For example, when there are multiple versions of a solution to the same problem, the system will automatically select the optimal version and mark it as a recommended memory based on the actual application effect and update frequency of each version, while archiving or downgrading other versions. This method not only effectively reduces memory redundancy but also improves the retrieval efficiency of the memory bank while ensuring information consistency.

[0091] In practical implementation, feedback can be obtained based on the execution results. This feedback can come from whether the task was successful, whether the tool executed successfully, whether the user accepted the results, and a cost factor can also be included. The simplest approach is to compress the feedback into a scalar representation. Therefore, step S40 can include steps A11 to A13:

[0092] Step A11: Obtain task execution information, invocation cost, and execution feedback based on the execution results;

[0093] It's important to note that task execution information indicates whether the task was successful; for example, a successful task is assigned a value of 1, and a failed task is assigned a value of 0. The call cost refers to the resources consumed during task execution, including time, computing power, or other relevant metrics. Execution feedback, on the other hand, represents negative results such as execution errors, factual inaccuracies, or user rejection. It's a comprehensive evaluation of the task outcome and can be derived from user satisfaction, system self-evaluation, or external verification mechanisms. This information collectively constitutes the foundational data for adjusting the target memory, providing a quantitative basis for subsequent optimization.

[0094] Step A12: Determine the scalar reward value based on the task execution information, the invocation cost, and the execution feedback;

[0095] In practical implementation, the weight coefficients corresponding to each parameter can be set in advance, and then the scalar reward value can be calculated based on the weight coefficients, task execution information, the call cost, and the execution feedback, as shown below:

[0096]

[0097] In the above formula, s is the scalar reward value. For task execution information, For the cost of calling, To implement feedback, , , These are weighting coefficients, which can be set to fixed values ​​in advance and do not require repeated adjustments.

[0098] This is a scalar feedback value calculated by the result evaluation module based on the task results. This result evaluation module can be a rules module, execution verification module, testing module, user feedback module, or a combination thereof. For example, in a tool invocation scenario, it can evaluate whether the tool returned a successful response, whether the output format is correct, or whether it passed external validation. In code scenarios, results can be obtained based on whether the test passes or the error message disappears. In multi-turn dialogue scenarios, results can be obtained based on user confirmation, human scoring, or consistency checks. .

[0099] In a preferred embodiment, The original values ​​can be further normalized to a fixed interval, such as [-1, 1]. Here, a value close to 1 indicates that the use of the target memory is significantly effective, a value close to 0 indicates a moderate effect, and a value close to -1 indicates that the use of the target memory has a significant negative impact. The advantage of using a fixed interval is that it allows different tasks to use the same set of update rules and maintains stable score updates.

[0100] Step A13: Based on the scalar reward value, dynamically update the revenue field of the target memory that has been used.

[0101] In practice, the already memorized profit points can be dynamically updated using a scalar reward value, as shown in the following formula:

[0102]

[0103] In the above formula, These are weighting coefficients, which can be set according to requirements. The reward score is determined by the following formula: if the target memory is effective, the scalar reward value is high, and the dynamically updated reward score is high; if the target memory is ineffective, the scalar reward value is low, and the dynamically updated reward score is low. The update speed is controlled by this formula. The update method is simple, stable, and suitable for online operation, requiring no additional model parameter training. In practical implementation, normalization can also be used to keep the values ​​within a range, making the system more stable. This can be understood from the perspective of reinforcement learning. This can be seen as an estimate of the value of that memory. This can be seen as a reward signal after this use. Instead of writing the problem in a complete state and action space, and without training complex policy networks, the idea of ​​updating value based on reward in reinforcement learning is directly applied to the reward score. The solution has the core logic of reinforcement learning without making it too heavy or causing latency issues.

[0104] This embodiment provides a method for managing the long-term memory of an intelligent agent. Upon receiving a current task, the method extracts the current problem representation of the current task; based on the current problem representation, it performs multiple screenings in a long-term memory to obtain target memories. The long-term memory stores multiple long-term memories, each including an intent field, a process field, and a benefit score field; the current task and the target memories are combined and input into a large language model to obtain execution results; and the target memories are adjusted based on the execution results. This multiple screening mechanism effectively improves the utilization efficiency of the long-term memory and ensures that the selected target memories are highly relevant to the current task. Simultaneously, combined with the processing capabilities of the large language model, it can generate execution results that better meet actual needs, and continuously optimize the content of the target memories through a feedback mechanism. This enhances the agent's task execution capabilities, reduces interference from irrelevant memories, and improves the agent's adaptability and accuracy in complex scenarios. Furthermore, the dynamic adjustment of target memories further strengthens the self-updating and optimization capabilities of the long-term memory.

[0105] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S20 includes steps S201 to S204:

[0106] Step S201: Based on the current problem representation and the intent fields of each long memory in the long memory bank, perform semantic matching and filtering to obtain the semantic similarity between the current problem representation and the intent fields of each long memory.

[0107] It should be noted that the first stage of recall can be performed based on the current question representation, that is, query each long memory in the long memory, match the current question representation with the intent field of each long memory, and calculate the semantic similarity between the current question representation and the intent field of each long memory.

[0108] Step S202: Select long-term memories with semantic similarity greater than a preset similarity threshold as candidate memories to obtain a candidate memory set.

[0109] It should be noted that long-term memories with semantic similarity greater than a preset similarity threshold can be used as candidate memories to initially identify a batch of candidate memories that are semantically similar and broadly match the task type. This does not directly determine the final injected content, but only serves as a preliminary screening.

[0110] Candidate memory set It can be represented as:

[0111]

[0112] In the above formula, To indicate the current problem, This represents a problem related to long-term memory. For similarity threshold, For semantic similarity, This indicates whether the current task type and the memory label match, represented by a Boolean function: 1 for a match and 0 for a mismatch. In practice, the top K candidates can also be selected from this list.

[0113] Step S203: Calculate the comprehensive score of each candidate memory in the candidate memory set.

[0114] In practice, after the first screening is completed, the comprehensive score of all candidate memories in each candidate memory set can be calculated to conduct the second screening. The comprehensive score can be calculated based on the benefit score, time information, etc. of each candidate memory.

[0115] In one feasible implementation, step S203 may include steps B11-B12:

[0116] Step B11: Obtain the candidate semantic similarity, historical reward score, and time decay factor for each candidate memory;

[0117] It should be noted that candidate semantic similarity is the question similarity between candidate memory and the current question representation, historical benefit score is the benefit score of each candidate memory, and time decay factor is the time correction term, which is used to appropriately prioritize newer and still valid memories.

[0118] In one feasible implementation, the step of obtaining the time decay factor may include: obtaining usage data and the current time for each candidate memory; if the usage data indicates that the candidate memory has not been effectively used again since it was written into the long-term memory bank, taking the writing time of the candidate memory as the most recent effective timestamp; if the usage data indicates that the candidate memory has been effectively used since it was written into the long-term memory bank, taking the time of the most recent effective use of the candidate memory as the most recent effective timestamp; and obtaining the time decay factor for each candidate memory based on the time difference between the current time and the most recent effective timestamp.

[0119] In practical implementation, usage data for each candidate memory and the current time can be obtained. The usage data includes whether each candidate memory has been effectively used. If a candidate memory has not been effectively used since being written into the long-term memory, the writing time can be used as the most recent valid timestamp. If a candidate memory has been effectively used since being written into the long-term memory, the most recent valid usage time is used as the most recent valid timestamp. Then, the time decay factor is calculated based on the most recent valid timestamp and the current time, as shown below:

[0120]

[0121] In the above formula, K is the number of candidate memories. For the current time, The most recent valid timestamp, The time decay factor can be defined as (0, 1) in its original range.

[0122] Step B12: The candidate semantic similarity, the historical benefit score, and the time decay factor are weighted and summed to obtain the comprehensive score of each candidate memory.

[0123] Understandably, a weighted sum of candidate semantic similarity, historical gain score, and time decay factor can be used to obtain a comprehensive score for each candidate memory, as shown below:

[0124]

[0125] In the above formula, and Let be the coefficient, where Used to balance similarity and payout scores This is an additional coefficient for the time decay factor. This refers to the semantic similarity of candidate memories. This refers to the historical return score, i.e., the return score of the candidate factor. This is used for comprehensive scoring. By combining similarity and usefulness, and adding a small time adjustment factor, the possibility of old experiences misleading the current task is reduced. The second part selects the few memories with the highest scores and incorporates them into the context.

[0126] Step S204: Filter the candidate memories according to the comprehensive score to obtain the target memory.

[0127] In practice, the overall scores can be ranked, and the candidate memory with the highest overall score can be used as the target memory.

[0128] This approach doesn't simply retrieve and then concatenate; instead, it divides memory selection into two layers. The first layer addresses similarity, and the second layer determines whether to include the memory. This differs from typical long-term memory retrieval.

[0129] This embodiment effectively filters out memories highly relevant to the problem by semantically matching the intent field of the current problem with long-term memory, thus improving the accuracy of the filtering. The comprehensive scoring step evaluates candidate memories by combining semantic similarity, historical benefit scores, and time decay factors. It considers not only semantic similarity but also historical benefits and time decay factors, allowing for dynamic adjustment of the weight of each memory for refined selection based on actual needs. Through this multi-dimensional filtering and scoring mechanism, the system can more intelligently select the most valuable memories to respond to the current problem, improving the relevance and accuracy of the response and enhancing the system's adaptability to cope with changing needs in different contexts. The inclusion of a time decay factor considers the time-sensitivity of memories, ensuring that information in the long-term memory does not become irrelevant or ineffective over time. This helps the agent maintain high memory quality when facing long-term tasks.

[0130] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 After step S40, the method further includes steps S41 to S43:

[0131] Step S41: Obtain task execution information based on the execution result.

[0132] It should be noted that the execution result includes execution task information, namely whether the execution was successful or failed, as well as the specific details of the execution.

[0133] Step S42: Based on the task execution information, determine whether there are preset valid process conditions that are met during the task execution process using the target memory.

[0134] It should be noted that, based on the task execution information, it can be determined whether the preset valid process conditions are met when performing the task with the target memory, so that the valid process can be written as a new memory.

[0135] An effective process is not an arbitrary trajectory, but rather a trajectory segment that satisfies the reuse condition. Three types of effective processes can be defined:

[0136] 1. The task was successful, and the chain of steps is complete and reusable;

[0137] 2. Although the task failed, the reason for the failure is clear, the correction path is clear, and it can be reused in subsequent tasks;

[0138] 3. This process corresponds to a new task type or toolchain, and is novel.

[0139] Therefore, if any of the above situations exist during the task execution process using target memory, it can be determined that the preset valid process conditions are met. If not, the benefit score of target memory needs to be downweighted.

[0140] Step S43: If the conditions for a valid process are met during the execution of the task using the target memory, the execution process of the task using the target memory is written into the long-term memory bank as a new long-term memory.

[0141] In practice, if the conditions for a valid process are met, the valid process can be written back as a new memory. The initial score of the new memory can be directly given from the current result, for example: Alternatively, a more stable initial value can be used during implementation. The reason for this design is that the system not only corrects its memory but also continuously accumulates new experience, forming a truly long-term, self-growing memory bank.

[0142] The write-back process can be divided into four steps: collecting execution traces, removing irrelevant and duplicate content, extracting key steps and correction information, and writing back as new memory. A process validity determination function can be defined. ;when When =1, perform a write-back. A valid process can include: a successful task with a complete step chain; a failed task with a clear reason and a clear correction path; or a process corresponding to a new task type or toolchain. The write-back process includes: collecting the original trajectory. Remove duplicate output, invalid logs, and irrelevant content; extract key steps, tool sequences, error points, and correction methods to generate... Finally, Together Timestamps and task tags are written to the memory.

[0143] In one feasible implementation, this solution also includes a lightweight time and version mechanism, which allows for memory updates and iterations without causing long-term operational conflicts. Therefore, after step S40, the method further includes: obtaining the time interval between the last effective use of each long-term memory in the long-term memory bank, the historical version of each long-term memory, and the historical memory content of each long-term memory; when the time interval is greater than a preset time interval threshold, reducing the benefit score field of the long-term memory by a preset decay rate; when the historical memory content and the content to be written to the new memory have the same meaning but the historical version is lower than the version of the new memory to be written, reducing the priority of the long-term memory or reducing the benefit score field of the long-term memory; when the historical memory content and the content to be written to the new memory are complementary, merging the historical memory content into the content to be written to the new memory and reducing the priority of the long-term memory.

[0144] It should be noted that long-unused memories can be gradually downgraded in weight. Therefore, the time interval since the last effective use of each long-term memory, the historical versions of each long-term memory, and the historical content of the memory can be obtained. For time intervals greater than a preset time interval threshold, the reward score of the long-term memory can be reduced at a set instantaneous speed, as shown below:

[0145]

[0146] In the above formula, t represents the time interval since the last valid use. This indicates the preset decay rate. Its purpose is not to induce forced forgetting, but rather to allow old memories to gradually fade into the background, preventing them from occupying memory space for extended periods. The preset decay rate can be determined empirically or based on a desired half-life. By reverse deduction, it can be represented, for example, as follows: The advantage of this setup is that it's more intuitive: if the business requires a certain type of memory to have a short half-life... After it decays to half of its original value, the corresponding decay rate can be obtained directly.

[0147] Understandably, if the historical memory and the new memory to be written express the same fact, but the version of the new memory to be written is relatively higher, indicating that the version has been updated, then the older version is marked as low priority, or its score is directly reduced. For example, it can be written as:

[0148]

[0149] In the above formula, ρ is a preset coefficient used to reduce the benefit score of the old version of the memory, thereby ensuring that the new version of the memory has a higher priority in subsequent tasks. This approach is suitable for multi-turn dialogues and long-term tool environments, where many errors are not due to a lack of memory, but rather to the retention of the old version of the memory.

[0150] Furthermore, if the historical memory content and the content to be written into the new memory are complementary, they can be merged to form a more complete and comprehensive memory content. If the two memories are not version updates but complementary in content, only the newer main version can be retained, while the supplementary steps from the old memory are incorporated into the new process. This helps prevent the memory bank from becoming increasingly fragmented.

[0151] Adjusting target memories based on execution results not only dynamically adjusts memory priorities but also allows for merging or downgrading of memory content according to actual task requirements, thereby avoiding interference from memory redundancy and outdated information. Furthermore, by introducing a time decay factor and a historical version comparison mechanism, the level of precision in memory management is further enhanced, ensuring that the long-term memory bank remains efficient and orderly at all times.

[0152] For example, to help understand the implementation process of the agent long-term memory management method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 4 , Figure 4 A simplified flowchart of an intelligent agent long-term memory management method is provided. Specifically, the current task q is obtained, and then two-stage retrieval is performed on the current task. In the first stage, similarity recall is performed to obtain a candidate set. In the second stage, the candidate set is used to calculate a comprehensive score to obtain the target memory to be injected into the context. Then, the retrieved target memory is injected into the context to enable the large model to perform reasoning and obtain the task result. The reward score is updated according to the task result, and the intent, process and reward score are obtained and written according to the execution result.

[0153] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the intelligent agent long-term memory management method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0154] This application also provides a long-term memory management device for intelligent agents; please refer to [reference needed]. Figure 5 The intelligent agent long-term memory management device includes:

[0155] The extraction module 10 is used to extract the current problem representation of the current task when the current task is received.

[0156] The filtering module 20 is used to perform multiple filtering operations in the long-term memory based on the current problem representation to obtain the target memory. The long-term memory stores multiple long-term memories, each of which includes an intent field, a process field, and a benefit score field.

[0157] Input module 30 is used to input the current task and the target memory into the large language model to obtain the execution result.

[0158] The adjustment module 40 is used to adjust the target memory based on the execution result.

[0159] The intelligent agent long-term memory management device provided in this application, employing the intelligent agent long-term memory management method in the above embodiments, can solve the technical problems of low efficiency and difficulty in accurately matching current task requirements in current long-term memory management. Compared with the prior art, the beneficial effects of the intelligent agent long-term memory management device provided in this application are the same as those of the intelligent agent long-term memory management method provided in the above embodiments, and other technical features in the intelligent agent long-term memory management device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0160] This application provides an intelligent agent long-term memory management device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the intelligent agent long-term memory management method in the first embodiment described above.

[0161] The following is for reference. Figure 6 The diagram illustrates a structural schematic suitable for implementing the intelligent agent long-term memory management device in the embodiments of this application. The intelligent agent long-term memory management device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The illustrated intelligent agent long-term memory management device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0162] like Figure 6As shown, the agent long-term memory management device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the agent long-term memory management device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the agent long-term memory management device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows agent long-term memory management devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0163] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0164] The intelligent agent long-term memory management device provided in this application, employing the intelligent agent long-term memory management method in the above embodiments, can solve the technical problems of low efficiency and difficulty in accurately matching current task requirements in current long-term memory management. Compared with the prior art, the beneficial effects of the intelligent agent long-term memory management device provided in this application are the same as those of the intelligent agent long-term memory management method provided in the above embodiments, and other technical features in this intelligent agent long-term memory management device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0165] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0166] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0167] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the agent long-term memory management method in the above embodiments.

[0168] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0169] The aforementioned computer-readable storage medium may be included in the agent's long-term memory management device; or it may exist independently and not be assembled into the agent's long-term memory management device.

[0170] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the intelligent agent long-term memory management device, the intelligent agent long-term memory management device: upon receiving a current task, extracts the current problem representation of the current task; performs multiple filtering operations in a long-term memory bank based on the current problem representation to obtain a target memory, wherein the long-term memory bank stores multiple long-term memories, each long-term memory including an intent field, a process field, and a benefit score field; combines the current task and the target memory and inputs them into a large language model to obtain an execution result; and adjusts the target memory according to the execution result.

[0171] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0172] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0173] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0174] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the above-described intelligent agent long-term memory management method. This solves the technical problems of low efficiency and difficulty in accurately matching current task requirements in current long-term memory management. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the intelligent agent long-term memory management method provided in the above embodiments, and will not be elaborated upon here.

[0175] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the intelligent agent long-term memory management method described above.

[0176] The computer program product provided in this application can solve the technical problems of low efficiency and difficulty in accurately matching current task requirements in long-term memory management. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the intelligent agent long-term memory management method provided in the above embodiments, and will not be repeated here.

[0177] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for managing the long-term memory of an intelligent agent, characterized in that, The agent long-term memory management method includes: Upon receiving the current task, extract the current problem representation of the current task; Based on the current problem representation, multiple filters are performed in the long-term memory to obtain the target memory. The long-term memory stores multiple long-term memories, each of which includes an intent field, a process field, and a benefit score field. The current task and the target memory are combined and input into the large language model to obtain the execution result; The target memory is adjusted based on the execution results.

2. The method as described in claim 1, characterized in that, The step of performing multiple screenings in the long-term memory based on the current problem representation to obtain the target memory includes: Based on the semantic matching and filtering of the current problem representation and the intent fields of each long memory in the long memory, the semantic similarity between the current problem representation and the intent fields of each long memory is obtained. Long-term memories with semantic similarity greater than a preset similarity threshold are selected as candidate memories to obtain a candidate memory set; Calculate the overall score of each candidate memory in the candidate memory set; The candidate memories are filtered based on the comprehensive score to obtain the target memories.

3. The method as described in claim 2, characterized in that, The steps for calculating the overall score of each candidate memory in the candidate memory set include: Obtain the candidate semantic similarity, historical score, and time decay factor for each candidate memory; The candidate semantic similarity, the historical benefit score, and the time decay factor are weighted and summed to obtain the comprehensive score for each candidate memory.

4. The method as described in claim 2, characterized in that, The steps to obtain the time decay factor for each candidate memory include: Obtain usage data and current time for each candidate memory; If the usage data is that the candidate memory has not been effectively used again after being written into the long-term memory bank, the writing time of the candidate memory shall be used as the most recent effective timestamp. In the case where the data used is the candidate memory being used effectively after it has been written into the long-term memory bank, the time when the candidate memory was last used effectively is taken as the most recent effective timestamp. The time decay factor of each candidate memory is obtained based on the time difference between the current time and the most recent valid timestamp.

5. The method as described in claim 1, characterized in that, The step of adjusting the target memory based on the execution result includes: Based on the execution results, task execution information, invocation cost, and execution feedback are obtained. The scalar reward value is determined based on the task execution information, the invocation cost, and the execution feedback; Based on the scalar reward value, the revenue field of the target memory that has been used is dynamically updated.

6. The method as described in claim 1, characterized in that, After the step of adjusting the target memory based on the execution result, the method further includes: The task execution information is obtained based on the execution results. Based on the task execution information, determine whether there are preset valid process conditions during the task execution process using the target memory; If the conditions for a valid process are met during the execution of a task using the target memory, the execution process using the target memory is written into the long-term memory bank as a new long-term memory.

7. The method as described in claim 1, characterized in that, The method further includes: Retrieve the time interval between the last effective use of each long-term memory in the long-term memory bank, the historical versions of each long-term memory, and the historical memory content of each long-term memory; When the time interval is greater than a preset time interval threshold, the benefit field of the long-term memory is reduced by a preset decay rate; When the content of the historical memory has the same meaning as the content of the new memory to be written, but the historical version is lower than the version of the new memory to be written, the priority of the long-term memory is reduced or the benefit score field of the long-term memory is reduced. When the historical memory content is complementary to the content to be written into the new memory, the historical memory content is incorporated into the content to be written into the new memory, and the priority of the long-term memory is reduced.

8. The method as described in claim 1, characterized in that, Before the step of performing multiple siftings in the long-term memory based on the current problem representation to obtain the target memory, the method further includes: Retrieve the intent field, process field, revenue field, version tag, and task information from long-term memory; Construct triplet information for each long-term memory based on the intent field, the process field, and the revenue distribution field; The triplet information is stored in the long-term memory, along with the version tag, task information, and write time of each long-term memory.

9. A long-term memory management device for an intelligent agent, characterized in that, The device includes: An extraction module is used to extract the current problem representation of the current task when the current task is received; The filtering module is used to perform multiple filtering operations in the long-term memory based on the current problem representation to obtain the target memory. The long-term memory stores multiple long-term memories, and each long-term memory includes an intent field, a process field, and a benefit score field. The input module is used to input the current task and the target memory into the large language model to obtain the execution result; An adjustment module is used to adjust the target memory based on the execution result.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the intelligent agent long-term memory management method as described in any one of claims 1 to 8.

Images