Intelligent agent memory compression method based on large model attention mechanism

Abstract

The invention relates to an agent memory compression method based on a large-model attention mechanism, and the method comprises the following steps: obtaining the attention weight of each memory in a historical memory sequence for a current conversation, and calculating the correlation degree of the attention weight; determining a demarcation point for distinguishing the early content from the recent content from the historical memory sequence based on the correlation degree and the system constraint condition; dividing the historical memory into an early memory section and a recent memory section by the demarcation point C; performing semantic extraction on the early memory segment to generate summary information, and replacing the original early memory segment with the summary information; and combining the abstracted early memory segment with the complete recent memory segment to form compressed memory input required by the next round of model reasoning. According to the dynamic memory management method based on early-stage content abstract extraction and recent content complete retention, the demarcation point of the early-stage content and the recent content is automatically determined through an attention weight calculation mode, and controllable compression and effective fidelity of the memory content are achieved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an agent memory compression method based on a large model attention mechanism. Background Technology

[0002] With the rapid improvement of Large Language Model (LLM) capabilities, agent systems built on LLM are widely used in complex task scenarios. These systems typically need to complete tasks through multi-turn dialogues and continuous tool calls. Therefore, they need to continuously maintain the memory of historical dialogue content during the reasoning process to ensure the continuity of the task chain and the understanding of long-term context.

[0003] However, in practical applications, the historical dialogue content of an agent accumulates rapidly with each interaction round, consuming a large number of tokens. On the one hand, excessive tokens significantly reduce the model's inference efficiency and response speed; on the other hand, LLM is limited by a fixed context window length, and when historical content exceeds the window limit, the excess portion must be discarded. Common processing methods in existing technologies mainly have the following shortcomings: directly truncating early historical content is simple, but it easily leads to the loss of key information and disrupts the continuity of task execution; global summary compression of the entire historical dialogue, while reducing token consumption, may lose important details in recent dialogues, and frequent rewriting of summaries may introduce inconsistencies.

[0004] For example, Chinese patent CN120688545A discloses a method for modeling long-term memory of intelligent agents based on memory networks. It employs a dual storage architecture of vector and relational databases, combined with an emotional temporal association graph for memory retrieval, aiming to improve the coherence of emotional interactions. This method focuses on the construction of the external memory bank and complex association retrieval, failing to directly address the token compression problem of the internal context window during LLM inference, and its graph construction and community detection algorithms incur significant computational overhead.

[0005] For example, Chinese patent CN119903125A discloses a memory retrieval method based on a large language model, which manages memory through the division of short-term memory modules and long-term memory modules and dynamic information transfer. This method also focuses on the storage and retrieval mechanism of memory, rather than dynamically and adaptively compressing the historical information already loaded into the context window, and cannot directly solve the problem of degraded reasoning performance caused by the expansion of the number of tokens.

[0006] Furthermore, Chinese Patent CN118228842A discloses a data processing method that dynamically loads some parameters of a neural network based on the information entropy of the input data to reduce bandwidth requirements. This method is applicable to the loading process of model network parameters, and its technical field and the problem it addresses (bandwidth pressure during model deployment) differ significantly from the present invention (content management and compression of agent dialogue history), as it does not involve semantic extraction and segmentation management of serialized historical content.

[0007] In summary, current technologies for long-dialogue memory management in intelligent agents lack a dynamic memory compression mechanism that can efficiently control the length of input tokens while maintaining context validity and keeping computational overhead manageable. Achieving a real-time, adaptive balance between "retaining key information" and "reducing token consumption" has become a problem that needs to be solved in intelligent agent design. Summary of the Invention

[0008] In view of the above-mentioned technical problems, this invention proposes an agent memory compression method based on a large model attention mechanism. It is based on a dynamic memory management method of "early content summary extraction + recent content complete retention" and automatically determines the boundary point between early and recent content through attention weight calculation, so as to achieve controllable compression and effective fidelity preservation of memory content.

[0009] The technical solution to the technical problem of this invention is: a method for agent memory compression based on a large model attention mechanism, comprising the following steps:

[0010] Step S1: Obtain the attention weight of each memory in the historical memory sequence to the current dialogue, and calculate its correlation.

[0011] Step S2: Based on the correlation degree and system constraints, determine the dividing point from the historical memory sequence to distinguish between early and recent content; the dividing point C divides the historical memory into an early memory segment with relatively low correlation degree and a recent memory segment with relatively high correlation degree.

[0012] Step S3: Semantically extract the early memory segment to generate summary information, and replace the original early memory segment with the summary information;

[0013] Step S4: Merge the summarized early memory segments with the complete recent memory segments to form the compressed memory input required for the next round of model inference.

[0014] According to one technical solution of the present invention, in step S1, calculating the correlation between each historical memory and the current conversation specifically includes:

[0015] Step S11: Call the attention calculation module of the large language model in response to the most recent round of dialogue to obtain the attention weight matrix between the output token and the historical input token;

[0016] Step S12: For each memory in the historical memory sequence, aggregate the attention weights between all input tokens and output tokens contained therein to obtain the initial attention contribution value of that memory;

[0017] Step S13: Normalize the initial attention contribution value to obtain the relevance of each memory to the current dialogue.

[0018] According to one technical solution of the present invention, in step S13, the normalization process includes:

[0019] Step S131: Perform first-level normalization based on the number of tokens in each memory;

[0020] Step S132: Perform a second-level normalization on all memories using the softmax function to obtain the relative correlation distribution.

[0021] According to one technical solution of the present invention, step S2 specifically includes:

[0022] Step S21: Sort all N historical memories from oldest to most recent in time as follows: , ,…, The corresponding correlation is , ,…, };

[0023] Step S22, from the most recent memory Start moving forward, calculating the cumulative sum of correlations. , is represented as:

[0024]

[0025] Among them, From Indexes decreasing to 1;

[0026] Step S23: Based on the preset cumulative correlation threshold θ, find the smallest index C that satisfies the condition, such that... And for any All have ;

[0027] Step S24, memorize The boundary point is determined as follows: memories indexed 1 to C-1 belong to the early memory segment, and memories indexed C to N belong to the recent memory segment.

[0028] According to one technical solution of the present invention, before step S21, the method further includes:

[0029] Step S20: Determine whether the cumulative number of tokens in the historical memory sequence exceeds the preset trigger compression threshold. ;

[0030] If yes, proceed with steps S21-S24; otherwise, do not perform compression.

[0031] According to a technical solution of the present invention, the preset trigger compression threshold Based on the memory system's maximum allowed token quantity limit And the token budget reserved for storing early memory summaries Confirmed, expressed as: .

[0032] According to one technical solution of the present invention, in step S4, the cumulative correlation threshold θ ranges from 0.9 to 0.95.

[0033] According to one technical solution of the present invention, step S4 specifically includes:

[0034] Step S41: Construct a prompt word containing the summary generation instruction, summary length requirement, and early memory segment content;

[0035] Step S42: Input the prompt words into a large language model to guide it in generating a structured summary that covers the core semantics of early memory segments.

[0036] According to one technical solution of the present invention, in step S41, constructing a prompt word containing a summary generation instruction, a summary length requirement, and early memory segment content includes at least one of the following operations: retaining core facts and task objectives, deleting irrelevant details and duplicate content, using concise and objective language for description, and controlling the length of the output text within a predetermined range.

[0037] According to one aspect of the present invention, an agent memory compression system based on a large model attention mechanism is proposed, comprising:

[0038] The memory management module is used to store and manage historical conversation records of interactions with users;

[0039] The attention analysis module is used to obtain the attention weight of each memory in the historical memory sequence to the current dialogue and to calculate its correlation.

[0040] The attention analysis module is also used to determine the boundary point from the historical memory sequence that distinguishes between early and recent content based on the correlation degree and system constraints.

[0041] The summary generation module is used to semantically extract the early memory segments, generate summary information, and replace the original early memory segments with the summary information;

[0042] The memory integration module is used to merge the summarized early memory segments with the complete recent memory segments to form the compressed memory input required for the next round of model inference.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] This invention provides an agent memory compression method based on a large model attention mechanism. Through a hybrid strategy of "early summarization + recent complete retention", it achieves dynamic and controllable compression of token usage. It can significantly reduce the number of tokens occupied by historical dialogues in the context window without losing recent key details, which helps to alleviate the problems of declining model inference performance and breaking the context window limit.

[0045] This invention utilizes the attention weights generated by LLM itself when generating responses to quantify the importance of historical memories. The compression decision is based on the model's internal attention mechanism, which is more accurate and makes the boundary between early and recent memories reliable. Compared with methods based on fixed rules or simple time decay, it can more accurately identify low-association memories that can be compressed and improve the fidelity of the compressed context.

[0046] This invention performs summary compression only on earlier content with low relevance, ensuring that the integrity and detail of recent information most relevant to the current task are not destroyed, thus maintaining the logical coherence of the dialogue. At the same time, the summary processing preserves the macro-level semantics of earlier content, avoiding the loss of key information that might result from direct truncation, thus achieving a balance between compression efficiency and information preservation.

[0047] This invention does not rely on complex rule systems or additional training models. It directly utilizes the attention output of existing LLMs to achieve a clear process with relatively low computational overhead. It is suitable for various LLM agent systems that require long-term, multi-round interactions, providing them with an efficient and reliable memory management solution. Attached Figure Description

[0048] Figure 1 This diagram illustrates a flowchart of an agent memory compression method based on a large model attention mechanism in one embodiment of the present invention.

[0049] Figure 2 This illustration represents a schematic example of memory management based on a summary extraction method in one embodiment of the present invention;

[0050] Figure 3The flowchart illustrates a method for calculating the attention weight of each memory in the current dialogue in one embodiment of the present invention. Detailed Implementation

[0051] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0052] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The embodiments cannot be described in detail here, but the embodiments of the present invention are not limited to the following embodiments.

[0053] like Figures 1 to 3 As shown, this invention presents an agent memory compression method based on a large model attention mechanism. Through a dynamic memory mechanism of "early memory summarization processing + complete retention of recent content," it significantly reduces the token occupancy of historical content while maintaining contextual validity. Compared to direct truncation methods, this invention retains the core semantics while compressing early information, avoiding the deletion of important context. Compared to pure summarization strategies, this invention only extracts early content, ensuring that the details of the latest information are not compressed and reducing inconsistencies caused by frequent rewriting. Compared to importance scoring and semantic clustering methods, this invention does not rely on complex rules or high-cost computation; it achieves long-term retention of key information and natural filtering of redundant content through structured segmentation management. This scheme achieves a better balance between memory fidelity, computational overhead, and inference efficiency, making it more suitable for agent systems requiring long-term interaction.

[0054] like Figure 1 As shown, the present invention provides an agent memory compression method based on a large model attention mechanism, which mainly includes the following steps:

[0055] Step S1: Obtain the attention weight of each memory in the historical memory sequence to the current dialogue, and calculate its correlation.

[0056] After the agent completes the latest round of dialogue response, the system obtains the attention weight matrix output by the last layer of the LLM attention module during the response generation process. This matrix reflects the degree of attention each output token receives from all historical input tokens (corresponding to historical dialogue memories). The system groups the input tokens according to their respective historical dialogue rounds (i.e., "memory bars"). For each historical memory, it calculates the aggregate value (e.g., summation) of the attention weights corresponding to all input tokens contained within it, and performs normalization processing (e.g., first dividing by the number of tokens in that memory, then normalizing across all memories using the softmax function). Finally, it obtains the "relevance" of each historical memory to the current latest dialogue. Relevance is a value between 0 and 1; a higher value indicates that the memory is referenced more when generating the current response, and the stronger its relevance to the current context.

[0057] Step S2: Based on the correlation degree and system constraints, determine the dividing point from the historical memory sequence to distinguish between early and recent content; the dividing point C divides the historical memory into an early memory segment with relatively low correlation degree and a recent memory segment with relatively high correlation degree.

[0058] The system presets two key parameters: maximum capacity of the context window. (Number of Tokens) and the budget reserved for storing digests The system continuously calculates the total number of tokens for all current historical memories. If the total number of tokens does not exceed ( If the time limit is exceeded, compression will not be triggered, and the next round of dialogue will continue. If the time limit is exceeded, the compression process will be triggered. After entering the compression process, the system sorts all historical memories in order of time from oldest to newest as { , ,…, The corresponding correlation is , ,…, From recent memories Start by accumulating the correlation degree forward, expressed by the formula: Meanwhile, the system presets a cumulative correlation threshold θ (e.g., 0.9 or 0.95). The decision objective is to find a minimum index C such that the cumulative correlation... And for all All have Then the memory will be... Defined as the dividing point. Memories from index 1 to C-1 are divided into "early memory segments", and memories from index C to N are divided into "recent memory segments".

[0059] Step S3: Semantically extract the early memory segment to generate summary information, and replace the original early memory segment with the summary information;

[0060] For the segmented early memory segments (i.e. to The system takes all its original text content as input and constructs a summary generation prompt. This prompt contains explicit instructions requiring the LLM to semantically extract key information from the early dialogue content, generating a concise summary that covers core facts, task objectives, user preferences, and removes irrelevant details and repetitive content, while controlling the summary length. Within budget. The system then invokes LLM to generate a summary text based on this prompt.

[0061] Step S4: Merge the summarized early memory segments with the complete recent memory segments to form the compressed memory input required for the next round of model inference.

[0062] The system uses the structured summary text generated in step S3 to completely replace the early memory segments in the original history. to Then, compare this summary with the unmodified, complete recent memory segment (). to The tokens are pieced together chronologically to form a new, compressed historical memory input. The total number of tokens in this input is effectively controlled and used for the agent's next round of reasoning.

[0063] This invention achieves automated and intelligent compression of the historical memory of an intelligent agent. When the redundancy of historical information is high, the system tends to compress more early content; when all historical information is strongly correlated with the present, the system will reduce the compression ratio, thereby achieving a dynamic balance between compression rate and information retention.

[0064] In some embodiments of the present invention, step S1, which involves calculating the correlation between each historical memory and the current conversation, specifically includes:

[0065] Step S11: Call the attention calculation module of the large language model in response to the most recent round of dialogue to obtain the attention weight matrix between the output token and the historical input token;

[0066] Suppose in the most recent dialogue, the user input was a piece of text, and the LLM generated a response based on this. During the inference process of the LLM (typically a decoder-only Transformer architecture), the last decoding layer computes the attention distribution of each newly generated output token with respect to all previous input tokens. This distribution is defined by the attention weight matrix. This indicates that for the attention weight matrix any one of the elements Represented as:

[0067]

[0068] in, This indicates that the token number will be output. This indicates that you need to input a token number. and These represent the query and the key vector, respectively. This represents the dimension of the key vector. The system extracts this weight matrix from the model after this inference iteration. .

[0069] Step S12: For each memory in the historical memory sequence, aggregate the attention weights between all input tokens and output tokens contained therein to obtain the initial attention contribution value of that memory;

[0070] Historical dialogues are typically organized as a series of records (memories) ordered chronologically. For the m-th memory, it contains a set of indices T(m) of input tokens. The overall attention contribution of this memory to the output of the current dialogue round is then determined. It can be calculated using the following formula:

[0071]

[0072] Where O represents the set of indices of all output tokens in the most recent round of dialogue, and T(m) represents the set of indices of all input tokens in memory entry m. This represents the attention weight of the output token i on the input token j.

[0073] The larger the value, the more fully the memory was involved in generating the current response and the higher its semantic relevance.

[0074] Step S13: Normalize the initial attention contribution value to obtain the relevance of each memory to the current dialogue.

[0075] To eliminate the impact of differences in the number of different message tokens, firstly, normalize the attention and token count as described above:

[0076]

[0077] in, The normalized weight representing each memory. This represents the number of tokens for the m-th memory.

[0078] Then all memories were proportionally... Normalization, expressed as:

[0079]

[0080] in, Represents all memories Normalized weights Represents the set of all memory entries. This serves as the index for traversing memory entries.

[0081] The final result This is the normalized correlation degree of each memory to the current conversation, and the sum of the correlation degrees of all memories is 1.

[0082] By utilizing the model's internal attention mechanism, this method can accurately capture the semantic relevance of each historical memory in the current context. Compared to methods based on keyword matching or simple time decay, this method better reflects the actual information utilization during the LLM inference process, thus providing a more reliable and fine-grained basis for subsequent compression boundary decisions and helping to improve the semantic coherence of the compressed context.

[0083] In some embodiments of the present invention, step S2 specifically includes:

[0084] Step S20: Determine whether the cumulative number of tokens in the historical memory sequence exceeds the preset trigger compression threshold. ;

[0085] If yes, proceed with steps S21-S24; otherwise, do not perform compression.

[0086] The system maintains a maximum token capacity. (For example, to Set to 8192) and summary budget (For example, to Set to 512). Calculate the total number of tokens T in the current historical memory sequence. If T ≤ ( If the current total memory count is not yet at the critical point requiring compression, the system skips the compression process and retains all memory for the next round. Otherwise, the subsequent boundary point search step is executed.

[0087] Step S21: Sort all N historical memories from oldest to most recent in time as follows: , ,…, The corresponding correlation is , ,…, };

[0088] The above information is obtained through step S1.

[0089] Step S22, from the most recent memory Start moving forward, calculating the cumulative sum of correlations. , is represented as:

[0090]

[0091] Among them, From Indexes decreasing to 1;

[0092] From the latest memory Begin by calculating the cumulative sum of associations backwards (towards earlier memories). Define the cumulative sum. This represents the sum of the associations from the k-th memory to the N-th memory. Clearly, , And so on.

[0093] Step S23: Based on the preset cumulative correlation threshold θ, find the smallest index C that satisfies the condition, such that... And for any All have ;

[0094] Step S24, memorize The boundary point is determined as follows: memories indexed 1 to C-1 belong to the early memory segment, and memories indexed C to N belong to the recent memory segment.

[0095] A threshold θ is preset, representing the minimum proportion of relevance that the "important memories" should contain, typically set to 0.90 or 0.95, indicating the retention of 90% or 95% of the information weight. The search objective is to find the smallest index C such that the cumulative relevance... And for all indexes smaller than C. (i.e., earlier memories), their cumulative association This means that from memory It begins to accumulate forward, its relevance ratio just below or reaching the allowable compression ratio of "secondary information" (1-θ). Therefore, it will... Defined as the dividing point. The memory after dividing point C ( arrive The segments ) are classified as "recent memory segments," which have high relevance and are preserved intact; memories before the dividing point C ( arrive The segments} are classified as “early memory segments”, which have relatively low relevance and will be summarized.

[0096] By setting a threshold θ, the degree of compression can be intuitively controlled: the higher the θ value, the greater the information weight required for the retained recent memory segments, the later the dividing point C will be (i.e., fewer memories will be classified as early segments), the smaller the compression ratio, and the more complete the information retention; conversely, the lower the θ value, the greater the compression ratio. This weight-based dynamic partitioning, compared to the static strategy of fixedly retaining the most recent N memories, can better adapt to the information density distribution of different dialogue content, achieving more refined memory management.

[0097] In some embodiments of the present invention, step S4 specifically includes:

[0098] Step S41: Construct a prompt word containing the summary generation instruction, summary length requirement, and early memory segment content;

[0099] Constructing a cue word that includes summary generation instructions, summary length requirements, and early memory segment content includes at least one of the following operations: retaining core facts and task objectives, deleting irrelevant details and duplicate content, using concise and objective language for description, and controlling the length of the output text within a predetermined range.

[0100] Let's take a specific prompt word as an example:

[0101] "The following are multiple historical conversations (early messages). Please generate a concise, complete, and unique summary based on all the content, with the following requirements:"

[0102] 1. Retain the core facts, mission objectives, user preferences, or long-term status information;

[0103] 2. Remove details, small talk, or repetitive content that are irrelevant to the current task;

[0104] 3. Use objective and concise language to describe the meaning without changing the original intent;

[0105] 4. The output must be a highly compressed summary text, without any speculation, fictional content or additional explanations.

[0106] 5. The output summary content is stored in {T summary Within the token limit. Please output the digest directly:

[0107] [Early summary content] [Input early dialogue content]

[0108] Step S42: Input the prompt words into a large language model to guide it in generating a structured summary that covers the core semantics of early memory segments.

[0109] The constructed cue words are input into an LLM (which can be the same model used for reasoning or a dedicated summarization model). The LLM then analyzes, understands, and compresses the input early memory text according to the instructions, generating a summary text that meets the requirements.

[0110] In step S4 of this invention, the summarized earlier content is merged with the unmodified recent content to form the memory input used in the next round of multi-turn dialogue reasoning. This input retains the key information required for LLM reasoning while avoiding token inflation caused by the infinite accumulation of historical information, thus achieving a balance between memory compression and semantic fidelity.

[0111] According to one aspect of the present invention, an agent memory compression system based on a large model attention mechanism is proposed, comprising:

[0112] The memory management module is used to store and manage historical conversation records of interactions with users;

[0113] The attention analysis module is used to obtain the attention weight of each memory in the historical memory sequence to the current dialogue and to calculate its correlation.

[0114] The attention analysis module is also used to determine the boundary point from the historical memory sequence that distinguishes between early and recent content based on the correlation degree and system constraints.

[0115] The summary generation module is used to semantically extract the early memory segments, generate summary information, and replace the original early memory segments with the summary information;

[0116] The memory integration module is used to merge the summarized early memory segments with the complete recent memory segments to form the compressed memory input required for the next round of model inference.

[0117] According to one aspect of the present invention, an electronic device is provided, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory; when the electronic device is running, the processor executes the one or more computer programs stored in the memory to cause the electronic device to perform an agent memory compression method based on a large model attention mechanism as described in any of the above technical solutions.

[0118] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0119] The memory can be an internal storage unit of the terminal device, such as a hard drive or RAM. Alternatively, it can be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal and external storage units. The memory is used to store the computer program and other programs and data required by the terminal device. It can also be used to temporarily store data that has been output or will be output.

[0120] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement an agent memory compression method based on a large model attention mechanism as described in any of the above technical solutions.

[0121] For example, computer-readable storage media can be read-only memory (ROM), random access memory (RAM), read-only optical disc (CD-ROM), magnetic tape, floppy disk, and optical data storage devices. They can be implemented using computer-executable program code, thus allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, this invention is not limited to any particular hardware and software combination.

[0122] In summary, this invention proposes a memory compression method for intelligent agents based on a large model attention mechanism. It aims to address the problems of excessive token counts, impaired inference performance, and limitations in overcoming context window constraints caused by the accumulation of historical memories in long dialogue scenarios using large language models. Traditional methods, such as direct truncation, easily lose key information, while simple summarization may lose details. This invention innovatively proposes a dynamic hybrid management strategy of "extracting summaries from early memories + fully preserving recent content." By intelligently defining the boundaries between early and recent memories, it effectively compresses the number of tokens while maintaining contextual coherence and the integrity of key information. The core of this method lies in the boundary point determination scheme: by analyzing attention weights and quantifying the contribution of each memory to recent dialogue, the boundary point is accurately located. This invention combines versatility and practicality, providing an efficient and reliable memory management solution for intelligent agents in long-cycle, multi-turn interaction tasks.

[0123] Furthermore, it should be noted that the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0124] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0126] It should also be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0127] Finally, it should be noted that the above description represents a preferred embodiment of the present invention. It should be pointed out that although preferred embodiments have been described, those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles described herein. These improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A method for dynamic compression of agent memory based on the attention mechanism of a large language model, characterized in that, Includes the following steps: Step S1: Obtain the attention weight of each memory in the historical memory sequence to the current dialogue, and calculate its correlation. Step S2: Based on the correlation degree and system constraints, determine the dividing point from the historical memory sequence to distinguish between early and recent content; the dividing point C divides the historical memory into an early memory segment with relatively low correlation degree and a recent memory segment with relatively high correlation degree. Step S3: Semantically extract the early memory segment to generate summary information, and replace the original early memory segment with the summary information; Step S4: Merge the summarized early memory segments with the complete recent memory segments to form the compressed memory input required for the next round of model inference.

2. The method according to claim 1, characterized in that, In step S1, calculating the correlation between each historical memory and the current conversation specifically includes: Step S11: Call the attention calculation module of the large language model in response to the most recent round of dialogue to obtain the attention weight matrix between the output token and the historical input token; Step S12: For each memory in the historical memory sequence, aggregate the attention weights between all input tokens and output tokens contained therein to obtain the initial attention contribution value of that memory; Step S13: Normalize the initial attention contribution value to obtain the relevance of each memory to the current dialogue.

3. The method according to claim 2, characterized in that, In step S13, the normalization process includes: Step S131: Perform first-level normalization based on the number of tokens in each memory; Step S132: Perform a second-level normalization on all memories using the softmax function to obtain the relative correlation distribution.

4. The method according to claim 1, characterized in that, Step S2 specifically includes: Step S21: Sort all N historical memories from oldest to most recent in time as follows: , ,…, The corresponding correlation is , ,…, }; Step S22, from the most recent memory Start moving forward, calculating the cumulative sum of correlations. , is represented as: Among them, From Indexes decreasing to 1; Step S23: Based on the preset cumulative correlation threshold θ, find the smallest index C that satisfies the condition, such that... And for any All have ; Step S24, memorize The boundary point is determined as follows: memories indexed 1 to C-1 belong to the early memory segment, and memories indexed C to N belong to the recent memory segment.

5. The method according to claim 4, characterized in that, Before step S21, the method further includes: Step S20: Determine whether the cumulative number of tokens in the historical memory sequence exceeds the preset trigger compression threshold. ; If yes, proceed with steps S21-S24; otherwise, do not perform compression.

6. The method according to claim 5, characterized in that, The preset trigger compression threshold Based on the memory system's maximum allowed token quantity limit And the token budget reserved for storing early memory summaries Confirmed, expressed as: .

7. The method according to claim 6, characterized in that, In step S4, the cumulative correlation threshold θ ranges from 0.9 to 0.

95.

8. The method according to claim 1, characterized in that, Step S4 specifically includes: Step S41: Construct a prompt word containing the summary generation instruction, summary length requirement, and early memory segment content; Step S42: Input the prompt words into a large language model to guide it in generating a structured summary that covers the core semantics of early memory segments.

9. The method according to claim 8, characterized in that, In step S41, constructing a prompt containing summary generation instructions, summary length requirements, and early memory segment content includes at least one of the following operations: retaining core facts and task objectives, deleting irrelevant details and duplicate content, using concise and objective language for description, and controlling the length of the output text within a predetermined range.

10. An agent memory compression system based on a large model attention mechanism, characterized in that, include: The memory management module is used to store and manage historical conversation records of interactions with users; The attention analysis module is used to obtain the attention weight of each memory in the historical memory sequence to the current dialogue and to calculate its correlation. The attention analysis module is also used to determine the boundary point from the historical memory sequence that distinguishes between early and recent content based on the correlation degree and system constraints. The summary generation module is used to semantically extract the early memory segments, generate summary information, and replace the original early memory segments with the summary information; The memory integration module is used to merge the summarized early memory segments with the complete recent memory segments to form the compressed memory input required for the next round of model inference.

Images