Data interaction system, agent and agent memory processing method

Abstract

This application discloses a data interaction system, an intelligent agent, and an intelligent agent memory processing method, relating to the fields of artificial intelligence and data storage and management technology. The method includes: an intelligent agent receiving a dialogue request carrying a first experience identifier, the dialogue request being used to request the reuse of a key-value cache associated with the first experience identifier to perform a first inference task; retrieving implicit memories associated with the first experience identifier from a first implicit memory bank, where one implicit memory is associated with one experience identifier; in response to the retrieval of the first implicit memory, decompressing the compressed key-value cache in the first implicit memory into a first key-value cache, and performing the first inference task based on the first key-value cache. Through this application, the key-value cache is reused as a long-term asset, which can solve the technical problems in related technologies where the key-value cache is released upon the end of the session, and existing experience is stored only in plaintext, leading to repeated calculations of similar tasks and wasted computing power.

Description

Technical Field

[0001] This application relates to the field of electronic device technology, and in particular to data interaction systems, intelligent agents, and methods for processing memory in intelligent agents. Background Technology

[0002] During the inference process of the large language model, the system stores the key and value matrix in the attention mechanism through a key-value cache (KVCache) to avoid repeated calculations of historical inputs.

[0003] However, in existing technical architectures, key-value caches are primarily stored in the agent's GPU memory or high-speed memory. Their lifecycle is typically limited to a single session or short-term temporary storage, and they are released once the session ends. They are not managed persistently as long-term reusable experience assets. Furthermore, related technologies store user experience and domain knowledge only in plaintext natural language. Each time a task is executed, the plaintext must be appended to the user's query, and the key-value cache must be re-encoded using a large model. This repetitive computation of similar tasks results in a significant waste of computing power. Summary of the Invention

[0004] This application provides a data interaction system, an intelligent agent, and an intelligent agent memory processing method to at least solve the problems in related technologies, such as the repeated calculation of similar tasks and waste of computing power caused by the key-value cache being released when the session ends and existing experience being stored only in plaintext.

[0005] This application provides a data interaction system, including an agent and a client. The agent includes an inference engine and a first implicit memory. An implicit memory in the first implicit memory contains a compressed key-value cache obtained by compressing a key-value cache corresponding to a historical experience. An implicit memory in the first implicit memory is associated with an experience identifier. The client, during a first session, is configured to, in response to receiving an input query, retrieve a historical experience identifier matching the input query; and, in response to retrieving the first experience identifier, send a dialogue request carrying the first experience identifier to the agent, wherein the dialogue request requests the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task. The inference engine, in response to receiving the dialogue request, retrieves the implicit memory associated with the first experience identifier from the first implicit memory; and, in response to retrieving the first implicit memory, decompresses the compressed key-value cache in the first implicit memory into a first key-value cache, and performs the first inference task based on the first key-value cache.

[0006] This application also provides an intelligent agent, including: an inference engine and a first implicit memory bank, wherein an implicit memory in the first implicit memory bank contains a compressed key-value cache obtained by compressing a key-value cache corresponding to a historical experience, and an implicit memory in the first implicit memory bank is associated with an experience identifier; the inference engine is configured to, in response to receiving a dialogue request, retrieve the implicit memory associated with the first experience identifier carried in the dialogue request from the first implicit memory bank; the dialogue request is configured to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task; in response to retrieving the first implicit memory, decompress the compressed key-value cache in the first implicit memory into a first key-value cache, and perform the first inference task based on the first key-value cache.

[0007] This application also provides a client, which, during a first session, in response to receiving an input query statement, retrieves a historical experience identifier that matches the input query statement; and in response to retrieving the first experience identifier, sends a dialogue request carrying the first experience identifier to the intelligent agent, wherein the dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task.

[0008] This application also provides a method for processing intelligent agent memory, comprising: in response to receiving a dialogue request carrying a first experience identifier, retrieving an implicit memory associated with the first experience identifier carried in the dialogue request from a first implicit memory bank, wherein an implicit memory in the first implicit memory bank includes a compressed key-value cache obtained by compressing a key-value cache corresponding to a historical experience, and an implicit memory in the first implicit memory bank is associated with an experience identifier; the dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task; in response to retrieving the first implicit memory, decompressing the compressed key-value cache in the first implicit memory into a first key-value cache, and performing the first inference task based on the first key-value cache.

[0009] This application also provides an intelligent agent memory processing method, including: during a first session, in response to receiving an input query statement, retrieving a historical experience identifier that matches the input query statement;

[0010] In response to the retrieval of a first experience identifier, a dialogue request carrying the first experience identifier is sent to the agent, wherein the dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task.

[0011] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described intelligent agent memory processing methods when executing the computer program.

[0012] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described intelligent agent memory processing methods.

[0013] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described intelligent agent memory processing methods.

[0014] This application receives a dialogue request carrying a first experience identifier, extracts the first experience identifier from the dialogue request, retrieves the associated first implicit memory from the first implicit memory bank based on the first experience identifier, decompresses the compressed key-value cache in the first implicit memory into a first key-value cache, and directly executes the first inference task based on the first key-value cache. This replaces the original need to re-execute the complete forward computation with the direct loading and reuse of the pre-generated key-value cache, solving the technical problem of repeated computation of similar tasks and wasted computing power due to the key-value cache being released when the session ends and existing experience being stored only in plaintext. This improves the accuracy of task execution and achieves the technical effect of reducing computing power consumption. Attached Figure Description

[0015] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This application provides an illustration of an intelligent agent memory processing method.

[0017] Figure 2 An interaction diagram between a server and a client is provided for an embodiment of this application;

[0018] Figure 3 An architecture diagram of an edge-cloud collaborative intelligent agent memory management system provided in an embodiment of this application;

[0019] Figure 4 A flowchart illustrating a multi-level compression method provided in this application embodiment;

[0020] Figure 5 A flowchart illustrating an agent memory processing method executed by an agent, provided as an embodiment of this application;

[0021] Figure 6 A flowchart illustrating a client-executed intelligent agent memory processing method provided in this application embodiment. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0023] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0024] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Large Language Models (LLMs) are artificial intelligence models based on deep learning techniques. They are trained using massive amounts of text data to learn and generate natural language text. LLMs are characterized by large numbers of parameters, deep learning architectures, pre-training capabilities, fine-tuning flexibility, contextual understanding, multi-task learning, and high computational resource requirements. These models typically contain tens of billions of parameters, enabling them to generate natural language text, deeply understand text meaning, and handle various natural language tasks such as text summarization, question answering, and translation.

[0026] An intelligent agent is an entity capable of thought and interaction with its environment. For example, an intelligent agent can be a computer system or part of a computer system within a specific environment. Based on its own perception of the environment, following existing instructions or through autonomous learning, and by communicating and collaborating with other intelligent agents, an intelligent agent can autonomously achieve predetermined goals within its environment. An intelligent agent can be software or a combination of software and hardware.

[0027] Large Language Model-based Agent (LLM-based agent): Based on a large language model, it combines task planning, decision-making mechanisms, and environmental interaction capabilities, enabling it to perform complex tasks in dynamic environments. In this application's embodiments, "agent" refers to this type of large language model-based agent.

[0028] KV Cache: Short for Key-Value Cache, it is a cache of intermediate computation results (key-value pairs) stored during the inference process of a large language model. It is used to avoid redundant calculations and improve inference efficiency. It is stored on storage media such as the agent's video memory or RAM.

[0029] System prompts: Pre-defined instructions given to the model to set its behavior patterns, role, response style, or constraints. They are typically injected before the dialogue begins and are invisible to the user.

[0030] Video memory: Dedicated high-speed memory on artificial intelligence computing chips, used to store data required for large language model calculations, such as model parameters, activation values, KV cache, etc.

[0031] Experience: User-verified, reusable task execution paths and their corresponding KV Cache sets.

[0032] Semantic fingerprint: A low-dimensional vector identifier generated by a lightweight language model for fast similarity matching.

[0033] Dual-track storage: A storage paradigm that combines an explicit memory (EM) layer and an implicit memory (IM) layer. The explicit memory layer stores experience descriptions, few-shot examples, operation manuals, scenario descriptions, etc., in natural language. The implicit memory layer stores verified execution paths, scenario-based reasoning patterns, complex task solution trajectories, etc., in pre-generated key-value cache format.

[0034] The three-layer memory architecture consists of a Common Memory (CM) layer and a Personalized Memory (PM) layer. The Personalized Memory layer includes an implicit memory sublayer (also known as a long-term memory sublayer) and a short-term memory sublayer. The Common Memory layer stores shared contextual content for all users, such as system prompts, tool definitions, and general instruction templates. This layer's key-value cache is stored in the agent's high-speed cache pool (such as a Redis cluster or distributed memory storage) for all users to share and reuse, avoiding redundant computation. The Long-Term Memory sublayer stores verified execution paths, scenario-based reasoning patterns, complex task solution trajectories, and domain-specific experience in the form of pre-generated key-value caches. The Short-Term Memory sublayer stores temporary content such as the real-time interaction context of the current session, user input queries, and the current dialogue round.

[0035] Optionally, in this embodiment, a method such as... Figure 1 The data interaction system shown includes a client 102 and an agent 104. The agent 104 can connect to the client 102 via a network and can be used to provide services (e.g., intelligent inference services, etc.) to the client 102 or clients installed on the client 102.

[0036] Client 102, acting as the front-end entry point for user interaction, captures the user's natural language prompts in real time and encapsulates them into natural language requests, sending them to agent 104. Upon receiving the natural language response from agent 104, client 102 parses and renders it for the user, completing an intuitive human-computer interaction loop. If the user triggers the "save experience" or "mark as a high-quality path" operation during the interaction, client 102 not only records the current text content but also sends a marking instruction to agent 104. This assists agent 104 in associating the valid execution path generated and verified by the user during this interaction with its corresponding KV Cache state, thereby transforming temporary session data into reusable long-term memory assets.

[0037] Agent 104, acting as the core inference engine, receives natural language requests from client 102 via secure transmission channels (such as Transport Layer Security, Forward Secrecy, Hypertext Transfer Protocol, and Fast User Datagram Protocol). Based on system-preset instructions and currently available context resources, agent 104 invokes a large model to perform forward computation, generating a natural language response that matches the user's intent and returning it to client 102. Simultaneously, agent 104 identifies reusable execution paths based on preset strategies (such as user satisfaction or task complexity) and persistently stores the complete context state and metadata corresponding to these paths, building a long-term retention experience base. This allows for direct access to stored state resources when similar requests are received subsequently, significantly reducing redundant computation overhead and improving inference efficiency.

[0038] The aforementioned network may include, but is not limited to, at least one of the following: wired network and wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network (WAN), metropolitan area network (MAN), and local area network (LAN). The aforementioned wireless network may include, but is not limited to, at least one of the following: Wireless Fidelity (WIFI) and Bluetooth. Client 102 may be, but is not limited to, a personal computer (PC), mobile phone, tablet computer, etc.

[0039] In this embodiment, the intelligent agent 104 includes intelligent terminals such as robots, vehicle-mounted devices, or drones. The intelligent agents involved can include various handheld devices, robots, vehicle-mounted devices, drones, wearable devices, computing devices, or other processing devices. For example, the intelligent agent can be a mobile station (MS), subscriber unit, user equipment (UE), cellular phone, smartphone, wireless data card, personal digital assistant (PDA) computer, tablet computer, wireless modem, handset, laptop computer, machine-type communication (MTC) terminal, etc.

[0040] The data interaction system in this application embodiment can be applied, but is not limited to, to the field of intelligent agent systems based on large language models. For example, it can be applied to scenarios where intelligent agents need to perform complex tasks and reuse historical experience, such as intelligent software development assistants, intelligent customer service, code generation, and knowledge question answering. For example, in an intelligent software development scenario, when an intelligent agent solves a certain type of code problem for the first time, it obtains the correct execution path after multiple attempts. The key-value cache corresponding to the correct execution path is compressed and stored as implicit memory in the long-term memory sublayer of the intelligent agent. When the user encounters a similar code problem again, the client-side hit determination module determines the hit based on semantic fingerprints. The intelligent agent retrieves the associated implicit memory from the first implicit memory library, decompresses and reuses it, avoiding re-executing the complete forward calculation, thereby improving task execution efficiency and reducing computing power consumption. In an intelligent customer service scenario, the intelligent agent can retain the key-value cache corresponding to the solution path of frequently similar consultation questions as experience assets for long-term retention, and directly load and reuse them across sessions to achieve rapid response.

[0041] Currently, user interaction with intelligent agents mainly relies on natural language text: users submit queries in plain text, the intelligent agent receives them, performs inference calculations based on a large language model, and returns the results in plain text.

[0042] During large language model inference, the system stores the key and value matrices in the attention mechanism through a key-value cache (KV Cache) to avoid repeated calculations of historical inputs, significantly reducing computational overhead in long sequence or multi-turn dialogue scenarios. However, in existing technical architectures, the KV Cache is mainly stored in the agent's GPU memory or high-speed memory, and its lifecycle is usually limited to a single session or short-term temporary storage, rather than being persistently managed as a long-term reusable experience asset.

[0043] Current paradigms treat key-value (KV) caches as temporary computational byproducts rather than long-term computational resources. This "computation-heavy, storage-light" design philosophy has become a fundamental bottleneck restricting the development of intelligent agent systems. The computational demands of large-scale model inference are growing exponentially. The KV cache data size corresponding to a single token is approximately 35KB (key and value matrices combined), while the corresponding plaintext token requires only about 2 bytes, resulting in an expansion rate of approximately 17,500 times. Since the KV cache is released upon the end of the session, when a user submits a semantically similar query or performs a similar task again after a period of time, the system needs to re-execute the entire forward computation to rebuild these massive KV caches. This redundant computation not only causes significant waste of computational resources and energy consumption but also causes the operating costs of intelligent agent applications to expand linearly or even superlinearly with the user base.

[0044] However, related technologies store user experience and domain knowledge only in plaintext natural language (such as few-shot examples, historical dialogue records, and knowledge base documents). Each time a task is executed, the plaintext must be appended to the user's query, and the data must be re-encoded into a KV cache using a large model. When an agent performs complex tasks, the correct execution path obtained through multiple rounds of trials often corresponds to a KV cache containing millions of tokens. This repetitive computation of the same type of task results in a huge waste of computing power.

[0045] Key-value (KV) caches are the most valuable computational output in large-scale model inference. Although their generation consumes a significant amount of GPU computing power, they can be reused multiple times once generated to avoid redundant computation. By transforming KV caches from "use-and-discard" session-level temporary data into core computational assets that can retain value long-term and be reused across sessions, and establishing a new computational paradigm for persistent reuse of KV caches, we can replace computational consumption with storage investment, restructure the cost structure of intelligent agent systems, and achieve green and sustainable AI services.

[0046] In related technologies, long-term memory mainly relies on two approaches: one is to store the text embedding vectors of historical dialogues in an external vector database, but after retrieval, it is still necessary to re-encode and generate KV cache, which fails to avoid redundant calculations; the other is to solidify knowledge through model fine-tuning, which is only suitable for static knowledge updates, cannot adapt to the dynamic and personalized needs of users, and is extremely costly.

[0047] To address the problem of "repeated computation and wasted computing power due to the release of key-value cache upon session termination and the storage of existing experience in plaintext," this application provides a data interaction system including a client and an agent. The client, in response to an input query from a first object, retrieves historical experience identifiers matching the input query. Upon finding the first experience identifier, the client sends a dialogue request carrying the first experience identifier to the agent. The agent receives the dialogue request from the client, extracts the first experience identifier, retrieves the associated first implicit memory from the first object's first implicit memory based on the first experience identifier, decompresses the compressed key-value cache in the first implicit memory into a first key-value cache, and directly executes the first inference task based on the first key-value cache. This replaces the original need for re-execution of the complete forward computation with direct loading and reuse of the pre-generated key-value cache. This solution, through a mechanism of "long-term retention of experience in compressed key-value cache form and reuse based on experience identifiers," avoids repeated forward computation for similar tasks and enables direct loading of experience assets across sessions.

[0048] In this embodiment of the application, the client is used to, in response to receiving an input query statement, retrieve a historical experience identifier that matches the input query statement during the first session; and in response to retrieving the first experience identifier, send a dialogue request carrying the first experience identifier to the intelligent agent, wherein the dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform the first inference task.

[0049] In this context, the first session refers to the current interactive session initiated by the client. In this application, the client receives the query statement input by the user and performs a hit determination during the first session.

[0050] The input query statement refers to the natural language query text that the first object inputs to the client in the first session, such as a question, instruction, or description of a requirement posed by the user. The first object refers to the object currently interacting with the agent through the client.

[0051] In this embodiment, the client locally stores a semantic fingerprint information list. This list contains semantic fingerprints of multiple implicit memories of the first object and their associated empirical identifiers. Each semantic fingerprint in the list includes a semantic fingerprint and an empirical identifier. The semantic fingerprint information list is generated by the agent based on the first implicit memory of the first object and sent to the client of the first object to support the client in performing semantic fingerprint similarity retrieval locally.

[0052] An Experience ID (EID) is an identifier used to uniquely identify a historical experience. In this application, historical experience IDs are used to associate implicit memories with corresponding plaintext experiences, achieving logical binding and physical separation. The first experience ID refers to the historical experience ID carried in the dialogue request for requesting reuse.

[0053] Finding a first experience identifier that matches the input query means that the client has similar historical experience locally, which can be reused without regeneration. In this case, the client sends a dialogue request carrying the first experience identifier to the agent. The dialogue request is a message sent by the client to the agent requesting the reuse of a key-value cache associated with a specific experience identifier to perform a first inference task. The first inference task is the inference task performed by the agent based on the reused key-value cache. In this application, the first inference task no longer needs to re-execute the complete forward computation, but is directly executed based on the decompressed key-value cache.

[0054] If no first experience identifier matching the input query is found, it means that the client does not have similar historical experience locally. The client then sends the complete natural language query (the original text content) directly to the agent, instead of an experience identifier. Upon receiving the natural language query (the input query), the agent triggers a standard, complete large-model forward computation inference process. This means the system needs to process the input from scratch, generate the corresponding KVCache, and execute the inference task.

[0055] Key-value caching refers to the intermediate computation results stored during the inference process of a large model, i.e., key-value pair caching. It is used to avoid repeated computation of historical inputs, improve inference efficiency, and is stored on storage media such as the agent's video memory or RAM. In this application, historical experience refers to user-verified, reusable task execution paths and their corresponding key-value cache sets. In this application, one historical experience corresponds to one key-value cache set, which is compressed and stored as implicit memory.

[0056] In this embodiment, an intelligent agent refers to an artificial intelligence system capable of perceiving the environment, making decisions, and performing actions. It can interact with users in multiple rounds and complete complex tasks. In this application, the intelligent agent will undergo multiple attempts during task execution to finally obtain a correct execution path. The key-value cache corresponding to this correct execution path is the reusable experience asset.

[0057] The agent comprises an inference engine and a first implicit memory. An implicit memory in the first implicit memory contains a compressed key-value cache, obtained by compressing the key-value cache corresponding to a historical experience. Each implicit memory in the first implicit memory is associated with an experience identifier.

[0058] The first implicit memory is the repository for persistently storing key-value caches corresponding to the first object, representing historical experience. It's important to note that conventional key-value caches are intermediate computation results generated during large model inference, typically limited to a single session or short-term temporary storage, released upon session termination. The first implicit memory, however, does not store such session-level temporary key-value caches, but rather key-value caches corresponding to user-verified historical experience. These key-value caches are compressed and permanently stored in the first implicit memory as implicit memory, retrievable and reusable across sessions. The first implicit memory is the implicit memory of the first object. Each implicit memory in the first implicit memory is associated with an experience identifier. Implicit memory refers to verified execution paths, scenario-based inference patterns, complex task solution trajectories, and other experience assets stored in the form of pre-generated key-value caches; implicit memory is long-term experience, also known as long-term memory. Implicit memory includes compressed key-value cache, which is obtained by compressing the key-value cache corresponding to a historical experience. This means that the key-value cache corresponding to a historical experience is compressed and stored as implicit memory, thereby reducing the storage footprint of the key-value cache and making the long-term storage of the key-value cache economically feasible.

[0059] Compressed key-value cache refers to key-value cache data obtained by compressing the key-value cache corresponding to a historical experience. In this application, compressed key-value cache is used to reduce the storage footprint of key-value cache, making long-term storage of key-value cache economically feasible.

[0060] The inference engine refers to the engine that executes the first inference task, that is, the agent inference engine used by the agent to perform large model inference computation based on the key-value cache. After the agent receives a dialogue request, the inference engine, in response to receiving a dialogue request carrying a first experience identifier, retrieves the implicit memory associated with the first experience identifier carried in the dialogue request from the first implicit memory; in response to retrieving the first implicit memory, it decompresses the compressed key-value cache in the first implicit memory into a first key-value cache, and executes the first inference task based on the first key-value cache.

[0061] The first implicit memory refers to the implicit memory retrieved and associated with the first experience identifier. The agent's retrieval of the first implicit memory signifies that it has successfully located and obtained the pre-computed key-value cache bound to the first experience identifier. This first implicit memory can be directly loaded for the first inference task, avoiding large-scale model forward computation on similar inputs, significantly reducing computational consumption and improving response speed. The first key-value cache refers to the key-value cache obtained after decompressing the compressed key-value cache in the first implicit memory.

[0062] The embodiments provided in this application provide a data interaction system, which includes a client and an agent. After retrieving a first experience identifier that matches the input query statement provided by a first object, the client sends a dialogue request carrying the first experience identifier to the agent. After receiving the dialogue request carrying the first experience identifier, the agent extracts the first experience identifier from the dialogue request, and then retrieves the associated first implicit memory from the first object's first implicit memory based on the first experience identifier. The agent then decompresses the compressed key-value cache in the first implicit memory into a first key-value cache, and directly executes the first inference task based on the first key-value cache. This replaces the original need to re-execute the complete forward computation with the direct loading and reuse of the pre-generated key-value cache, solving the technical problem of repeated computation of similar tasks and wasted computing power due to the key-value cache being released upon the end of the session and existing experience being stored only in plaintext. This improves task execution accuracy and achieves the technical effect of reducing computing power consumption.

[0063] In one exemplary embodiment, the client is provided with a local semantic fingerprint database. A semantic fingerprint in the local semantic fingerprint database is a feature vector obtained by encoding a historical experience. A semantic fingerprint in the local semantic fingerprint database is associated with an experience identifier.

[0064] The local semantic fingerprint database, also known as the semantic fingerprint retrieval database, refers to a repository stored locally on the client that contains multiple semantic fingerprints and their associated experience identifiers. In this application, the local semantic fingerprint database is generated by the agent based on the first implicit memory database and distributed to the client to support the client in performing fast hit determination locally, avoiding the client retaining the original key-value cache.

[0065] In some embodiments, the client is further configured to convert the input query statement into a semantic feature vector and retrieve semantic fingerprints that satisfy similarity conditions with the semantic feature vector from a local semantic fingerprint database; in response to retrieving similar semantic fingerprints that satisfy similarity conditions, the client determines the empirical identifier of the similar semantic fingerprint as a first empirical identifier.

[0066] Here, the semantic feature vector refers to the low-dimensional vector representation obtained after encoding and transforming the input query statement. In this application, the client calls a lightweight language model (semantic feature extraction model) isomorphic to the agent to encode the input query statement into a low-dimensional semantic feature vector, which is used for similarity retrieval with the semantic fingerprint in the local semantic fingerprint database.

[0067] In some embodiments, Figure 2 An interaction flowchart between an agent and a client is provided as an embodiment of this application, such as... Figure 2As shown, during the first session, in response to receiving an input query, the client converts the input query into a semantic feature vector and performs a similarity search between the semantic feature vector and the semantic fingerprints stored in the local semantic fingerprint database. Here, a semantic fingerprint in the local semantic fingerprint database is a feature vector obtained by encoding a historical experience, and a semantic fingerprint in the local semantic fingerprint database is associated with an experience identifier. In response to finding a similar semantic fingerprint, the client generates a dialogue request and sends the dialogue request to the agent. The dialogue request carries a first experience identifier associated with the similar semantic fingerprint. The dialogue request is used to request the reuse of the key-value cache associated with the first experience identifier to perform the first inference task.

[0068] Figure 3 An architecture diagram of an edge-cloud collaborative intelligent agent memory management system provided in this application embodiment is shown below. Figure 3 As shown in this embodiment, the client includes a hit determination module for performing semantic fingerprint similarity retrieval on the user-input query locally. Upon a hit, the module extracts the identifier information and natural language description of the similar semantic fingerprint and generates a dialogue request based on these identifiers and descriptions. The agent includes an inference engine, a personalized memory storage pool, and a key-value cache compression / decompression module. The client's hit determination module extracts semantic features from the user-input query and performs a similarity retrieval between the extracted semantic feature vector and the locally stored semantic fingerprint. Upon finding a similar semantic fingerprint (i.e., a hit), the client generates a dialogue request carrying a first experience identifier and sends it to the agent via a secure transmission channel. Upon receiving the dialogue request, the agent retrieves the first implicit memory associated with the first experience identifier from the personalized memory storage pool using the inference engine. The key-value cache decompression module decompresses the compressed key-value cache in the first implicit memory. Finally, the inference engine executes a first inference task based on the decompressed first key-value cache. If no similar semantic fingerprint is found, i.e., a miss is determined, the client directly sends the complete natural language query (i.e., the original input text content) instead of the empirical identifier. After receiving the natural language query (i.e., the input query statement), the agent triggers the standard, complete large-scale model forward computation inference process.

[0069] In this embodiment, the input query statement is converted into a semantic feature vector, and semantic fingerprints that meet the similarity conditions between the semantic feature vector and the local semantic fingerprint database are retrieved, which can quickly locate relevant information. The empirical identifier of similar semantic fingerprints is determined as the first empirical identifier, which helps to improve the relevance and reliability of the retrieval, thereby improving the overall retrieval speed and efficiency.

[0070] In one exemplary embodiment, frameworks such as SGLang implement key-value cache reuse of system prompts through prefix trees or hash functions, accelerating the pre-computation of shared prefixes within the same active session. However, this mechanism only applies to common content shared by all users, limiting the reuse scope to multi-request parallel batch processing scenarios within the agent. For user-specific personalized key-value caches (such as those based on personal documents or historical conversations), existing systems adopt a "use and discard" strategy, not persisting them to the user's end or reserving them as long-term computing resources, resulting in recalculation for each user access. Furthermore, existing architectures do not differentiate between common memories and personalized content at the data privacy level; both types of data are stored in a homogeneous manner within the agent, which undoubtedly increases the risk of privacy leaks.

[0071] To address the aforementioned issues of "the reuse of common content being limited to within the intelligent agent, repeated calculation of personalized content, and increased privacy leakage risks due to the lack of differentiation between common memory and personalized content," this embodiment addresses these problems by having the dialogue request carry the current session interaction data. Before executing the first inference task based on the first key-value cache, a matching common memory is retrieved from the common memory bank based on the current session interaction data. If a matching common memory is found, the common key-value cache is extracted from the matching common memory to obtain the current common key-value cache. This allows the first inference task to be executed simultaneously based on both the current common key-value cache and the first key-value cache. Consequently, the key-value caches corresponding to common content such as system prompts and tool definitions are globally shared and reused in a read-only manner, avoiding repeated forward calculations of common content. Furthermore, by storing common memory and personalized content in a layered and differentiated manner, the risk of privacy leakage is reduced.

[0072] In some embodiments, the dialogue request carries current session interaction data; the agent also includes a common memory library; the common memory library is deployed in a common memory storage pool; a common memory in the common memory library contains a common key-value cache corresponding to a common context content, and the access permissions of the common memory in the common memory library are set to read-only.

[0073] Before executing the first inference task based on the first key-value cache, the inference engine is also used to retrieve common memories matching the current session interaction data from the common memory bank based on the current session interaction data; if a matching common memory is retrieved, the common key-value cache is extracted from the matching common memory to obtain the current common key-value cache, wherein the first inference task is also executed based on the current common key-value cache.

[0074] Current session interaction data refers to real-time interaction data entered by the user in the current session that is not covered by personalized long-term experience, such as the query statement entered by the user in the current session, the preceding content involved in the current dialogue turn, etc. In this application, the current session interaction data is carried in the dialogue request and used to retrieve matching common context content from the common memory.

[0075] A common memory is a repository used to store common memories, deployed within the common memory layer of the agent. Common memories refer to the context content shared by all users and its corresponding key-value caches, such as system prompts, tool definitions, and general instruction templates. A common key-value cache refers to a key-value cache corresponding to a common context content, pre-generated through forward computation of the large model and stored in the agent's cache pool. The connection between these three is as follows: the common memory is used to store common memories; common memories contain the correspondence between common context content and common key-value caches; and the access permissions for common memories are set to read-only to prevent client tampering.

[0076] The current common key-value cache refers to the common key-value cache corresponding to the common context content that matches the current session interaction data, which is extracted from the matching common memory.

[0077] In this embodiment, retrieving a matching common memory means that there is a part in the current session interaction data that semantically matches a certain common context content in the common memory library. This part of the content belongs to the common content shared by all users, and its corresponding key-value cache has been pre-generated and stored in the agent's high-speed cache pool. It can be directly reused without re-performing forward calculation.

[0078] In an exemplary embodiment, the process of executing the first inference task based on the current common key-value cache and the first key-value cache includes: before executing the first inference task based on the first key-value cache, the inference engine concatenates the decompressed first key-value cache with the current common key-value cache extracted from the matching common memory to obtain the concatenated target key-value cache; the inference engine executes the first inference task based on the target key-value cache. Here, the first key-value cache corresponds to the pre-generated key-value cache in personalized long-term experience, and the current common key-value cache corresponds to the pre-generated key-value cache of common context content. The concatenation of these two caches constitutes the complete key-value cache required by the first inference task, thereby simultaneously reusing the pre-generated key-value caches of common content and personalized long-term experience, avoiding repeated forward computation of both common content and personalized long-term experience.

[0079] In this embodiment, common memories are globally shared and reused in the form of read-only key-value caches, and common memories and personalized content are stored in a hierarchical and differentiated manner. On the one hand, this avoids repeated forward calculations of common content such as system prompts and tool definitions for all users, reducing the computing power consumption of the intelligent agent. On the other hand, by setting common memories to read-only permissions and storing them separately from other content, it prevents common memories from being tampered with by the client, reduces the risk of privacy leakage caused by the homogeneous storage of common memories and personalized content, and achieves the technical effects of reducing computing power consumption and enhancing privacy and security.

[0080] In an exemplary embodiment, in the prior art, although common content such as system prompts and tool definitions can be partially reused through prefix caching, temporary interactive content unique to the user's current session and not covered by common memory still needs to be recalculated one by one to generate key-value cache. Furthermore, the prior art does not finely separate "reusable common content" from "remaining content that needs to be recalculated", which means that even if some content has been pre-cached, the system may still repeatedly perform forward calculations on all context content, resulting in a waste of computing power.

[0081] To address the aforementioned issue of "failure to finely separate common and remaining content in the current session interaction data, leading to redundant calculations of the remaining content," this embodiment, before executing the first inference task based on the first key-value cache, determines whether there is any remaining session interaction data in the current session interaction data for which no matching common memory has been retrieved. If remaining session interaction data exists, forward calculation is performed only on the remaining session interaction data to obtain the second key-value cache corresponding to the remaining session interaction data. This allows the first inference task to be executed simultaneously based on the first key-value cache, the current common key-value cache, and the second key-value cache. Thus, while reusing the pre-generated key-value cache of common content and personalized long-term experience, forward calculation is performed only on the remaining uncached content, maximizing the reuse of the cached key-value cache and minimizing redundant calculations.

[0082] In some embodiments, before executing the first inference task based on the first key-value cache, the inference engine is further configured to: perform forward calculation on the remaining session interaction data in the current session interaction data where no matching common memory has been retrieved, to obtain the second key-value cache corresponding to the remaining session interaction data, wherein the first inference task is also executed based on the second key-value cache.

[0083] Among them, the remaining conversation interaction data in the current conversation interaction data that cannot be matched with common memory means that part of the content in the current conversation interaction data can be retrieved in the common memory bank to match with it (that is, this part of the content belongs to common content shared by all users), while another part of the content cannot be retrieved in the common memory bank to match with it. This part that cannot be retrieved to match with common memory is the remaining conversation interaction data. The remaining conversation interaction data belongs to temporary content that is unique to the user's current conversation and is not covered by common memory.

[0084] Remaining session interaction data refers to the portion of current session interaction data for which no matching common memory has been retrieved in the common memory repository. In this application, the remaining session interaction data needs to generate its corresponding key-value cache through forward computation.

[0085] The second key-value cache refers to the key-value cache obtained after performing forward computation on the remaining session interaction data. In this application, the second key-value cache corresponds to the uncached content of the remaining session interaction data and is one of the components of the key-value cache reused by the first inference task.

[0086] In an exemplary embodiment, the process of executing the first inference task based on the second key-value cache further includes: before executing the first inference task based on the first key-value cache, the inference engine concatenates the decompressed first key-value cache, the current common key-value cache extracted from the matching common memory, and the second key-value cache obtained by forward computation of the remaining session interaction data to obtain the concatenated target key-value cache; the inference engine executes the first inference task based on the target key-value cache. Here, the first key-value cache corresponds to a pre-generated key-value cache of personalized long-term experience, the current common key-value cache corresponds to a pre-generated key-value cache of common context content, and the second key-value cache corresponds to a newly generated key-value cache of the remaining session interaction data after forward computation. The three concatenated key-value caches together constitute the complete key-value cache required for the first inference task.

[0087] This embodiment identifies the remaining session interaction data in the current session interaction data for which no matching common memory has been retrieved, and performs forward computation only on the remaining session interaction data. This allows the first inference task to be executed simultaneously based on the pre-generated first key-value cache, the current common key-value cache, and the newly generated second key-value cache. This achieves a fine-grained separation of "reusable cached content" and "content that needs to be recalculated," maximizing the reuse of pre-generated key-value caches of common content and personalized long-term experience. Forward computation is performed only on the remaining uncached content, further reducing redundant computing power consumption and improving task execution efficiency.

[0088] In one exemplary embodiment, in the prior art, short-term memory is released immediately upon the end of a session, causing similar tasks to still need to be recalculated within a short period. To solve the problem of "short-term memory being released immediately upon the end of a session, leading to repeated calculation and transmission of similar tasks within a short period," in this embodiment, by having the dialogue request carry a first session identifier for identifying the first session, before executing the first inference task based on the first key-value cache, the time-sensitive memory associated with the first session is retrieved from the time-sensitive memory bank based on the first session identifier. If the associated time-sensitive memory is retrieved, the compressed key-value cache in the associated time-sensitive memory is decompressed into a third key-value cache, so that the first inference task is also executed based on the third key-value cache. This allows the short-term memory to be stored in the time-sensitive memory bank in the form of a compressed key-value cache, enabling direct reuse of similar tasks within a short period and avoiding repeated calculation and transmission.

[0089] In some embodiments, the dialogue request carries a first session identifier, which is used to identify the first session; the dialogue request is sent by a first object, and the first implicit memory is the implicit memory of the first object; the agent also includes a personalized memory storage pool; the personalized memory storage pool includes an implicit memory sublayer and a time-sensitive memory sublayer; the implicit memory sublayer is used to store the first implicit memory; the time-sensitive memory sublayer is used to store the time-sensitive memory of the first object; one time-sensitive memory in the time-sensitive memory contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical session interaction data;

[0090] In some embodiments, before performing the first inference task based on the first key-value cache, the inference engine is further configured to: retrieve the time-sensitive memory associated with the first session from the time-sensitive memory bank based on the first session identifier; and, if the associated time-sensitive memory is retrieved, decompress the compressed key-value cache in the associated time-sensitive memory into a third key-value cache, wherein the first inference task is also performed based on the third key-value cache.

[0091] In this embodiment, the first session refers to an interaction session currently initiated by the user. The first session identifier refers to the identification information used to identify the first session, such as a session identifier (Session_ID). The first session identifier serves as a unique identifier for the first session and is used to retrieve the time-sensitive memories associated with the first session in the time-sensitive memory bank.

[0092] In this embodiment, in addition to the common memory storage pool, the intelligent agent also includes a personalized memory storage pool. The personalized memory storage pool refers to the storage unit used to store the user's personalized memory data (such as implicit memory and time-sensitive memory). It is used to store implicit memory and time-sensitive memory of different objects. Implicit memory, also known as long-term memory, refers to verified execution paths, scenario-based reasoning patterns, complex task solution trajectories, and other experience assets stored in the form of pre-generated key-value caches. The key-value cache corresponding to long-term experience refers to the key-value pair cache set generated after the verified execution paths are computed forward by the large model. This key-value pair cache set is pre-generated, compressed, and persistently stored for a long time. When used, it is directly decompressed and reused without recalculation. Time-sensitive memory, also known as short-term memory, refers to temporary content such as the real-time interaction context of the current session, user input queries, and the current round of dialogue. The key-value cache corresponding to short-term experience refers to the key-value pair cache set generated after the real-time interaction context of the current session is computed by the large model. This key-value pair cache set is generated with the session, has a short lifespan, is retained for a short period after compression, and is periodically automatically forgotten.

[0093] Therefore, the key-value cache corresponding to long-term experience is a key-value cache of the verified correct execution path, which is pre-generated and persisted for a long time and can be reused across sessions; the key-value cache corresponding to short-term experience is a key-value cache of the temporary interaction content of the current session, which is generated with the session and has a short lifespan, and is only used for short-term reuse.

[0094] To achieve the separation of long-term and short-term memory, this embodiment further divides the personalized memory storage pool into an implicit memory sublayer and a time-sensitive memory sublayer, constructing a three-layer memory architecture and implementing the following differentiated management strategies:

[0095] like Figure 3 As shown in the embodiments of this application, the intelligent agent establishes a three-layer memory architecture and implements the following differentiated management strategies:

[0096] (1) Common Memory (CM): Stores contextual content shared by all users, such as system prompts, tool definitions, and general instruction templates. This layer's key-value cache is stored in the agent's high-speed cache pool (such as a Redis cluster or distributed memory storage) for all connected users to share and reuse, avoiding redundant calculations. The key-value cache of the common memory layer is set to read-only permission to prevent client tampering, and the agent updates it periodically to synchronize with system capability evolution. In this embodiment, the common memory library is deployed in the common memory layer.

[0097] (2) Personalized Memory (PM): Contains user-specific experience assets, divided into an implicit memory sublayer (also known as a long-term memory sublayer) and a time-sensitive memory sublayer (also known as a short-term memory sublayer). The long-term memory sublayer stores verified execution paths, scenario-based reasoning patterns, complex task solving trajectories, and domain-specific experiences in the form of pre-generated key-value caches, used to store implicit memories of different objects. It can be understood that the implicit memory sublayer stores implicit memory libraries corresponding to different objects, and the first implicit memory library is deployed in the implicit memory sublayer and is a storage unit in the implicit memory sublayer. In this embodiment, the short-term memory sublayer stores temporary content such as the real-time interaction context of the current session, user input queries, and the current round of dialogue, i.e., used to store time-sensitive memories of different objects. In this embodiment, the first implicit memory library is deployed in the long-term memory sublayer.

[0098] The time-sensitive memory corresponding to the first object refers to a repository used to store the time-sensitive memories of the first object. This time-sensitive memory is deployed in the time-sensitive memory sublayer of the agent. The time-sensitive memory of the first object refers to a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical session interaction data of the first object. The time-sensitive memory of the first object, etc., is used to store the time-sensitive memories of the first object. The time-sensitive memory of the first object is a compressed key-value cache corresponding to a historical session interaction data of the first object, stored in the short-term memory sublayer after multi-level compression.

[0099] In this embodiment, the time-limited memory corresponding to the first object contains the real-time interaction context of the first object in the current session. The real-time interaction context is stored in the time-limited memory sublayer of the agent after multi-level compression. The key-value cache corresponding to the time-limited memory (i.e., short-term memory) in the time-limited memory sublayer adopts isolated storage bound to the user ID, and its size is controllable after three-level compression. The agent uses AES-256-GCM encryption, and the key is derived from the user's master password or managed by the agent's HSM, ensuring that even if the agent is attacked, the attacker cannot access the short-term memory content across users.

[0100] Each time-sensitive memory in the time-sensitive memory corresponding to the first object is associated with a session identifier (Session_ID). The session identifier can be used to locate the time-sensitive memory of the corresponding session, thereby enabling fast retrieval and reuse when the session is initiated again. In this embodiment, the time-sensitive memory is deployed in the time-sensitive memory sublayer.

[0101] In this embodiment, retrieving the associated time-sensitive memory means that a time-sensitive memory associated with the first session identifier has been retrieved in the time-sensitive memory database. This indicates that the first session was initiated again in a short period, and the compressed key-value cache of its corresponding historical session interaction data has been stored in the time-sensitive memory database, which can be directly decompressed and reused. The associated time-sensitive memory refers to the time-sensitive memory associated with the first session identifier. The third key-value cache refers to the key-value cache obtained after decompressing the compressed key-value cache in the associated time-sensitive memory. The associated time-sensitive memory is retrieved based on the first session identifier, and the compressed key-value cache in the associated time-sensitive memory is decompressed to obtain the third key-value cache.

[0102] In an exemplary embodiment, the process of executing the first inference task based on the third key-value cache includes: before executing the first inference task based on the first key-value cache, concatenating the decompressed first key-value cache, the current common key-value cache extracted from matching common memory, the second key-value cache obtained by forward calculation of the remaining session interaction data, and the third key-value cache obtained by decompressing the compressed key-value cache in the associated time-sensitive memory to obtain the concatenated target key-value cache; the inference engine executes the first inference task based on the target key-value cache, thereby simultaneously reusing the pre-generated key-value caches of personalized long-term experience, common content, and short-term memory.

[0103] In some embodiments, this embodiment also provides a mechanism for reusing time-sensitive memories. Specifically, when a user initiates a similar query again within a short period of time (e.g., within 7 days), the client hit determination module matches the locally stored short-term memory semantic fingerprint and transmits the session identifier (Session_ID) to the agent. The agent extracts the corresponding compressed KV Cache from the short-term memory sub-database and decompresses and concatenates it to avoid repeatedly transmitting the complete context and re-encoding.

[0104] In this embodiment, short-term memory is stored in a time-sensitive memory bank in the form of compressed key-value cache. After retrieving the associated time-sensitive memory based on the first session identifier, it is decompressed and reused. This avoids the repeated calculation and transmission of similar tasks in the short term caused by the release of short-term memory when the session ends. It reduces the computing power consumption and network transmission overhead of the agent, improves the response efficiency of similar tasks in the short term, and achieves the technical effect of saving computing power and accelerating response speed.

[0105] In one exemplary embodiment, short-term memory in related technologies is released immediately upon the end of a session, leading to the need for repeated calculations of similar tasks within a short period. Furthermore, existing short-term memory management lacks flexible retention strategies and cannot differentiate the lifespan of short-term memory based on access frequency. This results in frequently accessed short-term memory being prematurely cleared, while infrequently accessed short-term memory occupies storage space for extended periods, making it difficult to balance storage efficiency and privacy control. To address the aforementioned problems of "short-term memory being released immediately upon the end of a session and lacking a flexible strategy for dynamically adjusting the retention period based on access frequency," this embodiment aims to achieve a balance between storage efficiency and privacy control by incorporating short-term memory into the unified management of the intelligent agent, establishing a periodic automatic forgetting mechanism (e.g., clearing after 7 days) and a dynamic retention strategy (automatic extension for high-frequency access), supporting manual deletion by users and policy configuration.

[0106] In some embodiments, the inference engine is further configured to: scan the access frequency of time-sensitive memories in the time-sensitive memory bank; adjust the retention period of time-sensitive memories in the time-sensitive memory bank based on the access frequency of time-sensitive memories in the time-sensitive memory bank; and clean up expired time-sensitive memories in the time-sensitive memory bank in response to the existence of expired time-sensitive memories that have exceeded the retention period.

[0107] In this system, the agent sets retention periods for time-sensitive memories in the time-sensitive memory bank and records the access frequency of these memories. The access frequency of a time-sensitive memory refers to the number of times or occurrences it is accessed or hit within a certain time period, used to measure its reusability. The retention period of a time-sensitive memory refers to the maximum duration for which it is retained in the time-sensitive memory bank; memories exceeding this period are considered expired. Expired time-sensitive memories are those stored in the time-sensitive memory bank for longer than their retention period. The agent's time-sensitive memory sub-layer periodically (e.g., daily) scans the access frequency and automatically cleans up short-term memories that have exceeded their retention period (e.g., 7 days) and have not been hit.

[0108] In this embodiment, the retention period of time-sensitive memories is dynamically adjusted according to the access frequency of the time-sensitive memories. The agent scans the time-sensitive memories in the time-sensitive memory bank at a specified time period (e.g., daily), calculates the retention period corresponding to each time-sensitive memory based on the access frequency of each time-sensitive memory, and updates the retention period of each time-sensitive memory to the calculated retention period. During the scanning process, if it is detected that the current storage duration of a time-sensitive memory has exceeded its updated retention period, the time-sensitive memory is identified as an expired time-sensitive memory and deleted from the time-sensitive memory bank.

[0109] In some embodiments, the inference engine is further configured to perform one of the following: extend the retention period of time-sensitive memories in the time-sensitive memory bank whose access frequency is greater than or equal to a specified frequency threshold to a specified retention period; adjust the retention period of time-sensitive memories in the time-sensitive memory bank according to the frequency-period correspondence based on the access frequency of time-sensitive memories in the time-sensitive memory bank, wherein the frequency-period correspondence is the correspondence between access frequency and retention period.

[0110] The specified frequency threshold refers to a preset frequency limit used to determine whether a time-sensitive memory is accessed frequently. Time-sensitive memories with an access frequency greater than or equal to this specified frequency threshold are considered high-frequency accessed time-sensitive memories. The specified retention period refers to the retention period of time-sensitive memories with high access frequency, dynamically extended. For example, the inference engine dynamically extends the retention period of short-term memories with high access frequency to a specified retention period (such as extending it to 30 days).

[0111] The frequency-period correspondence refers to the relationship between access frequency and retention period. Specifically, it maps different retention periods based on the access frequency's location within a given interval. For example, the frequency-period correspondence can be set as follows: access frequency in the first frequency interval corresponds to the first retention period (e.g., 7 days); access frequency in the second frequency interval corresponds to the second retention period (e.g., 30 days); and access frequency in the third frequency interval corresponds to the third retention period (e.g., 90 days). The access frequency in the second frequency interval is higher than that in the first, and the access frequency in the third is higher than that in the second. The inference engine maps the access frequency of each time-sensitive memory to its corresponding retention period according to this frequency-period correspondence, assigning longer retention periods to time-sensitive memories with higher access frequencies.

[0112] In some embodiments, in addition to dynamically adjusting the retention period of time-sensitive memories based on the access frequency of time-sensitive memories as described above, the inference engine can also adjust the retention period of time-sensitive memories according to a configured retention mode. Specifically, in response to a received mode configuration instruction, the inference engine configures the initial retention period of time-sensitive memories in the time-sensitive memory bank according to the retention mode indicated by the mode configuration instruction. The retention mode indicated by the mode configuration instruction is one of multiple retention modes, and one retention mode corresponds to one retention period. The retention periods corresponding to different retention modes among the multiple retention modes are different from each other.

[0113] The schema configuration instruction refers to the configuration command used to indicate the retention mode, which can be triggered by the user through the client and sent to the inference engine. A retention mode refers to one of several retention modes, each corresponding to a retention period, and the retention periods for different retention modes are different. For example, retention modes include any of the following: "Strict Mode" (3-day cleanup), "Standard Mode" (7-day cleanup), or "Relaxed Mode" (30-day cleanup). In response to the schema configuration instruction, the inference engine configures the initial retention period of the time-sensitive memories in the time-sensitive memory bank to the retention period corresponding to that retention mode.

[0114] In some embodiments, this embodiment also supports users manually deleting time-limited memories of specific sessions. Specifically, in response to receiving a user's deletion instruction for a specific session, the agent deletes the time-limited memory corresponding to that specific session from the time-limited memory bank, granting the user full control over digital memory assets.

[0115] In some embodiments, the inference engine can also convert time-sensitive memories with an access frequency higher than a preset access frequency threshold and a hit count exceeding a threshold into implicit memories for storage. Specifically, during the process of scanning the access frequency of time-sensitive memories in the time-sensitive memory bank, the inference engine identifies time-sensitive memories with an access frequency higher than a preset access frequency threshold and a cumulative hit count exceeding a threshold as high-frequency reused experiences. The compressed key-value cache in these time-sensitive memories is then used as a compressed key-value cache corresponding to a historical experience. An experience identifier is assigned to these time-sensitive memories, an implicit memory associated with the experience identifier is generated, and the implicit memory is stored in the implicit memory bank of the long-term memory sublayer. This achieves the conversion of time-sensitive memories into implicit memories, upgrading high-frequency reused short-term experiences into long-term experiences that can be persisted for a long time.

[0116] In some embodiments, when the time-sensitive memory meets preset upgrade conditions, it can be converted into implicit memory for storage. In this embodiment, the upgrade conversion from time-sensitive memory to implicit memory is based on retention period management. Upgrade determination is made by statistically analyzing the number of times the retention period of the time-sensitive memory is extended and the access stability of the time-sensitive memory. After the upgrade, a long-term retention period is set for the implicit memory, realizing bidirectional dynamic flow between short-term memory and long-term memory.

[0117] Specifically, during each scan of the access frequency of time-sensitive memories in the time-sensitive memory bank and the process of extending the retention period of time-sensitive memories with high access frequency, the inference engine simultaneously records the number of times the retention period of each time-sensitive memory is continuously extended. For a certain time-sensitive memory, if the number of times its retention period is continuously extended exceeds a preset extension threshold, it indicates that the time-sensitive memory has continuously spanned multiple retention periods and has been accessed frequently without being cleared in multiple consecutive scan periods, which is considered a stable reuse experience. Furthermore, the inference engine collects the access time interval between each access of the time-sensitive memory. If the fluctuation of the access time interval meets the preset stability condition, the time-sensitive memory is determined to be a stable reuse experience.

[0118] In response to the determination that the time-sensitive memory is a stable and reusable experience, the inference engine identifies the time-sensitive memory as a memory to be upgraded, assigns an experience identifier to the memory to be upgraded, generates an implicit memory associated with the experience identifier based on the compressed key-value cache in the memory to be upgraded, stores the implicit memory in the implicit memory bank of the long-term memory sublayer, and sets a long-term retention period (such as 365 days) for the implicit memory; at the same time, the time-sensitive memory is removed from the time-sensitive memory bank, realizing the upgrade conversion of time-sensitive memory to implicit memory.

[0119] Furthermore, this embodiment also provides a two-way dynamic transfer mechanism between short-term memory and long-term memory: for long-term memory that has been upgraded to implicit memory, the inference engine also periodically scans the access frequency of each implicit memory in the implicit memory bank. For implicit memories whose access frequency is lower than the preset long-term access threshold and which have not been accessed during their long-term retention period, the implicit memory is downgraded to time-limited memory and re-included in the retention period management of the time-limited memory bank, or the implicit memory is directly cleared, thereby avoiding the infinite expansion of the long-term memory bank and realizing the dynamic elimination of long-term memory.

[0120] Through the aforementioned upgrades and bidirectional dynamic transfer, on the one hand, upgrade determination is based on the number of times the retention period is extended and the stability of the access time interval, which can accurately identify truly stable and continuously reused experiences and avoid misjudgments based solely on the frequency of a single access. On the other hand, by setting a long-term retention period for the upgraded implicit memory and supporting the downgrading or cleanup of long-term memory, bidirectional dynamic transfer between short-term and long-term memory is achieved. This ensures the long-term retention of stable reuse experiences while preventing the unlimited expansion of the long-term memory bank, achieving a technical effect of long-term dynamic balance between storage resources and reuse value.

[0121] This embodiment incorporates short-term memory into the unified management of the intelligent agent, establishing a periodic automatic forgetting mechanism (such as 7-day clearing) and a dynamic retention strategy (automatic extension for high-frequency access). It also supports manual deletion and strategy configuration by the user. On the one hand, it automatically extends the retention period of time-limited memories with high access frequency and automatically clears expired time-limited memories with low access frequency, achieving efficient use of storage space and avoiding the problem of high-frequency access short-term memories being cleared prematurely and low-frequency access short-term memories occupying storage space for a long time. On the other hand, by supporting manual deletion and retention mode configuration by the user, it gives the user control over the life cycle of short-term memory, achieving a balance between storage efficiency and privacy control, and achieving the technical effect of saving storage space.

[0122] In one exemplary embodiment, the data density of the KV Cache is much higher than that of plaintext. The size of a KV Cache with one token is approximately 35KB (the sum of the key matrix and value matrix), while the corresponding plaintext token typically only requires 2 bytes, resulting in an expansion rate of approximately 17,500 times. For the storage of KV Cache accumulated by tens of millions of users over a long period, the storage capacity often exceeds hundreds of exabytes (EB). Directly storing the original KV Cache for a long period would lead to an unacceptable explosion in storage capacity and cost. To address the aforementioned issues of "high key-value cache bloat rate and direct long-term storage leading to an explosion in storage capacity and cost," this embodiment, when compressing the first key-value cache, performs at least one compression operation on the first key-value cache to obtain a compressed representation vector corresponding to the first key-value cache. Based on the compressed representation vector, it searches for compressed key-value caches in the implicit memory set that satisfy the vector similarity condition. If a target compressed key-value cache exists, the difference vector between the target compressed key-value cache and the compressed representation vector, along with the index pointer of the target compressed key-value cache, is used as the compressed key-value cache corresponding to the first key-value cache. If it does not exist, the compressed representation vector is used as the compressed key-value cache corresponding to the first key-value cache. Through multi-level compression techniques (structured quantization, neural semantic compression, differential coding, and global deduplication), the key-value cache is compressed by several orders of magnitude, making long-term storage feasible instead of infeasible.

[0123] In some embodiments, the agent further includes a personalized memory storage pool; the personalized memory storage pool includes an implicit memory sublayer; the implicit memory sublayer is used to store a set of implicit memory libraries; one implicit memory library in the set of implicit memory libraries corresponds to one object.

[0124] The implicit memory set refers to a collection of multiple implicit memories. Each implicit memory in the set corresponds to a single object, meaning different objects correspond to different implicit memories. The first implicit memory is the implicit memory corresponding to the first object and is an element in the implicit memory set. The implicit memory set encompasses the implicit memories corresponding to multiple objects and is used to support global similarity comparison and deduplication across objects and users.

[0125] In some embodiments, the inference engine is further configured to: when the first key-value cache is to be compressed, perform at least one compression on the first key-value cache to obtain a compressed representation vector corresponding to the first key-value cache; based on the compressed representation vector, search for compressed key-value caches in the implicit memory set that satisfy the vector similarity condition, wherein the vector similarity condition is that the vector similarity between the compressed key-value cache and the compressed key-value cache is greater than or equal to a specified similarity threshold; if there is a target compressed key-value cache in the implicit memory set that satisfies the vector similarity condition, determine the difference vector between the target compressed key-value cache and the compressed representation vector, and the index pointer of the target compressed key-value cache as the compressed key-value cache corresponding to the first key-value cache; if there is no compressed key-value cache in the implicit memory set that satisfies the vector similarity condition, determine the compressed representation vector as the compressed key-value cache corresponding to the first key-value cache.

[0126] The compressed representation vector refers to the low-dimensional vector used to represent the first key-value cache after performing compression at least once. By compressing the original high-dimensional, large-size key-value cache into a low-dimensional compressed representation vector, the storage footprint of the key-value cache can be significantly reduced, allowing the key-value cache, which originally had an expansion rate of tens of thousands of times, to be stored economically for a long time, thus laying the foundation for the long-term persistent reuse of the key-value cache.

[0127] In some embodiments, existing inference systems employ methods such as low-bit quantization and low-rank decomposition to compress the key-value cache, thereby reducing the memory footprint of a single inference operation. These techniques only optimize the internal memory or RAM of the agent; the key-value cache is still released when the session ends. When a user submits a similar task again, the entire forward computation must be re-executed, inevitably leading to redundant computational consumption. Furthermore, existing compression algorithms lack cross-user and cross-session reuse and deduplication mechanisms, making it difficult to support persistent storage of the key-value cache on a scale of hundreds of exabytes as a long-term computing resource. To address the aforementioned issues of "existing compression techniques only optimizing memory footprint for a single inference operation and lacking cross-user and cross-session reuse and deduplication mechanisms," this embodiment first performs hierarchical mixed-precision structured quantization compression on the first key-value cache to obtain an intermediate representation vector. This intermediate representation vector is then input into a pre-trained neural network model for semantic compression to obtain a compressed representation vector. This reduces the key-value cache's storage footprint while preserving its semantic information, providing support for subsequent cross-user and cross-session similarity deduplication and reuse.

[0128] In some embodiments, the above reasoning is further used to: perform the following structured quantization compression on the first key-value cache to obtain an intermediate representation vector corresponding to the first key-value cache: compress the first level in the first key-value cache according to a first quantization precision, and compress the second level in the first key-value cache according to a second quantization precision; the number of levels in the first level is less than the number of levels in the second level, and the first quantization precision is lower than the second quantization precision; input the intermediate representation vector to a pre-trained neural network model to perform semantic compression on the intermediate representation vector through the neural network model to obtain a compressed representation vector.

[0129] In this context, the first level in the first key-value cache refers to the level with the smallest number of levels, corresponding to the shallow network; the second level in the first key-value cache refers to the level with the largest number of levels, corresponding to the deep network. Shallow networks primarily encode local syntactic and lexical information, while deep networks primarily encode global semantics and logical relationships.

[0130] The first quantization precision refers to the quantization precision (i.e., bit precision) used when compressing the first level in the first key-value cache, and the second quantization precision refers to the quantization precision used when compressing the second level in the first key-value cache. The first quantization precision is lower than the second quantization precision. That is, the shallow network (first level) uses low bit quantization, tolerating higher compression loss; the deep network (second level) retains higher bit precision to ensure inference quality. In this embodiment, a mixed-precision quantization algorithm is used to process the high-bit-precision key-value cache layer by layer. A hierarchical quantization strategy is implemented based on the functional differences of each layer of the Transformer. This stage can compress the data volume by one order of magnitude.

[0131] The intermediate representation vector corresponding to the first key-value cache is the vector obtained after performing structured quantization compression on the first key-value cache. It is the output of structured quantization compression and serves as the input for subsequent semantic compression.

[0132] Figure 4 A flowchart of a multi-level compression method provided for embodiments of this application is shown below. Figure 4 As shown, after the first-level compression (i.e., structured quantization compression), the system uses an offline-trained neural network model (VAE or Transformer architecture) to perform a second compression for the KV Cache of the personalized experience layer. This is optimized for the distribution characteristics of the KV Cache, achieving a compression ratio of 100:1 to 200:1. The neural representations (i.e. compressed representation vectors) obtained by the neural network model compression are associated with the semantic fingerprints corresponding to the first key-value cache and stored. The semantic fingerprints are generated by the neural network model based on the plaintext experience descriptions encoded by the first implicit memory, and are used to support the client's fast hit determination and retrieval.

[0133] A target compressed key-value cache refers to a compressed key-value cache in the implicit memory set that satisfies the vector similarity condition. The existence of a target compressed key-value cache in the implicit memory set that satisfies the vector similarity condition means that the vector similarity between the first key-value cache to be compressed and a previously stored compressed key-value cache in the implicit memory set is greater than or equal to a specified similarity threshold (e.g., 0.95). In other words, they are highly similar, and it is unnecessary to store the complete compressed representation vector of the first key-value cache; only the difference between it and the target compressed key-value cache needs to be stored.

[0134] The difference vector between the target compressed key-value cache and the compressed representation vector refers to the difference vector between the compressed representation vector corresponding to the target compressed key-value cache and the first key-value cache, used to represent the difference information between the two. The index pointer of the target compressed key-value cache refers to the pointer information pointing to the storage location of the target compressed key-value cache. Determining the difference vector between the target compressed key-value cache and the compressed representation vector, as well as the index pointer of the target compressed key-value cache, as the compressed key-value cache corresponding to the first key-value cache means that only one copy of the baseline target compressed key-value cache and the difference vector and index pointer of the first key-value cache relative to this baseline are retained. Thus, based on reusing the already stored target compressed key-value cache, only the difference information needs to be additionally stored to restore the first key-value cache, achieving ultimate deduplication optimization at the storage pool level and avoiding the repeated storage of highly similar content.

[0135] In this embodiment, at the personalized experience storage pool level, the inference engine performs global similarity comparison: calculating the similarity of KV Caches for different users and in different scenarios (based on cosine similarity or Euclidean distance of the key matrix); for layers or blocks with similarity higher than a threshold (e.g., 0.95), only one base KV Cache and multiple differential vectors and index pointers are retained, achieving ultimate deduplication optimization at the storage pool level. For the experience storage of tens of millions of users, the KV Caches containing a large amount of common knowledge (such as general programming standards and common legal clauses) have high similarity, and differential encoding can achieve an additional compression ratio of about 10 times. After multi-level compression processing, the KV Cache with an original expansion rate of tens of thousands of times can achieve a comprehensive compression ratio of several thousand times, making long-term storage feasible rather than economically infeasible. Taking the original KV Cache with a single token of 35KB as an example, after compression, it only requires 3.5B to 35B of storage, which is comparable to or even smaller than the size of the plaintext token.

[0136] This embodiment employs layered mixed-precision structured quantization compression, semantic compression of neural network models, and differential encoding and global deduplication based on vector similarity to compress the key-value cache by several orders of magnitude. This transforms long-term storage of the key-value cache from economically infeasible to feasible, and converts it from a temporary cache into a long-term reusable asset. After three levels of compression, high-frequency similarity queries can directly load the pre-generated key-value cache, avoiding the forward computation of the complete large model. This is expected to reduce redundant computing power consumption by 70%-90%, significantly reducing GPU computing power overhead. This achieves the technical effect of significantly reducing computing power consumption and making long-term persistent storage of the key-value cache economically feasible.

[0137] In an exemplary embodiment, the dialogue request is sent by a first object, and the first implicit memory is the implicit memory of the first object. One implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience. Before extracting the first experience identifier from the dialogue request in response to receiving it, the inference engine is further configured to: generate a semantic fingerprint information list based on the semantic fingerprints in the implicit memories of the first implicit memory and the experience identifiers associated with the implicit memories in the first implicit memory, wherein one semantic fingerprint in the semantic fingerprint information list includes a semantic fingerprint and an experience identifier; and send the semantic fingerprint information list to the client of the first object, wherein the semantic fingerprint information list is used to determine the key-value cache reused for performing the inference task based on the similarity between the feature vector of the query statement input by the client of the first object and the semantic fingerprints in the semantic fingerprint information list.

[0138] Here, the first object refers to the object that sends the dialogue request, that is, the user who owns the first implicit memory. The first implicit memory is the implicit memory of the first object, representing that the agent stores the implicit memory corresponding to different objects. The implicit memory of different objects is isolated from each other, and the first object can only access its own corresponding first implicit memory.

[0139] The semantic fingerprint corresponding to implicit memory refers to a low-dimensional vector identifier generated by the agent's lightweight language model based on the plaintext experience description encoded by implicit memory, used for fast similarity matching. It is typically a 128-512 dimensional floating-point vector, with a single experience being approximately 1KB in size. Each implicit memory in the first implicit memory bank also contains a semantic fingerprint corresponding to a historical experience, representing the association between each implicit memory and its corresponding semantic fingerprint. The semantic fingerprint allows for rapid determination of the similarity between the input query and the implicit memory.

[0140] The semantic fingerprint information list refers to a list containing semantic fingerprints of multiple implicit memories of the first object and their associated empirical identifiers. Each semantic fingerprint in the list includes a semantic fingerprint and an empirical identifier. The semantic fingerprint information list is generated by the agent based on the first implicit memory of the first object and sent to the client of the first object to support the client in performing semantic fingerprint similarity retrieval locally.

[0141] In an exemplary embodiment, the process by which the inference engine generates a semantic fingerprint information list based on semantic fingerprints in implicit memories in a first implicit memory bank and experience identifiers associated with implicit memories in the first implicit memory bank includes: traversing each implicit memory in the first implicit memory bank; for each implicit memory, reading the semantic fingerprint contained in the implicit memory and the experience identifier associated with the implicit memory; combining the semantic fingerprint and the experience identifier into a semantic fingerprint information; and summarizing the semantic fingerprint information corresponding to each implicit memory into a semantic fingerprint information list. After receiving the semantic fingerprint information list, the client of the first object stores it locally for local matching determination when the user inputs a query.

[0142] In an exemplary embodiment, the process by which the client of the first object determines the key-value cache to be reused for performing the reasoning task based on the similarity between the feature vector of the query statement and the semantic fingerprints in the semantic fingerprint information list includes: the client calling a lightweight language model isomorphic to the agent to encode the user-input query statement into a low-dimensional semantic feature vector; the client performing an approximate nearest neighbor search on the semantic feature vector and each semantic fingerprint in the local semantic fingerprint information list, and calculating the similarity between the semantic feature vector and each semantic fingerprint; if the similarity between a certain semantic fingerprint and the semantic feature vector exceeds a set threshold (e.g., 0.90), it is determined to be an experience hit, the experience identifier associated with the semantic fingerprint is extracted, and a dialogue request carrying the experience identifier is generated and sent to the agent, which then reuses the key-value cache associated with the experience identifier to perform the reasoning task.

[0143] In this embodiment, the inference engine generates a list of semantic fingerprint information containing only semantic fingerprints and experience identifiers and sends it to the client. This allows the client to quickly determine the hit locally by only retaining the semantic fingerprint of the lightweight, irreversible original text. After a hit, only the experience identifier is transmitted, avoiding the privacy leakage risk caused by the client retaining the original key-value cache. At the same time, it reduces the latency and burden caused by relying entirely on the intelligent agent's judgment, achieving the technical effects of enhancing privacy and security and improving the efficiency of hit determination.

[0144] In an exemplary embodiment, the dialogue request is sent by the first object, the first implicit memory is the implicit memory of the first object, and an implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience.

[0145] The intelligent agent also includes a personalized memory storage pool; the personalized memory storage pool includes an implicit memory sub-layer; the implicit memory sub-layer is used to store the first implicit memory library; the implicit memory sub-layer includes an object plaintext experience layer and a key-value cache experience layer; the key-value cache experience layer is used to store the first implicit memory library; the object plaintext experience layer is used to store the object plaintext experience library corresponding to the first object; the object plaintext experience library is used to store the historical experience of the first object in plaintext form, and the plaintext experience in the object plaintext experience library is associated with the implicit memory in the first implicit memory library through experience identifiers.

[0146] The key-value cache experience layer refers to the storage unit that pre-generates and stores compressed key-value caches. The object plaintext experience layer refers to the storage unit that stores experience description information and metadata in plaintext form.

[0147] like Figure 3As shown, the edge-cloud collaborative intelligent agent memory management system includes an intelligent agent's inference engine, a common memory storage pool, an individual memory storage pool, a key-value cache compression / decompression module, an edge-side hit determination module, and a secure transmission channel. This forms a three-layer memory management system of "common sharing + individual long-term + individual short-term" and a dual-track experience reuse closed loop of "plaintext guidance + key-value cache driving." This embodiment establishes a dual-track storage paradigm for the experience of the first object, serving as the core management mechanism for the long-term memory sublayer within the individual memory layer. The long-term memory sublayer is further divided into the following two categories:

[0148] (1) Plaintext Experience Layer (Explicit Memory, or EM for short): This layer stores experience descriptions, few-shot examples, operation manuals, scenario descriptions, etc., in natural language. This layer is easy for humans to read, understand, and edit quickly, but it needs to be re-encoded to generate key-value caches when used. It is suitable for experience content that is accessed infrequently or needs to be changed frequently. The plaintext experience library corresponding to the first object is deployed in the plaintext experience layer.

[0149] (2) Implicit Memory (IM): This layer stores verified execution paths, scenario-based reasoning patterns, and complex task solution trajectories in the form of pre-generated key-value caches. It directly stores the key-value caches calculated from large models, eliminating the need for recalculation during use. This layer is suitable for frequently reused and structurally stable experience assets. The first implicit memory corresponding to the first object is deployed in the IM layer.

[0150] This embodiment employs a dual-track association mechanism to store each implicit memory of the first object. Specifically, each key-value cached experience (IM) in the key-value cached experience layer and its corresponding plaintext experience (EM) in the plaintext experience layer are associated and managed through a unified experience ID (EID). An experience contains a complete key-value cache matrix and semantic fingerprint, while the EM contains readable text descriptions and metadata. The first object can quickly understand the experience content based on the EM and directly load and execute it via the IM.

[0151] It can be understood that the object plaintext experience repository refers to the repository in the object plaintext experience layer used to store the historical experience of the first object in plaintext form. The object plaintext experience repository stores historical experience in plaintext form of natural language, while the first implicit memory repository stores historical experience in the form of pre-generated key-value cache. The plaintext experience and implicit memory in both are associated through a unified experience identifier, achieving logical binding and physical separation.

[0152] In some embodiments, the inference engine is further configured to: in response to the existence of a fourth key-value cache to be stored, compress the fourth key-value cache to obtain a compressed key-value cache to be stored, wherein the fourth key-value cache is a key-value cache corresponding to the historical experience to be stored of the first object; obtain a semantic fingerprint to be stored, wherein the semantic fingerprint to be stored is obtained by encoding the historical experience to be stored; generate a second implicit memory to be stored based on the compressed key-value cache to be stored and the semantic fingerprint to be stored, store the second implicit memory in the first implicit memory bank, and store the historical experience to be stored in the plaintext experience bank of the object corresponding to the first object.

[0153] The fourth key-value cache refers to the key-value cache corresponding to the historical experience of the first object to be stored. That is, it is the key-value cache generated after the user-verified, reusable task execution path of the first object is forward-computed using the large model. The compressed key-value cache to be stored refers to the key-value cache obtained after compressing the fourth key-value cache. The compressed key-value cache to be stored is obtained by multi-level compression of the fourth key-value cache, used to reduce the storage footprint of the key-value cache and enable economical long-term storage.

[0154] The semantic fingerprint to be stored refers to the semantic fingerprint obtained by encoding the historical experience to be stored, which is used to support the client to perform fast similarity matching and hit determination of the historical experience to be stored.

[0155] The second implicit memory refers to the implicit memory to be stored, generated based on the compressed key-value cache to be stored and the semantic fingerprint to be stored. The second implicit memory stores the compressed key-value cache corresponding to the historical experience to be stored in the form of a pre-generated key-value cache, and associates it with the corresponding plaintext experience in the object plaintext experience library through experience identifiers.

[0156] Optionally, the generation process of the second implicit memory of the first object includes: the agent deployed on the server performs a complex task, obtains the correct execution path through multiple attempts, the server inputs the complete context (including system prompts, Few-shot examples, and historical inference trajectories) corresponding to the execution path into the forward computation of the large model, generates a fourth key-value cache to be stored (including a complete key-value cache matrix), the server calls the key-value cache compression engine to perform multi-level compression on the fourth key-value cache to obtain a compressed key-value cache to be stored, stores the compressed key-value cache to be stored in the first implicit memory bank, and generates IM entries; the first object can choose to edit the plaintext experience description of the fourth key-value cache to complete the experience metadata (such as application scenario, applicable model version, generation time, etc.), and generate a semantic fingerprint to be stored for the fourth key-value cache based on the plaintext experience description of the fourth key-value cache, bind the semantic fingerprint to be stored for the fourth key-value cache and the compressed key-value cache to be stored for the fourth key-value cache through experience identifier (EID), and store it in the long-term memory sublayer in the personalized memory layer.

[0157] This embodiment establishes a dual-track storage paradigm of "plaintext experience layer + key-value cache experience layer". Historical experience is stored in both plaintext and pre-generated key-value cache form and associated with a unified experience identifier. On the one hand, users can quickly understand and edit experience content through plaintext experience. On the other hand, users can directly load and reuse experience through pre-generated key-value cache, avoiding repeated forward calculations. This achieves the technical effect of facilitating experience management and reducing redundant computing power consumption.

[0158] In one exemplary embodiment, in the related technology, the agent directly transmits the decompressed key-value cache to the inference engine. The key-value cache is not encrypted during transmission, and the transmission channel is not encrypted. The key-value cache content is at risk of being eavesdropped on, tampered with, or leaked.

[0159] To address the aforementioned issue of "unencrypted key-value cache transmission, posing a risk of eavesdropping and tampering," this embodiment encrypts the first key-value cache before executing the first inference task based on it, resulting in an encrypted first key-value cache. This encrypted first key-value cache is then transmitted to the inference engine via an encrypted channel, thereby protecting the confidentiality and integrity of the key-value cache during transmission.

[0160] In some embodiments, before performing the first inference task based on the first key-value cache, the inference engine is further configured to: encrypt the first key-value cache to obtain an encrypted first key-value cache, and transmit the first key-value cache to the inference engine through an encrypted channel, wherein the inference engine is the engine that performs the first inference task.

[0161] The encryption method can include, but is not limited to, symmetric encryption algorithms, such as the AES-256-GCM encryption algorithm. The key is derived from the user's master password or managed by the agent's hardware security module (HSM). Since the first key-value cache, as the key-value cache corresponding to the verified correct execution path, carries the user's experience assets, if it is eavesdropped on or tampered with during transmission, it will lead to the leakage of experience assets or malicious manipulation of reasoning results. Therefore, the first key-value cache needs to be encrypted. Even if the transmitted content is intercepted by a third party, the third party cannot decipher the original content of the key-value cache, thereby protecting the confidentiality and integrity of the experience assets.

[0162] An encrypted channel refers to a channel established using encryption mechanisms for secure data transmission, such as an encrypted channel combined with forward secrecy. The agent returns a decompressed key-value cache, which is encrypted via the channel and then directly enters the inference engine without being written to disk. Upon receiving the encrypted first key-value cache transmitted via the encrypted channel, the inference engine decrypts the encrypted first key-value cache and executes the first inference task based on the decrypted first key-value cache.

[0163] In this embodiment, the first key-value cache is encrypted and transmitted to the inference engine through an encrypted channel, which protects the confidentiality and integrity of the key-value cache during transmission and avoids the risk of the key-value cache content being eavesdropped, tampered with or leaked during transmission, thus achieving the technical effect of enhancing the security of experience asset transmission.

[0164] In an exemplary embodiment, in the related art, when a long-cycle task executed by an agent is interrupted due to network interruption, device switching, or other reasons, since the key-value cache is released when the session ends or is interrupted, the user needs to retransmit the complete historical context and re-execute the complete forward calculation when resuming the task, resulting in large-scale context recalculation, low resuming efficiency, and inability to achieve rapid recovery after session interruption.

[0165] To address the aforementioned issue of "key-value cache being released after task interruption, requiring recalculation of large-scale historical context upon resumption," this embodiment, in response to receiving a task resumption request, extracts the second experience identifier, the second session identifier, and the current context information from the task resumption request. It then retrieves the implicit memory associated with the second experience identifier from the second implicit memory. If the third implicit memory is found, the compressed key-value cache in the third implicit memory is decompressed into a fifth key-value cache. The fifth key-value cache is then concatenated with the key-value cache corresponding to the current context information. The second inference task continues to be executed based on the concatenated key-value cache, thereby reusing the pre-generated key-value cache before the interruption and encoding only the current context information added during the interruption, avoiding large-scale context recalculation from the original text.

[0166] In some embodiments, the client is further configured to: during the second session, in response to the task interruption time of the second inference task exceeding a specified time threshold, save the working status information of the second session to a local interruption status file, wherein the working status information includes the second session identifier of the second session and the second experience identifier used by the second inference task; in the case of resuming the interruption of the second inference task, generate a task resumption request and send the task resumption request to the agent, wherein the task resumption request carries the second experience identifier, the second session identifier and the current context information, the task resumption request is used to request the reuse of the key-value cache associated with the second experience identifier to resume the interrupted second inference task, and the current context information is the context information generated from the time the interruption of the second inference task occurred to the time the request for the task resumption request was initiated.

[0167] In some embodiments, the inference engine is further configured to: in response to receiving a task continuation request, extract a second experience identifier, a second session identifier, and current context information from the task continuation request, wherein the second session identifier is used to identify the second session, the task continuation request is used to request the reuse of the key-value cache associated with the second experience identifier to continue the second inference task of the interrupted second session, and the current context information is context information generated from the time the second inference task was interrupted to the time the task continuation request was initiated; retrieve an implicit memory associated with the second experience identifier from the second implicit memory, wherein an implicit memory in the second implicit memory contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience, and an implicit memory in the second implicit memory is associated with an experience identifier; in response to retrieving a third implicit memory, decompress the compressed key-value cache in the third implicit memory into a fifth key-value cache; concatenate the fifth key-value cache with the key-value cache corresponding to the current context information, and continue to execute the second inference task based on the obtained concatenated key-value cache.

[0168] A task continuation request is a request sent by the client to reuse the key-value cache associated with the second experience identifier and resume the second inference task of the interrupted second session. When the interruption time exceeds the agent's key-value cache lifetime (e.g., 30 minutes), the client saves the key-value cache identifier (including experience identifier EID and session identifier Session_ID) of the current task (i.e., the second session) to the local interruption status file. When resuming the task, the client sends a task continuation request to the agent's inference engine, carrying the second experience identifier and the second session identifier. The inference engine extracts the specified experience (including immediate memory or implicit memory) from the individual memory layer, combines it with the key-value cache corresponding to the current context information uploaded by the client, and re-executes the fusion calculation to generate a concatenated key-value cache, avoiding large-scale context recalculation from the original text.

[0169] The second experience identifier refers to the experience identifier carried in the task continuation request, used to identify the experience reused by the interrupted second inference task. The second session identifier refers to the identifier used to identify the second session (such as Session_ID). The second inference task refers to the inference task corresponding to the interrupted second session. The current context information refers to the context information generated from the time the second inference task was interrupted to the time the task continuation request was initiated, that is, the context content added during the interruption.

[0170] The second implicit memory bank refers to the implicit memory bank used to retrieve implicit memories associated with the second experience identifier. The difference between the second implicit memory bank and the first implicit memory bank is that the second implicit memory bank may be the first implicit memory bank, or it may be the implicit memory bank corresponding to other objects. That is, which object the second implicit memory bank is specifically the implicit memory bank of depends on which implicit memory bank the implicit memory associated with the second experience identifier is stored in.

[0171] The third implicit memory refers to the implicit memory retrieved and associated with the second experience identifier. The fifth key-value cache refers to the key-value cache obtained by decompressing the compressed key-value cache in the third implicit memory.

[0172] The concatenated key-value cache refers to the key-value cache obtained by concatenating the fifth key-value cache with the key-value cache corresponding to the current context information. The concatenated key-value cache includes both the pre-generated key-value cache corresponding to the historical experience before the interruption and the key-value cache corresponding to the new context added during the interruption, forming the complete key-value cache required to continue the second inference task.

[0173] In an exemplary embodiment, the process of concatenating the fifth key-value cache with the key-value cache corresponding to the current context information and continuing to execute the second inference task based on the concatenated key-value cache includes: after the inference engine decompresses the compressed key-value cache in the third implicit memory into the fifth key-value cache, it performs forward computation on the current context information to obtain the key-value cache corresponding to the current context information; concatenates the fifth key-value cache with the key-value cache corresponding to the current context information in hierarchical order to obtain the concatenated key-value cache; and the inference engine continues to execute the second inference task based on the concatenated key-value cache.

[0174] In this embodiment, the pre-generated key-value cache corresponding to the historical experience before the interruption is reused in the task continuation request. Only the current context information added during the interruption is encoded and concatenated with the pre-generated key-value cache. This avoids large-scale context recalculation from the original text, realizes rapid recovery of working state after session interruption, and achieves the technical effects of ensuring cross-session work continuity and saving computing power consumption.

[0175] In an exemplary embodiment, the inference engine is further configured to: in response to receiving an experience transfer request from a second object, extract a third experience identifier and transfer authorization information from the experience transfer request; in response to the transfer authorization information verification passing, retrieve the implicit memory associated with the third experience identifier from a third implicit memory bank; and in response to retrieving the fourth implicit memory, create a copy instance of the fourth implicit memory in the storage space of the second object, and bind the access permissions of the copy instance to the object identifier of the second object.

[0176] In existing technologies, the sharing of user experience relies on plaintext formats (e.g., documents, code, process descriptions), which poses a risk of experience leakage and makes it difficult to effectively protect user data security. To address this, this embodiment leverages the characteristic of key-value caching—that the original text generated by a large model is difficult to recover (i.e., irreversible)—to provide a secure experience transaction (experience transfer) and propagation mechanism based on key-value caching: A user's historical experience circulates among users in the form of key-value caches, and the buyer cannot reverse-engineer the plaintext information, achieving a secure transaction model where experience can be propagated but the original text is irreversible.

[0177] It should be noted that the key-value cache is a high-dimensional vector space of floating-point numbers or integers calculated by the large model, and has no direct and reversible mapping with the original token (word). Therefore, it is difficult to reverse the key-value cache to recover the original text after it is generated by the large model. In addition, the key-value cache is circulated in a compressed form (for example, after at least one compression as mentioned above). The transaction target is a compressed personalized experience key-value cache, which further increases the difficulty of reversal, thereby ensuring that the buyer can use it but cannot steal it.

[0178] Optionally, tradable historical experience can take many forms, including but not limited to all or part of verified execution paths, scenario-based reasoning patterns, complex task solution trajectories, and domain-specific experience stored in the form of pre-generated key-value caches. For example, personalized long-term memories such as correct execution paths formed during reasoning processes for different scenarios can circulate among users in the form of key-value caches, and the buyer cannot reverse-engineer the plaintext information.

[0179] The transfer of experience can be authorized by the creator (i.e., the owner of the experience). The creator can authorize the method of experience transfer, the type or specific recipient of the experience, the time limit for experience transfer, etc. The entire experience flow needs to be executed under the authorization of the creator to ensure the reliability and effectiveness of experience transfer.

[0180] To ensure the secure transfer of experience assets, an EID-authorized transaction mechanism can be adopted: the transaction target is circulated with an EID identifier. When experience is traded, only the identifier and authorization are transferred without exposing the original text. The buyer can call the interface of the inference engine but cannot reverse extract the plaintext, thereby realizing the reuse and circulation of experience assets while ensuring privacy and security.

[0181] For the agent, the inference engine can receive an experience transfer request from the second object. This request carries a third experience identifier and transfer authorization information. The experience transfer request is used to request the transfer of implicit memory associated with the third experience identifier to the second object. The transfer authorization information is used to authorize the transfer of implicit memory associated with the third experience identifier to the second object. The second object and the first object can be the same or different; the third experience identifier can be the same as or different from the first or second experience identifier, and this embodiment does not impose any limitations on this.

[0182] Upon receiving the aforementioned experience transfer request, the inference engine can extract the third experience identifier and transfer authorization information from the request, and verify the legality of the transfer authorization information. If the transfer authorization information verification passes, the inference engine can retrieve the implicit memory associated with the third experience identifier from the third implicit memory. An implicit memory in the third implicit memory contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience. An implicit memory in the third implicit memory is associated with an experience identifier. The third implicit memory can be the same as or different from the first or second implicit memory. This third implicit memory is the implicit memory corresponding to the experience transfer authorization object to which the historical experience corresponding to the third experience identifier belongs.

[0183] In response to the retrieval of the fourth implicit memory, the inference engine can create a copy instance of the fourth implicit memory in the storage space of the second object. This copy instance can be a complete replica of the fourth implicit memory or an access pointer to it. Furthermore, the inference engine can bind the access permissions of the copy instance to the object identifier of the second object. If the copy instance is an access pointer to the fourth implicit memory, the access pointer can be bound to the object identifier of the second object. The inference engine only allows the retrieval and decryption of the compressed key-value cache in the fourth implicit memory from the storage pool for inference tasks when the client's request carries a decryption key or access token matching the object identifier of the second object. The original plaintext experience data corresponding to the fourth implicit memory is not transferred to the second object along with the access pointer. Here, the buyer can call the transferred experience from the inference engine, but cannot reverse-engineer the original plaintext (e.g., the original execution path plaintext), thus limiting the use of the transferred experience.

[0184] It should be noted that the transaction process only transfers the experience identifier (EID) and authorization, without transmitting key-value cache content. Furthermore, only the irrecoverable implicit memory (IM) is transferred, and the buyer cannot reverse-engineer the original plaintext, thus protecting the creator's privacy. The transaction experience is bound to the buyer's ID (object identifier), and the creator retains the original experience identifier but cannot access the buyer's instance or track usage records, thereby protecting the buyer's privacy.

[0185] In this embodiment, experience transfer only transfers experience identifiers and authorizations, without transmitting key-value cache content. Furthermore, by utilizing the characteristic that key-value caches are difficult to reverse-engineer after being generated by a large model, only the unrecoverable implicit memory is transferred, which can protect the creator's privacy. Transactional experiences are bound to the buyer's identifier, and the creator cannot track usage records, which can protect the buyer's privacy, thereby improving the security of experience transfer.

[0186] In an exemplary embodiment, the inference engine is further configured to: in response to receiving an experience publishing request from a third object, extract a fourth experience identifier from the experience publishing request; obtain experience metadata corresponding to the implicit memory associated with the fourth experience identifier to obtain experience metadata to be published; generate experience publishing information based on the fourth experience identifier and the experience metadata to be published, and publish the experience publishing information to the experience circulation platform.

[0187] For creators, they can authorize the publication of their experiences on an experience sharing platform, which is used to facilitate the transfer of experiences between different audiences. The information published on the experience sharing platform can include experience identifiers and experience metadata. Experience identifiers can be used to locate the experience to be transferred (implicit memory), while experience metadata can help buyers understand the published experience, improving the accuracy and reliability of experience transfer.

[0188] For creators, they can trigger experience encapsulation and authorize its release. Experience encapsulation encapsulates the key-value cache (IM) generated by the creator and can also include metadata (e.g., application scenario, applicable model version, compression parameters, performance benchmarks, etc.). During the release authorization, the experience identifier and metadata can be published to the experience trading platform; the original plaintext and compressed key-value cache are not listed. For buyers, they can make a purchase and delivery. After paying certain resources, the buyer obtains the experience identifier and decryption authorization, and sends these to the inference engine. The inference engine can then copy the implicit memory (IM) to the buyer's account and bind it to the buyer's user ID.

[0189] For a single experience transfer, in response to receiving an experience publishing request from a third object (corresponding to the creator), the inference engine can extract a fourth experience identifier from the experience publishing request. The experience publishing request is used to request the publishing of the implicit memory associated with the fourth experience identifier, which is the implicit memory of the third object. The third object can be the same as or different from the aforementioned first or second object. The fourth experience identifier can be the same as or different from the aforementioned first, second, and third experience identifiers; this embodiment does not impose any limitations on this.

[0190] The inference engine can obtain the experience metadata corresponding to the implicit memory associated with the fourth experience identifier, and obtain the experience metadata to be published. The type of the experience metadata to be published can include at least one of the following: application scenario, applicable model version, compression parameters, and performance benchmark. Based on the fourth experience identifier and the experience metadata to be published, the inference engine can generate experience publication information (the experience publication information can include the fourth experience identifier and the experience metadata to be published), and publish the experience publication information to the experience circulation platform.

[0191] In this embodiment, the transfer target for experience sharing between different objects is a compressed experience IM (excluding plaintext experience, with optional de-identified summary), which circulates on the experience sharing platform with experience identifiers, thereby improving the security and convenience of experience sharing.

[0192] In one exemplary embodiment, this application also provides an intelligent agent, including:

[0193] The inference engine and the first implicit memory bank, where an implicit memory in the first implicit memory bank contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience, and an implicit memory in the first implicit memory bank is associated with an experience identifier;

[0194] The inference engine is used to retrieve the implicit memory associated with the first experience identifier from the first implicit memory bank in response to receiving a dialogue request carrying a first experience identifier; the dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform the first inference task; in response to retrieving the first implicit memory, the compressed key-value cache in the first implicit memory is decompressed into a first key-value cache, and the first inference task is performed based on the first key-value cache.

[0195] In an exemplary embodiment, the dialogue request carries current session interaction data; the agent also includes a common memory bank; the common memory bank is deployed in a common memory storage pool; a common memory in the common memory bank contains a common key-value cache corresponding to a common context content, and the access permissions of the common memory in the common memory bank are set to read-only.

[0196] The inference engine is also used to retrieve common memories that match the current session interaction data from the common memory bank based on the current session interaction data; when a matching common memory is retrieved, a common key-value cache is extracted from the matching common memory to obtain the current common key-value cache, wherein the first inference task is also executed based on the current common key-value cache.

[0197] In an exemplary embodiment, the inference engine is further configured to perform forward computation on the remaining session interaction data to obtain a second key-value cache corresponding to the remaining session interaction data when there is remaining session interaction data in the current session interaction data for which no matching common memory has been retrieved, wherein the first inference task is also executed based on the second key-value cache.

[0198] In an exemplary embodiment, the dialogue request carries a first session identifier, which is used to identify the first session; the dialogue request is sent by a first object, and the first implicit memory is the implicit memory of the first object; the agent also includes a personalized memory storage pool; the personalized memory storage pool includes an implicit memory sublayer and a time-sensitive memory sublayer; the implicit memory sublayer is used to store the first implicit memory; the time-sensitive memory sublayer is used to store the time-sensitive memory of the first object; one time-sensitive memory in the time-sensitive memory contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical session interaction data;

[0199] The inference engine is also used to retrieve time-sensitive memories associated with the first session from the time-sensitive memory bank based on the first session identifier; if the associated time-sensitive memory is retrieved, the compressed key-value cache in the associated time-sensitive memory is decompressed into a third key-value cache, wherein the first inference task is also executed based on the third key-value cache.

[0200] In an exemplary embodiment, the inference engine is further configured to scan the access frequency of time-sensitive memories in the time-sensitive memory bank; adjust the retention period of time-sensitive memories in the time-sensitive memory bank based on the access frequency of time-sensitive memories in the time-sensitive memory bank; and clean up expired time-sensitive memories in the time-sensitive memory bank in response to the existence of expired time-sensitive memories that have exceeded the retention period.

[0201] In an exemplary embodiment, the inference engine is further configured to extend the retention period of time-sensitive memories in the time-sensitive memory bank whose access frequency is greater than or equal to a specified frequency threshold to a specified retention period; and to adjust the retention period of time-sensitive memories in the time-sensitive memory bank according to the frequency-period correspondence based on the access frequency of the time-sensitive memories in the time-sensitive memory bank, wherein the frequency-period correspondence is the correspondence between access frequency and retention period.

[0202] In an exemplary embodiment, the inference engine is further configured to, in response to a received mode configuration instruction, configure the initial retention period of the time-sensitive memory in the time-sensitive memory bank according to the retention mode indicated by the mode configuration instruction, wherein the retention mode indicated by the mode configuration instruction is one of a plurality of retention modes, one of the multiple retention modes corresponds to one retention period, and the retention periods corresponding to different retention modes among the multiple retention modes are different from each other.

[0203] In one exemplary embodiment, the intelligent agent further includes a personalized memory storage pool; the personalized memory storage pool includes an implicit memory sublayer; the implicit memory sublayer is used to store a set of implicit memory libraries; one implicit memory library in the set of implicit memory libraries corresponds to one object;

[0204] The inference engine is used to perform at least one compression on the first key-value cache when the first key-value cache is to be compressed, so as to obtain the compressed representation vector corresponding to the first key-value cache.

[0205] The inference engine is also used to find compressed key-value caches in the implicit memory set that satisfy the vector similarity condition based on the compressed representation vector. The vector similarity condition is that the vector similarity between the target compressed key-value cache and the compressed representation vector is greater than or equal to a specified similarity threshold. If a target compressed key-value cache that satisfies the vector similarity condition exists in the implicit memory set, the difference vector between the target compressed key-value cache and the compressed representation vector, as well as the index pointer of the target compressed key-value cache, are determined as the compressed key-value cache corresponding to the first key-value cache. If no compressed key-value cache that satisfies the vector similarity condition exists in the implicit memory set, the compressed representation vector is determined as the compressed key-value cache corresponding to the first key-value cache.

[0206] In an exemplary embodiment, the inference engine is further configured to perform the following structured quantization compression on the first key-value cache to obtain an intermediate representation vector corresponding to the first key-value cache: compressing the first level in the first key-value cache according to a first quantization precision, and compressing the second level in the first key-value cache according to a second quantization precision; the number of levels in the first level is less than the number of levels in the second level, and the first quantization precision is lower than the second quantization precision; inputting the intermediate representation vector into a pre-trained neural network model to perform semantic compression on the intermediate representation vector through the neural network model to obtain a compressed representation vector.

[0207] In an exemplary embodiment, the dialogue request is sent by the first object, the first implicit memory is the implicit memory of the first object, and an implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience.

[0208] The inference engine is also used to generate a list of semantic fingerprint information based on the semantic fingerprints in the implicit memory in the first implicit memory bank and the experience identifiers associated with the implicit memory in the first implicit memory bank. Each semantic fingerprint in the list of semantic fingerprint information includes a semantic fingerprint and an experience identifier. The list of semantic fingerprint information is sent to the client of the first object. The list of semantic fingerprint information is used to determine the key-value cache reused for performing the inference task based on the similarity between the feature vector of the query statement input by the client of the first object and the semantic fingerprints in the list of semantic fingerprint information.

[0209] In an exemplary embodiment, the dialogue request is sent by a first object, and the first implicit memory is the implicit memory of the first object. An implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience. The agent further includes a personalized memory storage pool. The personalized memory storage pool includes an implicit memory sublayer. The implicit memory sublayer is used to store the first implicit memory. The implicit memory sublayer includes an object plaintext experience layer and a key-value cache experience layer. The key-value cache experience layer is used to store the first implicit memory. The object plaintext experience layer is used to store the object plaintext experience corresponding to the first object. The object plaintext experience library is used to store the historical experience of the first object in plaintext form, and the plaintext experience in the object plaintext experience library is associated with the implicit memory in the first implicit memory through an experience identifier.

[0210] The inference engine is also used to compress the data of the fourth key-value cache in response to the existence of a fourth key-value cache to be stored, so as to obtain a compressed key-value cache to be stored, wherein the fourth key-value cache is a key-value cache corresponding to the historical experience of the first object to be stored.

[0211] The inference engine is also used to obtain the semantic fingerprint to be stored, which is obtained by encoding the historical experience to be stored. Based on the compressed key-value cache to be stored and the semantic fingerprint to be stored, a second implicit memory to be stored is generated, the second implicit memory is stored in the first implicit memory bank, and the historical experience to be stored is stored in the plaintext experience bank of the object corresponding to the first object.

[0212] In one exemplary embodiment, the inference engine is further configured to encrypt the first key-value cache to obtain an encrypted first key-value cache.

[0213] In an exemplary embodiment, the inference engine is further configured to, in response to receiving a task continuation request, extract a second experience identifier, a second session identifier, and current context information from the task continuation request, wherein the second session identifier is used to identify a second session, the task continuation request is used to request the reuse of the key-value cache associated with the second experience identifier to continue the second inference task of the interrupted second session, and the current context information is context information generated from the time the interruption of the second inference task occurred to the time the request for task continuation was initiated; and retrieve an implicit memory associated with the second experience identifier from the second implicit memory, wherein an implicit memory in the second implicit memory contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience, and an implicit memory in the second implicit memory is associated with an experience identifier;

[0214] The inference engine is also used to decompress the compressed key-value cache in the third implicit memory into a fifth key-value cache in response to the retrieval of the third implicit memory;

[0215] The inference engine is also used to concatenate the fifth key-value cache with the key-value cache corresponding to the current context information, and to continue executing the second inference task based on the obtained concatenated key-value cache.

[0216] In an exemplary embodiment, the inference engine is further configured to, in response to receiving an experience transfer request from a second object, extract a third experience identifier and transfer authorization information from the experience transfer request, wherein the experience transfer request is used to request the transfer of implicit memory associated with the third experience identifier to the second object, and the transfer authorization information is used to authorize the transfer of implicit memory associated with the third experience identifier to the second object; in response to the successful verification of the transfer authorization information, retrieve the implicit memory associated with the third experience identifier from the third implicit memory bank, wherein an implicit memory in the third implicit memory bank includes a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience, and an implicit memory in the third implicit memory bank is associated with an experience identifier; in response to retrieving the fourth implicit memory, create a copy instance of the fourth implicit memory in the storage space of the second object, and bind the access permissions of the copy instance to the object identifier of the second object.

[0217] In an exemplary embodiment, the inference engine is further configured to, in response to receiving an experience publishing request from a third object, extract a fourth experience identifier from the experience publishing request, wherein the experience publishing request is used to request the publishing of implicit memory associated with the fourth experience identifier, and the implicit memory associated with the fourth experience identifier is the implicit memory of the third object; obtain experience metadata corresponding to the implicit memory associated with the fourth experience identifier to obtain experience metadata to be published; generate experience publishing information based on the fourth experience identifier and the experience metadata to be published, and publish the experience publishing information to the experience circulation platform.

[0218] It should be noted that the memory processing process of the above-mentioned intelligent agent has been explained in the above embodiments and will not be repeated here.

[0219] In one exemplary embodiment, this application also provides a client, configured to, during a first session, in response to receiving an input query statement, retrieve a historical experience identifier matching the input query statement; and in response to retrieving the first experience identifier, send a dialogue request carrying the first experience identifier to an intelligent agent, wherein the dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task.

[0220] In an exemplary embodiment, the client is provided with a local semantic fingerprint database. A semantic fingerprint in the local semantic fingerprint database is a feature vector obtained by encoding a historical experience. A semantic fingerprint in the local semantic fingerprint database is associated with an experience identifier.

[0221] The client is also used to convert the input query statement into a semantic feature vector and retrieve semantic fingerprints that meet the similarity conditions between the semantic feature vector and the local semantic fingerprint database; in response to the retrieval of similar semantic fingerprints that meet the similarity conditions, the empirical identifier of the similar semantic fingerprint is determined as the first empirical identifier.

[0222] In an exemplary embodiment, the client is further configured to, during the second session, in response to the task interruption time of the second inference task exceeding a specified time threshold, save the working status information of the second session to a local interruption status file, wherein the working status information includes the second session identifier of the second session and the second experience identifier used by the second inference task;

[0223] The client is also used to generate a task continuation request when the second inference task is interrupted and needs to be resumed, and send the task continuation request to the agent. The task continuation request carries a second experience identifier, a second session identifier, and current context information. The task continuation request is used to request the reuse of the key-value cache associated with the second experience identifier to resume the interrupted second inference task. The current context information is the context information generated from the time the second inference task was interrupted to the time the task continuation request was initiated.

[0224] It should be noted that the memory processing process of the client has been explained in the above embodiments and will not be repeated here.

[0225] In one exemplary embodiment, this application also provides an intelligent agent memory processing method, by... Figure 1 Agent 104 in the middle executes, such as Figure 5 As shown, it includes the following steps:

[0226] Step S502: In response to receiving a dialogue request carrying a first experience identifier, retrieve the implicit memory associated with the first experience identifier from the first implicit memory bank. The implicit memory in the first implicit memory bank includes a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience. The implicit memory in the first implicit memory bank is associated with an experience identifier. The dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform the first inference task.

[0227] In step S504, in response to the retrieval of the first implicit memory, the compressed key-value cache in the first implicit memory is decompressed into a first key-value cache, and a first inference task is performed based on the first key-value cache.

[0228] In an exemplary embodiment, the dialogue request carries current session interaction data; before executing the first inference task based on the first key-value cache, the method further includes: retrieving a common memory matching the current session interaction data from a common memory bank based on the current session interaction data, wherein a common memory in the common memory bank contains a common key-value cache corresponding to a common context content, and the access permissions of the common memory in the common memory bank are set to read-only; the common memory bank is deployed in a common memory storage pool; if a matching common memory is retrieved, the common key-value cache is extracted from the matching common memory to obtain the current common key-value cache, wherein the first inference task is also executed based on the current common key-value cache.

[0229] In an exemplary embodiment, before executing the first inference task based on the first key-value cache, the method further includes: if there is remaining session interaction data in the current session interaction data for which no matching common memory has been retrieved, performing forward calculation on the remaining session interaction data to obtain a second key-value cache corresponding to the remaining session interaction data, wherein the first inference task is also executed based on the second key-value cache.

[0230] In an exemplary embodiment, the dialogue request carries a first session identifier, which is used to identify the first session; before executing the first inference task based on the first key-value cache, the method further includes: retrieving a time-sensitive memory associated with the first session from a time-sensitive memory bank based on the first session identifier, wherein a time-sensitive memory in the time-sensitive memory bank contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical session interaction data; if an associated time-sensitive memory is retrieved, decompressing the compressed key-value cache in the associated time-sensitive memory into a third key-value cache, wherein the first inference task is also executed based on the third key-value cache.

[0231] In an exemplary embodiment, the method further includes: scanning the access frequency of time-sensitive memories in the time-sensitive memory bank; adjusting the retention period of time-sensitive memories in the time-sensitive memory bank based on the access frequency of time-sensitive memories in the time-sensitive memory bank; and cleaning up expired time-sensitive memories in response to the existence of expired time-sensitive memories in the time-sensitive memory bank that have exceeded the retention period.

[0232] In one exemplary embodiment, the retention period of time-sensitive memories in the time-sensitive memory bank is adjusted based on the access frequency of time-sensitive memories in the time-sensitive memory bank, including one of the following:

[0233] The retention period of time-sensitive memories in the time-sensitive memory bank whose access frequency is greater than or equal to a specified frequency threshold is extended to the specified retention period;

[0234] Based on the access frequency of time-sensitive memories in the time-sensitive memory bank, the retention period of time-sensitive memories in the time-sensitive memory bank is adjusted according to the frequency-period correspondence. The frequency-period correspondence is the correspondence between access frequency and retention period.

[0235] In one exemplary embodiment, the above method further includes:

[0236] In response to the received mode configuration instruction, the initial retention period of the time-sensitive memory in the time-sensitive memory bank is configured according to the retention mode indicated by the mode configuration instruction. The retention mode indicated by the mode configuration instruction is one of multiple retention modes, one of the multiple retention modes corresponds to one retention period, and the retention periods corresponding to different retention modes among the multiple retention modes are different from each other.

[0237] In one exemplary embodiment, the above method further includes:

[0238] When the first key-value cache is to be compressed, it is compressed at least once to obtain the compressed representation vector corresponding to the first key-value cache. Based on the compressed representation vector, a compressed key-value cache that satisfies the vector similarity condition in the implicit memory set is searched. In this case, one implicit memory in the implicit memory set corresponds to one object, and the vector similarity condition is that the vector similarity between the implicit memory and the compressed key-value cache is greater than or equal to a specified similarity threshold. The implicit memory set is deployed in the implicit memory sublayer. The implicit memory sublayer is a sublayer of the personalized memory storage pool. If there is a target compressed key-value cache that satisfies the vector similarity condition in the implicit memory set, the difference vector between the target compressed key-value cache and the compressed representation vector, as well as the index pointer of the target compressed key-value cache, are determined as the compressed key-value cache corresponding to the first key-value cache. If there is no compressed key-value cache that satisfies the vector similarity condition in the implicit memory set, the compressed representation vector is determined as the compressed key-value cache corresponding to the first key-value cache.

[0239] In one exemplary embodiment, performing at least one compression on a first key-value cache to obtain a compressed representation vector corresponding to the first key-value cache includes: performing the following structured quantization compression on the first key-value cache to obtain an intermediate representation vector corresponding to the first key-value cache: compressing a first level in the first key-value cache according to a first quantization precision, and compressing a second level in the first key-value cache according to a second quantization precision; the number of levels in the first level is less than the number of levels in the second level, and the first quantization precision is lower than the second quantization precision; inputting the intermediate representation vector into a pre-trained neural network model to perform semantic compression on the intermediate representation vector through the neural network model to obtain a compressed representation vector.

[0240] In an exemplary embodiment, the dialogue request is sent by a first object, the first implicit memory is the implicit memory of the first object, and an implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience; before retrieving the implicit memory associated with the first experience identifier from the first implicit memory in response to receiving a dialogue request carrying a first experience identifier, the method further includes: generating a semantic fingerprint information list based on the semantic fingerprint in the implicit memory in the first implicit memory and the experience identifier associated with the implicit memory in the first implicit memory, wherein a semantic fingerprint information in the semantic fingerprint information list includes a semantic fingerprint and an experience identifier; sending the semantic fingerprint information list to the client of the first object, wherein the semantic fingerprint information list is used to determine the key-value cache reused for performing the inference task based on the similarity between the feature vector of the query statement input by the client of the first object and the semantic fingerprint in the semantic fingerprint information list.

[0241] In an exemplary embodiment, the dialogue request is sent by the first object, the first implicit memory is the implicit memory of the first object, and an implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience.

[0242] The above method further includes: in response to the existence of a fourth key-value cache to be stored, compressing the fourth key-value cache to obtain a compressed key-value cache to be stored, wherein the fourth key-value cache is a key-value cache corresponding to the historical experience to be stored of the first object; obtaining a semantic fingerprint to be stored, wherein the semantic fingerprint to be stored is obtained by encoding the historical experience to be stored; generating a second implicit memory to be stored based on the compressed key-value cache to be stored and the semantic fingerprint to be stored, storing the second implicit memory in the first implicit memory bank, and storing the historical experience to be stored in the object plaintext experience bank corresponding to the first object, wherein the object plaintext experience bank is used to store the historical experience of the first object in plaintext form, and the plaintext experience in the object plaintext experience bank is associated with the implicit memory in the first implicit memory bank through experience identifiers.

[0243] In an exemplary embodiment, before performing the first inference task based on the first key-value cache, the method further includes: encrypting the first key-value cache to obtain an encrypted first key-value cache, and transmitting the encrypted first key-value cache to the inference engine through an encrypted channel, wherein the inference engine is the engine that performs the first inference task.

[0244] In an exemplary embodiment, the method further includes: in response to receiving a task continuation request, extracting a second experience identifier, a second session identifier, and current context information from the task continuation request, wherein the second session identifier is used to identify a second session, the task continuation request is used to request the reuse of a key-value cache associated with the second experience identifier to continue the second inference task of the interrupted second session, and the current context information is context information generated from the time the second inference task was interrupted to the time the task continuation request was initiated; retrieving an implicit memory associated with the second experience identifier from a second implicit memory bank, wherein an implicit memory in the second implicit memory bank contains a compressed key-value cache obtained by compressing a key-value cache corresponding to a historical experience, and an implicit memory in the second implicit memory bank is associated with an experience identifier; in response to retrieving a third implicit memory, decompressing the compressed key-value cache in the third implicit memory into a fifth key-value cache; concatenating the fifth key-value cache with the key-value cache corresponding to the current context information, and continuing to execute the second inference task based on the obtained concatenated key-value cache.

[0245] In an exemplary embodiment, the method further includes: in response to receiving an experience transfer request from a second object, extracting a third experience identifier and transfer authorization information from the experience transfer request, wherein the experience transfer request is used to request the transfer of implicit memory associated with the third experience identifier to the second object, and the transfer authorization information is used to authorize the transfer of implicit memory associated with the third experience identifier to the second object; in response to the successful verification of the transfer authorization information, retrieving implicit memory associated with the third experience identifier from a third implicit memory bank, wherein an implicit memory in the third implicit memory bank includes a compressed key-value cache obtained by compressing a key-value cache corresponding to a historical experience, and an implicit memory in the third implicit memory bank is associated with an experience identifier; in response to retrieving a fourth implicit memory, creating a copy instance of the fourth implicit memory in the storage space of the second object, and binding the access permissions of the copy instance to the object identifier of the second object.

[0246] In an exemplary embodiment, the method further includes: in response to receiving an experience publishing request from a third object, extracting a fourth experience identifier from the experience publishing request, wherein the experience publishing request is used to request the publishing of an implicit memory associated with the fourth experience identifier, and the implicit memory associated with the fourth experience identifier is the implicit memory of the third object; obtaining experience metadata corresponding to the implicit memory associated with the fourth experience identifier to obtain experience metadata to be published; generating experience publishing information based on the fourth experience identifier and the experience metadata to be published, and publishing the experience publishing information to the experience circulation platform.

[0247] It should be noted that the above-mentioned agent-executed memory processing method has already been explained in the agent's memory processing process, and will not be repeated here.

[0248] In one exemplary embodiment, this application also provides an intelligent agent memory processing method, by... Figure 1 Client 102 executes, such as Figure 6 As shown, it includes the following steps:

[0249] Step S602, during the first session, in response to receiving an input query statement, retrieve historical experience identifiers that match the input query statement;

[0250] Step S604: In response to the retrieval of the first experience identifier, a dialogue request carrying the first experience identifier is sent to the intelligent agent, wherein the dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform the first inference task.

[0251] In one exemplary embodiment, retrieving historical experience identifiers that match the input query statement includes:

[0252] The input query statement is converted into a semantic feature vector, and the semantic feature vector is compared with the semantic fingerprint stored in the local semantic fingerprint database for similarity retrieval. The semantic fingerprint in the local semantic fingerprint database is a feature vector obtained by encoding a historical experience, and a semantic fingerprint in the local semantic fingerprint database is associated with an experience identifier.

[0253] In response to the retrieval of similar semantic fingerprints, the empirical identifier associated with the similar semantic fingerprints is determined as the first empirical identifier.

[0254] In one exemplary embodiment, the above method further includes:

[0255] During the second session, in response to the task interruption time of the second inference task exceeding the specified time threshold, the working status information of the second session is saved to the local interruption status file. The working status information includes the second session identifier of the second session and the second experience identifier used by the second inference task.

[0256] In the case of resuming the interrupted second inference task, a task resumption request is generated and sent to the agent. The task resumption request carries a second experience identifier, a second session identifier, and current context information. The task resumption request is used to request the reuse of the key-value cache associated with the second experience identifier to resume the interrupted second inference task. The current context information is the context information generated from the time the second inference task was interrupted to the time the task resumption request was initiated.

[0257] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0258] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (e.g., read-only memory (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0259] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor is configured to perform the steps of any of the method embodiments described above via the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor. The electronic device may be a server or a client.

[0260] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0261] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein the program executes the steps in any of the above method embodiments when it is run.

[0262] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0263] According to another aspect of the embodiments of this application, a computer program product is also provided, the computer program product including a computer program, which, when executed by a processor, implements the steps in any of the embodiments of the intelligent agent memory processing method described above.

[0264] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described embodiments of the intelligent agent memory processing method.

[0265] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0266] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0267] The data interaction system, intelligent agent, and intelligent agent memory processing method provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data interaction system, characterized in that, include: The system comprises an intelligent agent and a client. The intelligent agent includes an inference engine and a first implicit memory. An implicit memory in the first implicit memory contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience. Each implicit memory in the first implicit memory is associated with an experience identifier. The client is configured to, during the first session, in response to receiving an input query statement, retrieve a historical experience identifier that matches the input query statement; and in response to retrieving the first experience identifier, send a dialogue request carrying the first experience identifier to the agent, wherein the dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task. The inference engine is configured to, in response to receiving the dialogue request, retrieve the implicit memory associated with the first experience identifier from the first implicit memory bank; in response to retrieving the first implicit memory, decompress the compressed key-value cache in the first implicit memory into a first key-value cache, and execute the first inference task based on the first key-value cache.

2. The system according to claim 1, characterized in that, The client is equipped with a local semantic fingerprint database. A semantic fingerprint in the local semantic fingerprint database is a feature vector obtained by encoding a historical experience. A semantic fingerprint in the local semantic fingerprint database is associated with an experience identifier. The client is further configured to convert the input query statement into a semantic feature vector, and retrieve semantic fingerprints that satisfy similarity conditions with the semantic feature vector from the local semantic fingerprint database; in response to retrieving similar semantic fingerprints that satisfy the similarity conditions, the client determines the experience identifier of the similar semantic fingerprint as the first experience identifier.

3. The system according to claim 1, characterized in that, The client is also configured to, during the second session, in response to the task interruption time of the second inference task exceeding a specified time threshold, save the working status information of the second session to a local interruption status file, wherein the working status information includes the second session identifier of the second session and the second experience identifier used by the second inference task; The client is further configured to generate a task continuation request when the second inference task is to be resumed after an interruption, and send the task continuation request to the agent. The task continuation request carries a second experience identifier, a second session identifier, and current context information. The task continuation request is used to request the reuse of the key-value cache associated with the second experience identifier to resume the interrupted second inference task. The current context information is context information generated from the time the second inference task is interrupted to the time the task continuation request is initiated.

4. The system according to claim 1, characterized in that, The dialogue request carries the current session interaction data; the agent also includes a common memory library; the common memory library is deployed in a common memory storage pool; a common memory in the common memory library contains a common key-value cache corresponding to a common context content, and the access permissions of the common memory in the common memory library are set to read-only. The inference engine is further configured to retrieve common memories matching the current session interaction data from the common memory bank based on the current session interaction data. If a matching common memory is retrieved, a common key-value cache is extracted from the matching common memory to obtain the current common key-value cache. The first inference task is also executed based on the current common key-value cache.

5. The system according to claim 4, characterized in that, The inference engine is further configured to perform forward calculations on the remaining session interaction data when there is no matching common memory in the current session interaction data, to obtain a second key-value cache corresponding to the remaining session interaction data, wherein the first inference task is also executed based on the second key-value cache.

6. The system according to claim 5, characterized in that, The dialogue request carries a first session identifier, which is used to identify the first session; the dialogue request is sent by a first object, and the first implicit memory is the implicit memory of the first object; the agent also includes a personalized memory storage pool; the personalized memory storage pool includes an implicit memory sublayer and a time-sensitive memory sublayer; the implicit memory sublayer is used to store the first implicit memory; the time-sensitive memory sublayer is used to store the time-sensitive memory of the first object; one time-sensitive memory in the time-sensitive memory contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical session interaction data; The inference engine is further configured to retrieve time-sensitive memories associated with the first session from the time-sensitive memory bank based on the first session identifier; and, if an associated time-sensitive memory is retrieved, decompress the compressed key-value cache in the associated time-sensitive memory into a third key-value cache, wherein the first inference task is further executed based on the third key-value cache.

7. The system according to claim 6, characterized in that, The inference engine is also used to scan the access frequency of time-sensitive memories in the time-sensitive memory bank; adjust the retention period of time-sensitive memories in the time-sensitive memory bank based on the access frequency of time-sensitive memories in the time-sensitive memory bank; and clean up the expired time-sensitive memories in the time-sensitive memory bank in response to the existence of expired time-sensitive memories that have exceeded the retention period.

8. The system according to claim 7, characterized in that, The inference engine is also used to perform one of the following: The retention period of time-sensitive memories in the time-sensitive memory bank whose access frequency is greater than or equal to a specified frequency threshold is extended to the specified retention period; Based on the access frequency of the time-sensitive memories in the time-sensitive memory bank, the retention period of the time-sensitive memories in the time-sensitive memory bank is adjusted according to the frequency-period correspondence, wherein the frequency-period correspondence is the correspondence between the access frequency and the retention period.

9. The system according to claim 7, characterized in that, The inference engine is further configured to, in response to a received mode configuration instruction, configure the initial retention period of the time-sensitive memory in the time-sensitive memory bank according to the retention mode indicated by the mode configuration instruction, wherein the retention mode indicated by the mode configuration instruction is one of a plurality of retention modes, one of the plurality of retention modes corresponds to one retention period, and the retention periods corresponding to different retention modes among the plurality of retention modes are different from each other.

10. The system according to claim 1, characterized in that, The intelligent agent also includes a personalized memory storage pool; the personalized memory storage pool includes an implicit memory sublayer; the implicit memory sublayer is used to store a set of implicit memory libraries; one implicit memory library in the set of implicit memory libraries corresponds to one object; The inference engine is further configured to: perform at least one compression on the first key-value cache to obtain a compressed representation vector corresponding to the first key-value cache when the first key-value cache is to be compressed; based on the compressed representation vector, search for compressed key-value caches in the implicit memory set that satisfy a vector similarity condition, wherein the vector similarity condition is that the vector similarity between the compressed key-value cache and the first key-value cache is greater than or equal to a specified similarity threshold; if a target compressed key-value cache satisfying the vector similarity condition exists in the implicit memory set, determine the difference vector between the target compressed key-value cache and the compressed representation vector, and the index pointer of the target compressed key-value cache, as the compressed key-value cache corresponding to the first key-value cache; if no compressed key-value cache satisfying the vector similarity condition exists in the implicit memory set, determine the compressed representation vector as the compressed key-value cache corresponding to the first key-value cache.

11. The system according to claim 10, characterized in that, The inference engine is further configured to perform the following structured quantization compression on the first key-value cache to obtain an intermediate representation vector corresponding to the first key-value cache: compressing the first level in the first key-value cache according to a first quantization precision, and compressing the second level in the first key-value cache according to a second quantization precision; the number of levels in the first level is less than the number of levels in the second level, and the first quantization precision is lower than the second quantization precision; inputting the intermediate representation vector into a pre-trained neural network model to perform semantic compression on the intermediate representation vector through the neural network model to obtain the compressed representation vector.

12. The system according to claim 1, characterized in that, The dialogue request is sent by the first object, the first implicit memory is the implicit memory of the first object, and an implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience. The inference engine is further configured to generate a semantic fingerprint information list based on the semantic fingerprints in the implicit memory in the first implicit memory bank and the experience identifiers associated with the implicit memory in the first implicit memory bank, wherein each semantic fingerprint information in the semantic fingerprint information list includes a semantic fingerprint and an experience identifier; and to send the semantic fingerprint information list to the client of the first object, wherein the semantic fingerprint information list is used to determine the key-value cache reused for performing the inference task based on the similarity between the feature vector of the query statement input by the client of the first object and the semantic fingerprints in the semantic fingerprint information list.

13. The system according to claim 1, characterized in that, The dialogue request is sent by the first object, and the first implicit memory is the implicit memory of the first object. One implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience. The intelligent agent also includes a personalized memory storage pool. The personalized memory storage pool includes an implicit memory sublayer. The implicit memory sublayer is used to store the first implicit memory. The implicit memory sublayer includes an object plaintext experience layer and a key-value cache experience layer. The key-value cache experience layer is used to store the first implicit memory. The object plaintext experience layer is used to store the object plaintext experience library corresponding to the first object; the object plaintext experience library is used to store the historical experience of the first object in plaintext form, and the plaintext experience in the object plaintext experience library is associated with the implicit memory in the first implicit memory library through experience identifiers; The inference engine is also configured to, in response to the existence of a fourth key-value cache to be stored, compress the data of the fourth key-value cache to obtain a compressed key-value cache to be stored, wherein the fourth key-value cache is a key-value cache corresponding to the historical experience to be stored of the first object. The inference engine is further configured to obtain a semantic fingerprint to be stored, wherein the semantic fingerprint to be stored is obtained by encoding the historical experience to be stored; based on the compressed key-value cache to be stored and the semantic fingerprint to be stored, a second implicit memory to be stored is generated, the second implicit memory is stored in the first implicit memory library, and the historical experience to be stored is stored in the object plaintext experience library.

14. The system according to claim 1, characterized in that, The inference engine is also used to encrypt the first key-value cache to obtain an encrypted first key-value cache.

15. The system according to claim 1, characterized in that, The inference engine is further configured to, in response to receiving a task continuation request, extract a second experience identifier, a second session identifier, and current context information from the task continuation request. The second session identifier identifies a second session, and the task continuation request requests the reuse of a key-value cache associated with the second experience identifier to continue the second inference task of the interrupted second session. The current context information is context information generated between the time the second inference task is interrupted and the time the task continuation request is initiated. The engine also retrieves an implicit memory associated with the second experience identifier from a second implicit memory bank. An implicit memory in the second implicit memory bank contains a compressed key-value cache obtained by compressing a key-value cache corresponding to a historical experience. Each implicit memory in the second implicit memory bank is associated with an experience identifier. The inference engine is also configured to decompress the compressed key-value cache in the third implicit memory into a fifth key-value cache in response to the retrieval of the third implicit memory; The inference engine is further configured to concatenate the fifth key-value cache with the key-value cache corresponding to the current context information, and continue to execute the second inference task based on the obtained concatenated key-value cache.

16. The system according to any one of claims 1 to 15, characterized in that, The inference engine is further configured to, in response to receiving an experience transfer request from a second object, extract a third experience identifier and transfer authorization information from the experience transfer request, wherein the experience transfer request is used to request the transfer of implicit memory associated with the third experience identifier to the second object, and the transfer authorization information is used to authorize the transfer of implicit memory associated with the third experience identifier to the second object; in response to the successful verification of the transfer authorization information, retrieve implicit memory associated with the third experience identifier from the third implicit memory bank, wherein an implicit memory in the third implicit memory bank includes a compressed key-value cache obtained by compressing a key-value cache corresponding to a historical experience, and an implicit memory in the third implicit memory bank is associated with an experience identifier; in response to the retrieval of a fourth implicit memory, create a copy instance of the fourth implicit memory in the storage space of the second object, and bind the access permissions of the copy instance to the object identifier of the second object.

17. The system according to claim 16, characterized in that, The inference engine is further configured to, in response to receiving an experience publishing request from a third object, extract a fourth experience identifier from the experience publishing request, wherein the experience publishing request is used to request the publishing of implicit memory associated with the fourth experience identifier, and the implicit memory associated with the fourth experience identifier is the implicit memory of the third object; obtain experience metadata corresponding to the implicit memory associated with the fourth experience identifier to obtain experience metadata to be published; generate experience publishing information based on the fourth experience identifier and the experience metadata to be published, and publish the experience publishing information to the experience circulation platform.

18. An intelligent agent, characterized in that, include: The inference engine and the first implicit memory bank, wherein an implicit memory in the first implicit memory bank contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience, and an implicit memory in the first implicit memory bank is associated with an experience identifier; The inference engine is configured to, in response to receiving a dialogue request carrying a first experience identifier, retrieve an implicit memory associated with the first experience identifier from the first implicit memory bank; the dialogue request is configured to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task; in response to retrieving the first implicit memory, decompress the compressed key-value cache in the first implicit memory into a first key-value cache, and perform the first inference task based on the first key-value cache.

19. The intelligent agent according to claim 18, characterized in that, The dialogue request carries the current session interaction data; the agent also includes a common memory library; the common memory library is deployed in a common memory storage pool; a common memory in the common memory library contains a common key-value cache corresponding to a common context content, and the access permissions of the common memory in the common memory library are set to read-only. The inference engine is further configured to retrieve common memories matching the current session interaction data from the common memory bank based on the current session interaction data; if a matching common memory is retrieved, a common key-value cache is extracted from the matching common memory to obtain the current common key-value cache, wherein the first inference task is also executed based on the current common key-value cache.

20. The intelligent agent according to claim 19, characterized in that, The inference engine is further configured to perform forward calculations on the remaining session interaction data when there is no matching common memory in the current session interaction data, to obtain a second key-value cache corresponding to the remaining session interaction data, wherein the first inference task is also executed based on the second key-value cache.

21. The intelligent agent according to claim 20, characterized in that, The dialogue request carries a first session identifier, which is used to identify the first session; the dialogue request is sent by a first object, and the first implicit memory is the implicit memory of the first object; the agent also includes a personalized memory storage pool; the personalized memory storage pool includes an implicit memory sublayer and a time-sensitive memory sublayer; the implicit memory sublayer is used to store the first implicit memory; the time-sensitive memory sublayer is used to store the time-sensitive memory of the first object; one time-sensitive memory in the time-sensitive memory contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical session interaction data; The inference engine is further configured to retrieve time-sensitive memories associated with the first session from the time-sensitive memory bank based on the first session identifier; and, if an associated time-sensitive memory is retrieved, decompress the compressed key-value cache in the associated time-sensitive memory into a third key-value cache, wherein the first inference task is further executed based on the third key-value cache.

22. The intelligent agent according to claim 21, characterized in that, The inference engine is also used to scan the access frequency of time-sensitive memories in the time-sensitive memory bank; adjust the retention period of time-sensitive memories in the time-sensitive memory bank based on the access frequency of time-sensitive memories in the time-sensitive memory bank; and clean up the expired time-sensitive memories in the time-sensitive memory bank in response to the existence of expired time-sensitive memories that have exceeded the retention period.

23. The intelligent agent according to claim 22, characterized in that, The inference engine is further configured to extend the retention period of time-sensitive memories in the time-sensitive memory bank whose access frequency is greater than or equal to a specified frequency threshold to a specified retention period; and to adjust the retention period of time-sensitive memories in the time-sensitive memory bank according to the frequency-period correspondence based on the access frequency of the time-sensitive memories in the time-sensitive memory bank, wherein the frequency-period correspondence is the correspondence between access frequency and retention period.

24. The intelligent agent according to claim 22, characterized in that, The inference engine is further configured to, in response to a received mode configuration instruction, configure the initial retention period of the time-sensitive memory in the time-sensitive memory bank according to the retention mode indicated by the mode configuration instruction, wherein the retention mode indicated by the mode configuration instruction is one of a plurality of retention modes, one of the plurality of retention modes corresponds to one retention period, and the retention periods corresponding to different retention modes among the plurality of retention modes are different from each other.

25. The intelligent agent according to claim 18, characterized in that, The intelligent agent also includes a personalized memory storage pool; the personalized memory storage pool includes an implicit memory sublayer; the implicit memory sublayer is used to store a set of implicit memory libraries; one implicit memory library in the set of implicit memory libraries corresponds to one object; The inference engine is used to perform at least one compression on the first key-value cache when the first key-value cache is to be compressed, so as to obtain the compressed representation vector corresponding to the first key-value cache. The inference engine is further configured to, based on the compressed representation vector, search for compressed key-value caches in the implicit memory set that satisfy the vector similarity condition, wherein the vector similarity condition is that the vector similarity between the compressed key-value cache and the compressed key-value cache is greater than or equal to a specified similarity threshold; if a target compressed key-value cache satisfying the vector similarity condition exists in the implicit memory set, the difference vector between the target compressed key-value cache and the compressed representation vector, and the index pointer of the target compressed key-value cache are determined as the compressed key-value cache corresponding to the first key-value cache; if no compressed key-value cache satisfying the vector similarity condition exists in the implicit memory set, the compressed representation vector is determined as the compressed key-value cache corresponding to the first key-value cache.

26. The intelligent agent according to claim 25, characterized in that, The inference engine is further configured to perform the following structured quantization compression on the first key-value cache to obtain an intermediate representation vector corresponding to the first key-value cache: compressing the first level in the first key-value cache according to a first quantization precision, and compressing the second level in the first key-value cache according to a second quantization precision; the number of levels in the first level is less than the number of levels in the second level, and the first quantization precision is lower than the second quantization precision; inputting the intermediate representation vector into a pre-trained neural network model to perform semantic compression on the intermediate representation vector through the neural network model to obtain the compressed representation vector.

27. The intelligent agent according to claim 18, characterized in that, The dialogue request is sent by the first object, the first implicit memory is the implicit memory of the first object, and an implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience. The inference engine is further configured to generate a semantic fingerprint information list based on the semantic fingerprints in the implicit memory in the first implicit memory bank and the experience identifiers associated with the implicit memory in the first implicit memory bank, wherein each semantic fingerprint information in the semantic fingerprint information list includes a semantic fingerprint and an experience identifier; and to send the semantic fingerprint information list to the client of the first object, wherein the semantic fingerprint information list is used to determine the key-value cache reused for performing the inference task based on the similarity between the feature vector of the query statement input by the client of the first object and the semantic fingerprints in the semantic fingerprint information list.

28. The intelligent agent according to claim 18, characterized in that, The dialogue request is sent by the first object, and the first implicit memory is the implicit memory of the first object. One implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience. The intelligent agent also includes a personalized memory storage pool. The personalized memory storage pool includes an implicit memory sublayer. The implicit memory sublayer is used to store the first implicit memory. The implicit memory sublayer includes an object plaintext experience layer and a key-value cache experience layer. The key-value cache experience layer is used to store the first implicit memory. The object plaintext experience layer is used to store the object plaintext experience library corresponding to the first object; the object plaintext experience library is used to store the historical experience of the first object in plaintext form, and the plaintext experience in the object plaintext experience library is associated with the implicit memory in the first implicit memory library through experience identifiers; The inference engine is also configured to, in response to the existence of a fourth key-value cache to be stored, compress the data of the fourth key-value cache to obtain a compressed key-value cache to be stored, wherein the fourth key-value cache is a key-value cache corresponding to the historical experience to be stored of the first object. The inference engine is further configured to obtain a semantic fingerprint to be stored, wherein the semantic fingerprint to be stored is obtained by encoding the historical experience to be stored; based on the compressed key-value cache to be stored and the semantic fingerprint to be stored, a second implicit memory to be stored is generated, the second implicit memory is stored in the first implicit memory bank, and the historical experience to be stored is stored in the plaintext experience bank corresponding to the first object.

29. The intelligent agent according to claim 18, characterized in that, The inference engine is also used to encrypt the first key-value cache to obtain an encrypted first key-value cache.

30. The intelligent agent according to claim 18, characterized in that, The inference engine is further configured to, in response to receiving a task continuation request, extract a second experience identifier, a second session identifier, and current context information from the task continuation request. The second session identifier identifies a second session, and the task continuation request requests the reuse of a key-value cache associated with the second experience identifier to continue the second inference task of the interrupted second session. The current context information is context information generated between the time the second inference task is interrupted and the time the task continuation request is initiated. The engine also retrieves an implicit memory associated with the second experience identifier from a second implicit memory bank. An implicit memory in the second implicit memory bank contains a compressed key-value cache obtained by compressing a key-value cache corresponding to a historical experience. Each implicit memory in the second implicit memory bank is associated with an experience identifier. The inference engine is also configured to decompress the compressed key-value cache in the third implicit memory into a fifth key-value cache in response to the retrieval of the third implicit memory; The inference engine is further configured to concatenate the fifth key-value cache with the key-value cache corresponding to the current context information, and continue to execute the second inference task based on the obtained concatenated key-value cache.

31. The intelligent agent according to any one of claims 18 to 30, characterized in that, The inference engine is further configured to, in response to receiving an experience transfer request from a second object, extract a third experience identifier and transfer authorization information from the experience transfer request, wherein the experience transfer request is used to request the transfer of implicit memory associated with the third experience identifier to the second object, and the transfer authorization information is used to authorize the transfer of implicit memory associated with the third experience identifier to the second object; in response to the successful verification of the transfer authorization information, retrieve implicit memory associated with the third experience identifier from the third implicit memory bank, wherein an implicit memory in the third implicit memory bank includes a compressed key-value cache obtained by compressing a key-value cache corresponding to a historical experience, and an implicit memory in the third implicit memory bank is associated with an experience identifier; in response to the retrieval of a fourth implicit memory, create a copy instance of the fourth implicit memory in the storage space of the second object, and bind the access permissions of the copy instance to the object identifier of the second object.

32. The intelligent agent according to claim 31, characterized in that, The inference engine is further configured to, in response to receiving an experience publishing request from a third object, extract a fourth experience identifier from the experience publishing request, wherein the experience publishing request is used to request the publishing of implicit memory associated with the fourth experience identifier, and the implicit memory associated with the fourth experience identifier is the implicit memory of the third object; obtain experience metadata corresponding to the implicit memory associated with the fourth experience identifier to obtain experience metadata to be published; generate experience publishing information based on the fourth experience identifier and the experience metadata to be published, and publish the experience publishing information to the experience circulation platform.

33. A client application, characterized in that: The client is configured to, during the first session, in response to receiving an input query statement, retrieve a historical experience identifier that matches the input query statement; and in response to retrieving the first experience identifier, send a dialogue request carrying the first experience identifier to the agent, wherein the dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task.

34. The client according to claim 33, characterized in that, The client is equipped with a local semantic fingerprint database. A semantic fingerprint in the local semantic fingerprint database is a feature vector obtained by encoding a historical experience. A semantic fingerprint in the local semantic fingerprint database is associated with an experience identifier. The client is further configured to convert the input query statement into a semantic feature vector, and retrieve semantic fingerprints that satisfy similarity conditions with the semantic feature vector from the local semantic fingerprint database; in response to retrieving similar semantic fingerprints that satisfy the similarity conditions, the client determines the experience identifier of the similar semantic fingerprint as the first experience identifier.

35. The client according to claim 33, characterized in that, The client is also configured to, during the second session, in response to the task interruption time of the second inference task exceeding a specified time threshold, save the working status information of the second session to a local interruption status file, wherein the working status information includes the second session identifier of the second session and the second experience identifier used by the second inference task; The client is further configured to generate a task continuation request when the second inference task is to be resumed after an interruption, and send the task continuation request to the agent. The task continuation request carries a second experience identifier, a second session identifier, and current context information. The task continuation request is used to request the reuse of the key-value cache associated with the second experience identifier to resume the interrupted second inference task. The current context information is context information generated from the time the second inference task is interrupted to the time the task continuation request is initiated.

36. A method for processing the memory of an intelligent agent, characterized in that, include: In response to receiving a dialogue request carrying a first experience identifier, an implicit memory associated with the first experience identifier is retrieved from a first implicit memory bank. The implicit memory in the first implicit memory bank includes a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience. Each implicit memory in the first implicit memory bank is associated with an experience identifier. The dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task. In response to the retrieval of the first implicit memory, the compressed key-value cache in the first implicit memory is decompressed into a first key-value cache, and the first inference task is executed based on the first key-value cache.

37. The method according to claim 36, characterized in that, The dialogue request carries the current session interaction data; Before executing the first inference task based on the first key-value cache, the method further includes: Based on the current session interaction data, a common memory matching the current session interaction data is retrieved from the common memory database. Each common memory in the common memory database contains a common key-value cache corresponding to a common context content. The access permissions of the common memories in the common memory database are set to read-only. The common memory database is deployed in a common memory storage pool. If a matching common memory is found, a common key-value cache is extracted from the matching common memory to obtain the current common key-value cache, wherein the first inference task is also executed based on the current common key-value cache.

38. The method according to claim 37, characterized in that, Before executing the first inference task based on the first key-value cache, the method further includes: If there is remaining session interaction data in the current session interaction data where no matching common memory is found, forward calculation is performed on the remaining session interaction data to obtain the second key-value cache corresponding to the remaining session interaction data, wherein the first inference task is also executed based on the second key-value cache.

39. The method according to claim 38, characterized in that, The dialogue request carries a first session identifier, which is used to identify the first session. Before executing the first inference task based on the first key-value cache, the method further includes: Based on the first session identifier, retrieve the time-sensitive memory associated with the first session from the time-sensitive memory bank, wherein a time-sensitive memory in the time-sensitive memory bank includes a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical session interaction data; If the associated time-sensitive memory is retrieved, the compressed key-value cache in the associated time-sensitive memory is decompressed into a third key-value cache, wherein the first inference task is also executed based on the third key-value cache.

40. The method according to claim 39, characterized in that, The method further includes: The access frequency of time-sensitive memories in the aforementioned time-sensitive memory bank is scanned; Based on the access frequency of the time-sensitive memories in the time-sensitive memory bank, the retention period of the time-sensitive memories in the time-sensitive memory bank is adjusted; In response to the existence of expired time-sensitive memories that have exceeded their retention period in the time-sensitive memory bank, the expired time-sensitive memories are cleared.

41. The method according to claim 40, characterized in that, The adjustment of the retention period of time-sensitive memories in the time-sensitive memory bank based on the access frequency of the time-sensitive memories in the time-sensitive memory bank includes one of the following: The retention period of time-sensitive memories in the time-sensitive memory bank whose access frequency is greater than or equal to a specified frequency threshold is extended to the specified retention period; Based on the access frequency of the time-sensitive memories in the time-sensitive memory bank, the retention period of the time-sensitive memories in the time-sensitive memory bank is adjusted according to the frequency-period correspondence, wherein the frequency-period correspondence is the correspondence between the access frequency and the retention period.

42. The method according to claim 40, characterized in that, The method further includes: In response to a received mode configuration instruction, the initial retention period of the time-sensitive memory in the time-sensitive memory bank is configured according to the retention mode indicated by the mode configuration instruction. The retention mode indicated by the mode configuration instruction is one of a plurality of retention modes, one of the plurality of retention modes corresponds to one retention period, and the retention periods corresponding to different retention modes among the plurality of retention modes are different from each other.

43. The method according to claim 36, characterized in that, The method further includes: When the first key-value cache is to be compressed, the first key-value cache is compressed at least once to obtain the compressed representation vector corresponding to the first key-value cache; Based on the compressed representation vector, find compressed key-value caches in the implicit memory set that satisfy the vector similarity condition, wherein one implicit memory in the implicit memory set corresponds to one object, and the vector similarity condition is that the vector similarity between the cache and the compressed key-value cache is greater than or equal to a specified similarity threshold; the implicit memory set is deployed in the implicit memory sublayer; the implicit memory sublayer is a sublayer of the personalized memory storage pool; If a target compressed key-value cache that satisfies the vector similarity condition exists in the implicit memory set, the difference vector between the target compressed key-value cache and the compressed representation vector, as well as the index pointer of the target compressed key-value cache, are determined as the compressed key-value cache corresponding to the first key-value cache. If there is no compressed key-value cache that satisfies the vector similarity condition in the implicit memory set, the compressed representation vector is determined as the compressed key-value cache corresponding to the first key-value cache.

44. The method according to claim 43, characterized in that, The step of performing at least one compression on the first key-value cache to obtain the compressed representation vector corresponding to the first key-value cache includes: The first key-value cache is subjected to the following structured quantization compression to obtain the intermediate representation vector corresponding to the first key-value cache: the first level in the first key-value cache is compressed according to the first quantization precision, and the second level in the first key-value cache is compressed according to the second quantization precision; the number of levels in the first level is less than the number of levels in the second level, and the first quantization precision is lower than the second quantization precision; The intermediate representation vector is input into a pre-trained neural network model to perform semantic compression on the intermediate representation vector, thereby obtaining the compressed representation vector.

45. The method according to claim 36, characterized in that, The dialogue request is sent by the first object, the first implicit memory is the implicit memory of the first object, and an implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience. Before retrieving the implicit memory associated with the first experience identifier from the first implicit memory bank in response to receiving a dialogue request carrying the first experience identifier, the method further includes: Based on the semantic fingerprints in the implicit memory in the first implicit memory bank and the experience identifiers associated with the implicit memory in the first implicit memory bank, a semantic fingerprint information list is generated, wherein a semantic fingerprint information in the semantic fingerprint information list includes a semantic fingerprint and an experience identifier. The semantic fingerprint information list is sent to the client of the first object, wherein the semantic fingerprint information list is used to determine the key-value cache reused for performing the inference task based on the similarity between the feature vector of the query statement input by the client of the first object and the semantic fingerprints in the semantic fingerprint information list.

46. ​​The method according to claim 36, characterized in that, The dialogue request is sent by the first object, the first implicit memory is the implicit memory of the first object, and an implicit memory in the first implicit memory also contains a semantic fingerprint corresponding to a historical experience. The method further includes: In response to the existence of a fourth key-value cache to be stored, the fourth key-value cache is compressed to obtain a compressed key-value cache to be stored, wherein the fourth key-value cache is a key-value cache corresponding to the historical experience to be stored of the first object. Obtain the semantic fingerprint to be stored, wherein the semantic fingerprint to be stored is obtained by encoding the historical experience to be stored; Based on the compressed key-value cache to be stored and the semantic fingerprint to be stored, a second implicit memory to be stored is generated, the second implicit memory is stored in the first implicit memory bank, and the historical experience to be stored is stored in the object plaintext experience bank corresponding to the first object. The object plaintext experience bank is used to store the historical experience of the first object in plaintext form, and the plaintext experience in the object plaintext experience bank is associated with the implicit memory in the first implicit memory bank through experience identifiers.

47. The method according to claim 36, characterized in that, Before executing the first inference task based on the first key-value cache, the method further includes: The first key-value cache is encrypted to obtain an encrypted first key-value cache, and the encrypted first key-value cache is transmitted to the inference engine through an encrypted channel, wherein the inference engine is the engine that executes the first inference task.

48. The method according to claim 36, characterized in that, The method further includes: In response to receiving a task continuation request, a second experience identifier, a second session identifier, and current context information are extracted from the task continuation request. The second session identifier is used to identify the second session. The task continuation request is used to request the reuse of the key-value cache associated with the second experience identifier to continue the second inference task of the interrupted second session. The current context information is the context information generated from the time when the second inference task was interrupted to the time when the task continuation request was initiated. Retrieve the implicit memory associated with the second experience identifier from the second implicit memory bank, wherein an implicit memory in the second implicit memory bank contains a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience, and an implicit memory in the second implicit memory bank is associated with an experience identifier. In response to the retrieval of the third implicit memory, the compressed key-value cache in the third implicit memory is decompressed into a fifth key-value cache; The fifth key-value cache is concatenated with the key-value cache corresponding to the current context information, and the second inference task is continued based on the obtained concatenated key-value cache.

49. The method according to any one of claims 36 to 48, characterized in that, The method further includes: In response to receiving an experience transfer request from a second object, a third experience identifier and transfer authorization information are extracted from the experience transfer request, wherein the experience transfer request is used to request the transfer of implicit memory associated with the third experience identifier to the second object, and the transfer authorization information is used to authorize the transfer of implicit memory associated with the third experience identifier to the second object; In response to the successful verification of the transfer authorization information, an implicit memory associated with the third experience identifier is retrieved from the third implicit memory bank. The implicit memory in the third implicit memory bank includes a compressed key-value cache obtained by compressing the key-value cache corresponding to a historical experience. The implicit memory in the third implicit memory bank is associated with an experience identifier. In response to the retrieval of the fourth implicit memory, a copy instance of the fourth implicit memory is created in the storage space of the second object, and the access permissions of the copy instance are bound to the object identifier of the second object.

50. The method according to claim 49, characterized in that, The method further includes: In response to receiving an experience publishing request from a third object, a fourth experience identifier is extracted from the experience publishing request, wherein the experience publishing request is used to request the publishing of implicit memory associated with the fourth experience identifier, and the implicit memory associated with the fourth experience identifier is the implicit memory of the third object. Obtain the experience metadata corresponding to the implicit memory associated with the fourth experience identifier to obtain the experience metadata to be published; Based on the fourth experience identifier and the metadata of the experience to be published, experience publishing information is generated and published to the experience circulation platform.

51. A method for processing the memory of an intelligent agent, characterized in that, include: During the first session, in response to receiving an input query, historical experience identifiers that match the input query are retrieved; In response to the retrieval of a first experience identifier, a dialogue request carrying the first experience identifier is sent to the agent, wherein the dialogue request is used to request the reuse of the key-value cache corresponding to the historical experience identified by the first experience identifier to perform a first inference task.

52. The method according to claim 51, characterized in that, The retrieval of historical experience identifiers that match the input query includes: The input query statement is converted into a semantic feature vector, and the semantic feature vector is compared with the semantic fingerprints stored in the local semantic fingerprint database for similarity retrieval. The semantic fingerprint in the local semantic fingerprint database is a feature vector obtained by encoding a historical experience, and a semantic fingerprint in the local semantic fingerprint database is associated with an experience identifier. In response to the retrieval of similar semantic fingerprints, the experience identifier associated with the similar semantic fingerprints is determined as the first experience identifier.

53. The method according to claim 51, characterized in that, The method further includes: During the second session, in response to the task interruption time of the second inference task exceeding a specified time threshold, the working status information of the second session is saved to a local interruption status file, wherein the working status information includes the second session identifier of the second session and the second experience identifier used by the second inference task; In the case of resuming the interrupted second inference task, a task resumption request is generated and sent to the agent. The task resumption request carries a second experience identifier, a second session identifier, and current context information. The task resumption request is used to request the reuse of the key-value cache associated with the second experience identifier to resume the interrupted second inference task. The current context information is the context information generated from the time the interruption of the second inference task occurred to the time the task resumption request was initiated.

Images