AI agent memory hierarchical management system based on distributed cloud storage

Abstract

The application discloses an AI intelligent agent memory hierarchical management system based on distributed cloud storage and belongs to the technical field of computer AI intelligence. The system comprises a local running module, a cloud memory storage module and an intelligent synchronization coordination module. The local running module is used for capturing environment information, executing inference decision and maintaining short-term memory cache. The cloud memory storage module is arranged in a distributed cloud computing environment and comprises a distributed memory storage cluster adopting a consistent hashing algorithm for data distribution management and used for persistent storage and indexing of memory data. The intelligent synchronization coordination module is in communication connection with the local running module and the cloud memory storage module respectively and is used for monitoring memory data changes, executing data synchronization, detecting cross-node concurrent update conflicts and performing arbitration processing. The application solves the problems of data islands and cross-regional cooperation difficulties in the prior art and realizes efficient synchronization and intelligent hierarchical management of local and cloud memories.

Description

Technical Field

[0001] This invention belongs to the technical field of computer AI intelligence, specifically relating to an AI intelligent agent memory hierarchical management system based on distributed cloud storage. Background Technology

[0002] The rapid development of artificial intelligence technology has driven the widespread application of AI agent systems. As intelligent entities capable of autonomously perceiving their environment, reasoning, making decisions, and executing tasks, one of the core capabilities of AI agents lies in their ability to continuously learn and understand context through memory mechanisms. Locally running AI agent systems aim to support agents in operating for extended periods in their local environment, accumulating experience and knowledge, thereby improving the accuracy and adaptability of their task processing.

[0003] As a key component of AI agents, the memory module is responsible for storing and managing various types of data generated during the agent's operation, including dialogue history, user preferences, task execution records, and domain knowledge. This memory data provides crucial support for the agent's reasoning and decision-making, and its storage architecture design directly affects the agent's performance and practical application value.

[0004] Existing memory storage solutions mostly employ local hierarchical storage or a single knowledge base. While local hierarchical storage can achieve data classification and management to some extent, it lacks effective correlation mechanisms between different levels, making it difficult to form a unified knowledge view. The single knowledge base approach, limited by storage capacity and access efficiency, cannot meet the application needs of large-scale data scenarios. More importantly, none of these solutions utilize a distributed cloud storage architecture, preventing AI agent systems in different regions or on different nodes from achieving real-time sharing and collaborative work of memory data. This creates data silos, severely limiting the large-scale application of AI agent systems in cross-regional and cross-platform scenarios. Furthermore, existing solutions lack efficient synchronization mechanisms between local caching and cloud storage, as well as intelligent hierarchical management methods for memory data, making it difficult to simultaneously meet the needs of low-latency local response and large-scale persistent cloud storage.

[0005] Therefore, there is an urgent need for an AI agent system that can integrate distributed cloud storage memory, support continuous local operation, and achieve efficient synchronization and intelligent hierarchical management to overcome the above-mentioned technical problems. Summary of the Invention

[0006] The purpose of this invention is to provide an AI intelligent agent memory hierarchical management system based on distributed cloud storage, so as to solve the problems of data silos, lack of hierarchical association and difficulty in cross-regional collaboration in the existing memory storage architecture.

[0007] The technical solution of the present invention includes: A local execution module, deployed on the terminal device, is used to capture environmental information, perform inference decisions, and maintain a short-term memory cache; A cloud-based memory storage module, deployed in a distributed cloud computing environment, is used for persistent storage of memory data, index building, and semantic association management. The cloud-based memory storage module includes a distributed memory storage cluster, which uses a consistent hashing algorithm for data distribution management and achieves data redundancy backup and load balancing through multiple storage nodes deployed across regions. An intelligent synchronization and coordination module is communicatively connected to both the local operation module and the cloud memory storage module. The intelligent synchronization and coordination module is used to monitor changes in the memory data in the local running module, and to perform data synchronization operations between the local running module and the cloud memory storage module when changes are detected. The intelligent synchronization and coordination module is also used to arbitrate when a concurrent update conflict across nodes is detected, based on a preset conflict resolution strategy. The intelligent synchronization and coordination module includes a memory aging unit; The memory aging unit is used to calculate a comprehensive importance score based on the access frequency and timeliness attributes of memory entries, and to store the memory data in the distributed memory storage cluster in different storage layers based on the comprehensive importance score.

[0008] Furthermore, the cloud memory storage module also includes a memory index engine; The memory index engine includes an inverted index sub-engine and a vector index sub-engine; The inverted index sub-engine is used to segment the text content of each memory entry into independent terms, and to establish a mapping relationship for each term to a list of memory entry identifiers containing that term, so as to support precise keyword-based retrieval. The vector indexing sub-engine is used to convert the semantic content of memory entries into high-dimensional vectors using a pre-trained language model, and to construct a multi-layer graph index structure for the semantic vectors of all memory entries using a hierarchical navigable small-world graph algorithm, so as to support approximate retrieval based on semantic similarity. The memory indexing engine also supports combined search modes; The combined retrieval mode first uses the inverted index sub-engine to filter out a set of candidate memory entries through keyword matching, and then uses the vector index sub-engine to sort the semantic vectors in the set of candidate memory entries by similarity and return the retrieval results.

[0009] Furthermore, the cloud memory storage module also includes a privacy protection unit; The privacy protection unit is used to encrypt sensitive memory data using a ciphertext policy attribute-based encryption algorithm before storing it in the distributed memory storage cluster. The workflow of the privacy protection unit performing encryption processing includes: determining an access strategy represented by an access tree structure based on the sensitivity level and access control requirements of the sensitive memory data; The privacy protection unit uses the system master public key and the access policy to encrypt the sensitive memory data to generate ciphertext, and stores the ciphertext in association with the access policy; When an access request carrying a set of requester attributes and a corresponding private key is received, the privacy protection unit determines whether the set of requester attributes satisfies the access policy embedded in the ciphertext. If the conditions are met, the privacy protection unit decrypts and returns the original memory data; If the conditions are not met, the privacy protection unit rejects the access request.

[0010] Furthermore, each storage node in the distributed memory storage cluster is configured with a local data management subsystem and a replica synchronization subsystem; The local data management subsystem is used to manage the data shards stored on this node and maintain the data version information and access logs; The replica synchronization subsystem is used to synchronize data with other storage nodes; The default number of data replicas is 3, stored on one master node and two slave nodes; The master node is responsible for handling write operations, and the slave nodes synchronize the data changes of the master node through an asynchronous replication mechanism. The default time interval for asynchronous replication is 1 second. The distributed memory storage cluster uses the heartbeat mechanism in the Raft protocol for master node election; When the master node fails, the distributed memory storage cluster automatically elects a slave node as the new master node to continue providing services.

[0011] Furthermore, the intelligent synchronization and coordination module also includes a conflict arbitration unit; The conflict arbitration unit is used to determine that a conflict has occurred when multiple update requests with the same version number for the same data entry are received, and to select the winning version based on a preset conflict resolution strategy. The conflict resolution strategies include a timestamp priority strategy and an authority node priority strategy; The timestamp priority strategy is used to compare the timestamps of each concurrent update operation and select the data version with the latest timestamp as the final result; The authoritative node priority strategy is used to prioritize the data version of the higher-authority node as the conflict arbitration result based on the preset authority level of each storage node. After completing the arbitration, the conflict arbitration unit generates an arbitration result notification. The arbitration result notification includes a unique identifier for the conflicting data, the data content of the winning version, and a timestamp of the winning version; The conflict arbitration unit updates the data version number to the current maximum value plus 1.

[0012] Furthermore, the intelligent synchronization and coordination module also includes a change detection unit and a synchronization execution unit; The change detection unit is used to monitor data changes in the local cache unit of the local running module through a Hook mechanism or a polling mechanism. The change detection unit generates a change record when it detects data addition, data modification, or data deletion operations. The change record includes a unique data identifier, change type, data snapshot before the change, data snapshot after the change, and change timestamp; The synchronization execution unit is used to maintain the synchronization task queue and supports two types of data synchronization operations: uplink synchronization and downlink synchronization. The uplink synchronization is used to synchronize new or updated data in the local cache unit to the cloud memory storage module. The downlink synchronization is used to download the retrieval results or the winning version of the conflict arbitration results from the cloud memory storage module to the local cache unit; When the synchronization execution unit detects that the target node is unreachable while executing a synchronization task, it marks the synchronization task as waiting for retry and retryes it using an exponential backoff method. The synchronous execution unit verifies data integrity using an MD5 checksum after data transmission is completed.

[0013] Furthermore, the memory aging unit in the intelligent synchronization and coordination module periodically scans the memory entries in the distributed memory storage cluster; The formula for calculating the overall importance score of the memory aging unit is as follows: Overall importance score = 0.4 × Normalized access frequency + 0.3 × Normalized reciprocal recent access time + 0.2 × Normalized semantic importance + 0.1 × Normalized creation time freshness; The normalized semantic importance is determined by the weighted sum of the number of in-degree edges and the number of out-degree edges of a node in the knowledge association network; The rules for hierarchical storage management of memory data by the memory aging unit based on the comprehensive importance score are as follows: Memory entries with an overall importance score of 0.7 or higher are stored in a high-performance storage layer composed of solid-state drives; Memory entries with an overall importance score between 0.3 and 0.7 are stored in a low-cost storage layer consisting of mechanical hard drives; Memory entries with an overall importance score between 0.1 and 0.3 and a last access time of more than 90 days are compressed and migrated to the archive layer composed of object storage services; Memory entries with an overall importance score below 0.1 and a last access time exceeding 365 days are migrated to a cold storage layer consisting of a distributed file system or tape storage.

[0014] Furthermore, the local operation module includes an environment perception unit, a reasoning and decision-making unit, and a local cache unit; The environmental sensing unit is used to continuously capture external environmental information and internal state data, and to perform standardized preprocessing on the captured data; The standardized preprocessing includes data format unification, timestamp marking, priority marking, and noise reduction filtering; The reasoning decision unit is used to perform semantic understanding and knowledge matching based on the short-term memory data stored in the local cache unit, and to select the optimal action strategy based on the reasoning results. The semantic understanding uses a pre-trained language model to convert the input text into semantic vectors; The knowledge matching is achieved by calculating the cosine similarity between the semantic vector and the semantic vector of the memory entry in the local cache unit. Memory entries with a similarity exceeding a preset threshold of 0.7 are identified as matching items. The local cache unit is used to temporarily store the interaction data and state variables generated by the agent in the current running cycle, and maintain an elimination queue based on elimination priority scores. When the cache space is insufficient, the local cache unit prioritizes clearing the data entry with the lowest eviction priority score.

[0015] Furthermore, the system also includes a multi-agent collaboration module; The multi-agent collaboration module is used to coordinate memory sharing and collaborative reasoning among multiple locally running instances; The multi-agent collaboration module maintains a collaborative task scheduler; The collaborative task scheduler breaks down the overall task into multiple sub-tasks based on the task dependency graph; The collaborative task scheduler evaluates the capability score of each instance based on the number of CPU cores, available memory, CPU utilization, and network bandwidth reported by each locally running instance. The collaborative task scheduler prioritizes instances with higher capability scores and shorter current task queues to assign the sub-tasks. The multi-agent collaboration module also supports collaboration session management, which is used to establish persistent collaboration sessions between multiple locally running instances. Shared memory data generated during the session is automatically synchronized to the cloud memory storage module for access by all session participants.

[0016] Furthermore, the specific implementation method of using the consistent hashing algorithm for data distribution management in the distributed memory storage cluster is as follows: The entire storage space is abstracted as a hash ring with a numerical range of 0 to 2^32-1; The hash value of the storage node identifier is calculated using the secure hash algorithm 256, and the first 32 bits are taken as the position of the node on the hash ring. When new memory data is written, the hash value of the unique identifier of the data is calculated and the first 32 bits are taken. The first storage node in the clockwise direction on the hash ring is then used as the storage node to which the data belongs. The consistent hashing algorithm also supports virtual node technology; Each physical storage node corresponds to multiple virtual nodes, and the number of virtual nodes is dynamically adjusted according to the hardware performance of the physical node and the business load. The adjustment rules are as follows: collect the CPU utilization and network bandwidth utilization of each physical node every 5 minutes; when the load score exceeds 60%, add 10 virtual nodes to that physical node; when the load score is below 30%, reduce 10 virtual nodes; and each physical node shall retain at least 20 virtual nodes. The load score is calculated as follows: Load score = 0.6 × CPU utilization + 0.4 × network bandwidth utilization.

[0017] In summary, this application includes at least one of the following beneficial technical effects: 1. By setting up a distributed memory storage cluster, this invention achieves unified storage and management of memory data across regions and nodes, effectively breaking through the data silo limitations formed by traditional local hierarchical storage and single knowledge base methods. This enables intelligent agent systems in different regions to share memory data in real time, significantly improving the large-scale application capability of AI intelligent agent systems in cross-regional scenarios.

[0018] 2. By setting up a knowledge association network and using a graph database structure to establish semantic relationships between memory entries, this invention can form a unified knowledge view, effectively making up for the lack of effective association mechanisms between different levels in the local hierarchical storage method, enabling the agent to obtain more complete and related knowledge support during the reasoning and decision-making process.

[0019] 3. By setting up a memory index engine, this invention supports both precise retrieval based on keywords and approximate retrieval based on semantic similarity, enabling efficient information location in large-scale memory data scenarios. This effectively solves the application bottleneck of single knowledge base methods being limited by storage capacity and access efficiency.

[0020] 4. By setting up an intelligent synchronization and coordination module, this invention realizes automatic synchronization and maintenance between local cache and cloud storage, and is equipped with a conflict arbitration mechanism to ensure data consistency. This enables the AI ​​agent that runs continuously locally to make full use of the large-scale memory resources in the cloud for long-term continuous operation and experience accumulation while maintaining low local storage overhead. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall solution for an AI intelligent agent memory hierarchical management system based on distributed cloud storage; Figure 2 This is a schematic diagram illustrating the core principles of knowledge association networks and memory index engines; Figure 3 It is a logical flowchart of the local running module's environment awareness, reasoning decision-making, and local caching; Figure 4 This is a schematic diagram illustrating the distributed memory storage cluster, interaction relationships, and data flow of the cloud-based memory storage module; Figure 5 This is a logic diagram of the intelligent synchronization and coordination module's change detection, conflict arbitration, and synchronous execution. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1 To be continued Figure 5 The present invention will be further described in detail below with reference to specific embodiments.

[0023] Example 1 The AI ​​intelligent agent memory hierarchical management system proposed in this invention, based on distributed cloud storage, mainly consists of three core parts: a local operation module, a cloud memory storage module, and an intelligent synchronization coordination module. These three modules interact and synchronize their states through a network communication link. The local operation module is deployed in the terminal device where the intelligent agent runs, responsible for capturing environmental information in real time, executing inference decisions, and maintaining short-term memory caches. The cloud memory storage module is deployed in a distributed cloud computing environment, responsible for the persistent storage, index construction, and semantic association management of large-scale memory data. The intelligent synchronization coordination module acts as a bridge connecting the local and cloud environments, responsible for monitoring data changes, arbitrating synchronization conflicts, and executing specific data synchronization operations. The internal composition, working principle, and data flow relationships of each module will be explained in detail below.

[0024] The first module is the local execution module, which includes an environment awareness unit, a reasoning and decision-making unit, and a local cache unit. The composition and working logic of these three units are explained below.

[0025] The environmental perception unit, acting as the front-end data acquisition layer, is responsible for continuously capturing external environmental information and internal state data during the agent's operation. External environmental information includes user-input commands, objective environmental parameters collected by sensors, cooperative signals sent by neighboring agents, and event notifications pushed by other external systems. Internal state data encompasses the agent's own operational state variables, intermediate results of the current inference context, traceability information of the decision path, and summary tags of historical interaction records.

[0026] After collecting the above data, the environmental sensing unit first performs standardization preprocessing. The preprocessing includes the following: Unified data format: Convert data from different sources into a unified data format agreed upon within this implementation, such as JSON or Protocol Buffers.

[0027] Timestamp annotation: Add a UTC timestamp of the time of collection to each data entry, accurate to the millisecond level.

[0028] Priority Labeling: Each piece of data is assigned a priority level based on its source type and business urgency. Priority levels are divided into high, medium, and low. User-input commands are assigned high priority by default, sensor parameters and cooperative signals are assigned medium priority by default, and event notifications and status summaries are assigned low priority by default.

[0029] Noise reduction and filtering: For sensor data, a sliding window midpoint filtering algorithm is used to remove abnormal fluctuation values; for text data, invalid character filtering and whitespace character compression are used based on regular expressions.

[0030] The processed data is pushed to the inference decision unit and the local cache unit, respectively.

[0031] The above process transforms raw data into usable input, providing a standardized data foundation for subsequent inference decisions and short-term caching.

[0032] The reasoning and decision-making unit is the core processing engine of the locally running module. Upon receiving input data pushed by the environmental awareness unit, the reasoning and decision-making unit performs real-time task processing and reasoning decisions based on the short-term memory data stored in the local cache unit.

[0033] The reasoning and decision-making unit integrates a reasoning engine and a decision-making algorithm module.

[0034] The inference engine is responsible for semantic understanding and knowledge matching of the input data. Semantic understanding uses a pre-trained language model (such as a lightweight variant of BERT or RoBERTa) to convert the input text into semantic vectors. Knowledge matching is achieved by calculating the cosine similarity between the semantic vectors and the semantic vectors of the stored entries. Memory entries with a similarity exceeding a preset threshold of 0.7 are identified as matches.

[0035] The decision-making algorithm module selects the optimal action strategy based on the reasoning results. This method predefines a set of action strategies, each associated with a scoring function. The inputs to the scoring function include the confidence level of the matching memory entry for the current task, the success rate of historical execution strategies, and the current system resource status. The decision-making algorithm module iterates through all candidate strategies, calculates their comprehensive scores, and selects the strategy with the highest score as the current optimal action strategy.

[0036] During the reasoning and decision-making process, the reasoning and decision-making unit can initiate a memory query request to the memory index engine in the cloud memory storage module. The memory query request is transmitted to the cloud via the intelligent synchronization and coordination module. In response to the memory query request, the memory index engine retrieves memory data relevant to the current reasoning task from the knowledge association network and returns the retrieval results to the reasoning and decision-making unit as a decision reference. This working mode, combining local reasoning with cloud knowledge retrieval, enables the agent to make more accurate reasoning decisions while maintaining low-latency local response and utilizing large-scale cloud memory resources.

[0037] The local cache unit serves as a short-term memory storage unit within the local runtime module, temporarily storing interaction data and state variables generated by the agent during the current runtime cycle. The storage capacity of the local cache unit is set to a limited value, which is configured based on the hardware resources of the terminal device and the application scenario requirements of the agent. In this embodiment, it is typically set to 20% to 30% of the device's available storage space.

[0038] When the occupancy rate of the local cache unit exceeds a preset threshold, the intelligent synchronization coordination module triggers an active synchronization process, pushing the data in the local cache unit to the cloud memory storage module for persistent storage. After being synchronized to the cloud, the data in the local cache unit is marked as persistent, and the local copy can be retained or deleted according to a policy.

[0039] The local cache unit also maintains an eviction queue. Each cache entry in the eviction queue corresponds to an eviction priority score, which is calculated as follows: Priority score for elimination = Access time decay factor × Importance score The access time decay factor is calculated as the current time minus the last access time of the entry, in hours, with an upper limit of 168 hours (7 days). The importance score is calculated as follows: Importance score = 0.5 × Normalized access frequency + 0.3 × Reciprocal of normalized last access time + 0.2 × User preset weight.

[0040] Normalized access frequency refers to the number of times an entry has been accessed in the past 7 days divided by the highest number of accesses for all entries in this embodiment; normalized reciprocal of the most recent access time refers to the reciprocal of the last access time of an entry from the current time, normalized to the range of [0,1]; user-preset weights are manually marked by the user or upper-layer application according to the business importance of the remembered entries, and the value range is 0 to 1.

[0041] When cache space is insufficient, the eviction queue prioritizes removing data entries with the lowest eviction priority score. If multiple entries have the same score, they are further removed in order of access time from earliest to latest.

[0042] In summary, the above three units together constitute the complete working logic of the local operation module: the environment perception unit completes data collection and standardized preprocessing, the reasoning and decision-making unit uses local short-term memory and cloud retrieval to achieve semantic understanding, knowledge matching and action strategy selection, and the local caching unit provides short-term storage, proactive synchronization triggering and cache eviction mechanism based on eviction priority score.

[0043] The local operation module enables the AI ​​agent to run continuously in the local environment and provides front-end support for subsequent data interaction with the cloud memory storage module and the intelligent synchronization and coordination module, ensuring the low-latency response and experience accumulation capabilities of the entire system.

[0044] Next is the cloud-based memory storage module, which serves as the memory data center for the entire system. Its design goal is to provide high-capacity, highly available, and easily searchable distributed memory storage services for multiple locally running instances. The cloud-based memory storage module consists of four parts: a distributed memory storage cluster, a memory indexing engine, a knowledge association network, and a privacy protection unit. These components work together to achieve unified management and efficient access to memory data.

[0045] This is the foundational storage layer for distributed memory storage clusters, serving as the base for cloud-based memory storage modules. The distributed memory storage cluster employs an architecture that deploys multiple storage nodes across geographical regions, with these nodes interconnected via a network to achieve data redundancy backup and load balancing.

[0046] In this embodiment, the distributed memory storage cluster uses a consistent hashing algorithm for data distribution management. The implementation of the consistent hashing algorithm is as follows: The entire storage space is abstracted as a hash ring, with values ​​ranging from 0 to 2^32 - 1, where 2^32 - 1 represents 2 to the power of 32 minus 1. The SHA-256 hash function is used to calculate the hash value of the storage node identifier, and the first 32 bits of the hash value are taken as the node's position on the hash ring. Each storage node is mapped to a point on the hash ring through the hash function.

[0047] Each position on the hash ring corresponds to a storage region of a data shard. When new memory data is written, a hash value uniquely identifying the data is calculated. The first 32 bits of the hash value are taken, and the first storage node is found clockwise on the hash ring to determine which node belongs to the data. When a storage node is added or removed, only the data mapping relationship between adjacent nodes on the hash ring is affected. This design reduces the system overhead of data migration and ensures the stability of the system when nodes change dynamically.

[0048] The consistent hashing algorithm also supports virtual node technology. Each physical storage node can correspond to multiple virtual nodes, and the number of virtual nodes is dynamically adjusted based on the hardware performance of the physical nodes and the business load. The adjustment rules are as follows: CPU utilization and network bandwidth utilization of each physical node are collected every 5 minutes. When the load score of a physical node exceeds 60%, 10 virtual nodes are added to that physical node; when the load score is below 30%, 10 virtual nodes are removed, but each physical node retains at least 20 virtual nodes. The load score is calculated as follows: Load score = 0.6 × CPU utilization + 0.4 × Network bandwidth utilization.

[0049] Each storage node in the distributed memory storage cluster is configured with a local data management subsystem and a replica synchronization subsystem.

[0050] The local data management subsystem is responsible for managing the data shards stored on the storage nodes, supporting CRUD operations on data, and maintaining data version information and access logs. The replica synchronization subsystem is responsible for synchronizing data with other storage nodes to ensure that each data shard remains consistent across multiple nodes.

[0051] In this embodiment, the default number of data replicas is three, stored on one master node and two slave nodes. The master node handles write operations, and the slave nodes synchronize data changes to the master node through an asynchronous replication mechanism. The default time interval for asynchronous replication is one second. When the master node fails, a slave node is automatically elected as the new master node to continue providing services. The election algorithm uses the heartbeat mechanism from the Raft protocol.

[0052] The aforementioned distributed memory storage cluster achieves data distribution and redundant backup across geographically distributed storage nodes through a consistent hashing algorithm and a replica synchronization mechanism, providing highly available and scalable basic storage capabilities for memory data.

[0053] The memory index engine is the core retrieval layer of the cloud-based memory storage module. Its function is to build a structured index of the memory data stored in the distributed memory storage cluster to support fast retrieval and accurate location. The memory index engine consists of two sub-components: an inverted index sub-engine and a vector index sub-engine.

[0054] The inverted index sub-engine is used to build a keyword index based on the text content of memory entries. The inverted index sub-engine works by splitting the text content of each memory entry into independent terms according to spaces and punctuation marks, removing stop words, and using either a standard English stop word list or a Chinese stop word list.

[0055] For each term, a mapping is established to a list of memory entry identifiers that contain that term. Through the inverted index, the memory index engine can quickly locate all memory entries containing a specified keyword, enabling precise keyword-based retrieval. For example, when a user queries "image classification," the inverted index sub-engine directly returns a list of all memory entry identifiers containing the term "image classification."

[0056] The vector indexing sub-engine is used to map memory entries into high-dimensional vectors. The vector indexing sub-engine is implemented by using a pre-trained language model to vectorize the semantic content of memory entries, converting the text-based memory content into a dense vector representation.

[0057] The pre-trained language model used in this embodiment is the publicly available BERT-base model. Released by Google, the BERT-base model has been pre-trained on large general-purpose corpora, namely English Wikipedia and BookCorpus, and can be directly used by those skilled in the art without retraining. The output vector dimension of the BERT-base model is 768 dimensions.

[0058] The specific process of vectorization encoding is as follows: The text of the memory entry is input into the BERT-base model, and the vector corresponding to the [CLS] tag output by the last layer of the BERT-base model is taken as the semantic vector of the memory entry. This vector has a dimension of 768, and the value of each dimension is a floating-point number. For memory entries with a length of more than 512 tags, a truncation strategy is adopted to retain the first 512 tags.

[0059] During retrieval, an approximate nearest neighbor search algorithm is employed. This embodiment uses a hierarchical navigable small-world graph algorithm for approximate nearest neighbor search. The hierarchical navigable small-world graph algorithm pre-constructs a multi-level graph index structure for the semantic vectors of all memorized entries. During retrieval, it starts from the top level and greedily traverses the nodes in the graph index structure, gradually approaching the nearest neighbor of the query vector.

[0060] The query vector is generated in the same way as the memory entries: the query text is input into the BERT-base model, and the 768-dimensional vector corresponding to the [CLS] label output from the last layer of the BERT-base model is taken as the query vector. The hierarchical navigable small-world graph algorithm returns the K memory entries most similar to the query vector. In this embodiment, the default value of K is 10. Users can adjust the value of K through query parameters, with a range from 1 to 100.

[0061] The inverted index sub-engine and the vector index sub-engine can be used independently or in combination to meet different search needs. The combined approach involves first using the inverted index sub-engine for keyword matching to filter out a set of candidate memory entries; then using the vector index sub-engine to sort the semantic vectors in the candidate set by similarity, finally returning the most relevant results. This combination approach addresses both the need for precise keyword matching and semantic similarity ranking.

[0062] The knowledge association network is the knowledge organization layer of the cloud-based memory storage module, used to establish semantic relationships between different memory entries and form a unified knowledge view. In this embodiment, the knowledge association network is constructed using a graph database structure. Nodes in the graph database correspond to entity concepts in memory entries, and edges in the graph database correspond to semantic relationships between entity concepts.

[0063] The types of semantic relations include the following four: Containment relationship: Used to represent the hierarchical structure between entity concepts. For example, artificial intelligence includes machine learning, and machine learning includes deep learning.

[0064] Causal relationship: Used to represent the causal connection between entity concepts. For example, data preprocessing leads to improved model training performance.

[0065] Similarity relation: used to represent the degree of similarity or relevance between entity concepts. For example, image classification and object detection have a high degree of similarity in application scenarios.

[0066] Temporal relationship: Used to represent the temporal sequence of relationships between entity concepts. For example, data acquisition occurs before feature engineering.

[0067] The knowledge association network not only stores entity concepts and relational edges, but also maintains attribute information for each node, including the entity's descriptive text, creation time, access frequency, and importance score. The calculation method for the importance score is handled by the memory aging unit, and the specific calculation method is detailed in the memory aging unit of the intelligent synchronization and coordination module. This attribute information provides data support for the hierarchical storage management of the memory aging unit.

[0068] The aforementioned knowledge association network organizes discrete memory items into an interconnected knowledge system through a graph database structure and four types of semantic relations, enabling intelligent agents to obtain more complete and relevant knowledge support during the reasoning and decision-making process.

[0069] The privacy protection unit serves as a security layer for the cloud memory storage module, used to encrypt sensitive memory data before performing distributed storage. In this embodiment, the encryption process employs the Ciphertext Policy Attribute-Based Encryption Algorithm (CP-ABE). CP-ABE is a public-key cryptosystem characterized by the ability of the encryptor to encrypt data according to a preset access policy, and only those who meet the access policy can successfully decrypt the data.

[0070] The CP-ABE algorithm involves the following mathematical elements and parameters: Let G0 and GT be two bilinear groups of order p, and g be a generator of G0. The bilinear mapping e: G0 × G0 → GT satisfies bilinearity, nondegeneracy, and computability.

[0071] During system initialization, the attribute authority selects random numbers α and β to calculate the master public key PK = (g, g^β, e(g,g)^α) and the master private key MSK = (β, g^α).

[0072] Each user is assigned a set of attributes S, and the attribute authority generates a private key SK for the user based on the user's attribute set and master private key.

[0073] The workflow of the privacy protection unit is as follows: Step 1: Upon receiving sensitive memory data requiring protection, the privacy protection unit first determines the access policy based on the data's sensitivity level and access control requirements. The access policy is represented in the form of an access tree structure, where each leaf node corresponds to an attribute, and internal nodes are threshold operators. For example, the access policy can be represented as Role: Administrator OR Department: R&D Department, with the root node of the access tree being the OR gate, the left subtree being the attribute Role: Administrator, and the right subtree being the attribute Department: R&D Department.

[0074] Step 2: The privacy protection unit encrypts the data using the master public key (PK) and access policy. The encryption process generates a session key, which is used to encrypt the memory data. Then, the access policy is used to encrypt the session key, forming ciphertext. The ciphertext and access policy are stored together in a distributed memory storage cluster.

[0075] Step 3: When the local runtime module needs to access sensitive memory data, it submits an access request to the privacy protection unit, carrying the requester's attribute set and the corresponding private key SK. The privacy protection unit inputs the ciphertext and the requester's private key into the CP-ABE decryption algorithm. The CP-ABE decryption algorithm determines whether the requester's attribute set satisfies the access policy embedded in the ciphertext. If it does, the session key is successfully decrypted, and the ciphertext data is then decrypted using the session key, returning the decrypted original memory data; otherwise, the access request is rejected, and an error message is returned.

[0076] In this embodiment, the attribute authority center is deployed separately in a secure environment. It is responsible for generating the master public key and master private key, and issuing a private key corresponding to each legitimate user's attribute set. The issuance of the private key requires authentication.

[0077] The CP-ABE algorithm supports fine-grained access control for encrypted data, enabling flexible role-level and attribute-level access control management while ensuring data encryption.

[0078] The above four components together constitute the complete architecture of the cloud-based memory storage module: the distributed memory storage cluster provides cross-regional, highly available basic storage, and uses consistent hashing algorithm and virtual node technology to achieve data distribution and load balancing; the memory index engine supports precise keyword retrieval and semantic similarity approximate retrieval through inverted index sub-engine and vector index sub-engine respectively; the knowledge association network uses graph database to establish four semantic association relationships between memory entries to form a unified knowledge view; and the privacy protection unit achieves fine-grained data security protection through ciphertext policy attribute base encryption algorithm.

[0079] The cloud-based memory storage module provides a large-capacity, highly available, and easily searchable distributed memory storage service for multiple locally running instances. It works in conjunction with the intelligent synchronization and coordination module to achieve data synchronization and consistency maintenance between local cache and cloud storage.

[0080] The intelligent synchronization coordination module in this embodiment enables automatic synchronization and maintenance between local cache and cloud storage, and is equipped with a conflict arbitration mechanism to ensure data consistency. The intelligent synchronization coordination module consists of four parts: a change detection unit, a conflict arbitration unit, a synchronization execution unit, and a memory aging unit.

[0081] The change detection unit monitors data changes in the locally running modules in real time and identifies memory update events that need to be synchronized to the cloud. The change detection unit monitors data changes in the local cache unit through either a Hook mechanism or a polling mechanism. The Hook mechanism involves embedding a callback function in the local cache unit's write interface, which is triggered immediately when data is modified. The polling mechanism involves scanning the data version number of the local cache unit at fixed time intervals, with a default interval of 100 milliseconds.

[0082] When the change detection unit detects a data addition, modification, or deletion operation, it generates a corresponding change record. The change record includes the following fields: unique data identifier, change type, data snapshot before change, data snapshot after change, change timestamp, and change source.

[0083] The unique identifier for data is formed by concatenating the data type and the identifier ID; the change type can be either addition, modification, or deletion; the change timestamp is UTC time accurate to milliseconds; and the change source is the identifier of the local running instance that caused the change.

[0084] Change records are pushed to the synchronization execution queue. The change detection unit also maintains a change log to record all historical change events. The change log is stored on the local disk, and each log record contains all fields of the aforementioned change record as well as an auto-incrementing log sequence number. The change log is used for backtracking and auditing when needed.

[0085] The aforementioned change detection unit captures data changes through hooking or polling and records the change events in a structured manner, providing accurate input for subsequent synchronization operations.

[0086] The conflict arbitration unit is used to arbitrate concurrent update conflicts across nodes based on a preset conflict resolution strategy. In a distributed storage environment, multiple locally running instances may simultaneously update the same memory data, leading to data version conflicts. After detecting a conflict, the conflict arbitration unit selects the winning version according to the preset conflict resolution strategy.

[0087] The conflict detection method is as follows: Each memory data entry carries a version number, which is a 64-bit integer, initially set to 1, and incremented by 1 with each update. When the synchronization execution unit receives update requests from different local running instances, if the two requests target the same data entry with the same version number, a conflict is determined.

[0088] In this embodiment, the conflict resolution strategy includes the following two: Timestamp priority strategy: Compare the timestamps of each concurrent update operation and select the data version with the latest timestamp as the final result. The timestamp accuracy is in milliseconds. If multiple timestamps are exactly the same, further compare the UUIDs of the local running instances and select the version of the instance with the smaller UUID value.

[0089] Authority Node Priority Strategy: Each storage node has a preset authority level, which is an integer ranging from 1 to 10, with a default value of 5. The data version from the highest authority node is prioritized as the arbitration result for conflicts. Users configure different authority levels for different storage nodes; the higher the authority level, the higher the priority of the data version used. If the nodes involved in the conflict have the same authority level, the strategy reverts to timestamp priority.

[0090] After completing arbitration, the conflict arbitration unit generates an arbitration result notification. The notification includes a unique identifier for the conflicting data, the data content of the winning version, the timestamp of the winning version, and a list of participating node identifiers. The notification is then pushed to the synchronization execution unit, which simultaneously updates the data version information related to the conflict, updating the data version number to the current maximum value plus 1.

[0091] The aforementioned conflict arbitration unit detects conflicts by version number and provides two arbitration strategies to ensure eventual data consistency during concurrent updates across nodes.

[0092] The synchronization execution unit is the execution layer of the intelligent synchronization coordination module. Its function is to perform corresponding data synchronization operations based on the output results of the change detection unit and the conflict arbitration unit, so as to achieve consistency maintenance between local memory and cloud memory.

[0093] The synchronization execution unit maintains a synchronization task queue. Each task in the queue represents a data synchronization operation that needs to be performed. The task structure includes a task identifier, task type, unique identifier of the target data, synchronization direction, number of retries, and creation timestamp.

[0094] Synchronization tasks include two types: uplink synchronization and downlink synchronization. Uplink synchronization: Synchronizes new or updated data from the local cache to the cloud memory storage module. Uplink synchronization is triggered when the local cache occupancy rate exceeds a threshold, when change records generated by the change detection unit are queued, or when manually triggered by the user.

[0095] Downlink synchronization: Downloads retrieval results or synchronized data from the cloud memory storage module to the local cache unit. Downlink synchronization is typically triggered by a memory query request or a winning version update in a conflict arbitration result.

[0096] When executing synchronization tasks, the synchronous execution unit performs the following steps: Step 1: Check the network connectivity and service status of the target node. The synchronous execution unit sends a TCP probe packet to the target node's health check port, with a timeout set to 2 seconds. If a correct response is received within 2 seconds, the target node is considered reachable; otherwise, it is marked as unreachable.

[0097] Step 2: If the target node is unreachable, mark the synchronization task as waiting for retry and set a retry strategy. The retry strategy uses an exponential backoff method: the first retry is delayed by 1 second, the second retry by 2 seconds, the third retry by 4 seconds, and so on, with a maximum of 5 retries. After more than 5 retries, the task is marked as failed and an error log is recorded.

[0098] Step 3: If the target node is reachable, complete the data transmission according to the preset synchronization protocol. The synchronization protocol uses a custom binary protocol, which includes a handshake phase, a data transmission phase, and a verification phase. The handshake phase exchanges metadata for the synchronization task; the data transmission phase transmits data in 1MB block sizes; the verification phase verifies data integrity using an MD5 checksum.

[0099] The synchronous execution unit supports two modes: incremental synchronization and full synchronization. Incremental synchronization: Only changed data fields are synchronized. Field-level changes for each data entry are obtained by comparing before-and-after snapshots of the data in the change record, and only the changed fields and their new values ​​are transmitted. Incremental synchronization is suitable for scenarios with large amounts of data, reducing network transmission volume.

[0100] Full synchronization: Retransmits the complete data records, including all fields. Full synchronization is suitable for scenarios involving data corruption or initial synchronization.

[0101] Users select the appropriate synchronization mode based on the data volume and network conditions. By default, full synchronization is used for data entries smaller than 100KB, and incremental synchronization is used for data entries greater than or equal to 100KB.

[0102] The memory aging unit is the storage optimization layer of the intelligent synchronization and coordination module. It is used to calculate the importance score of each memory entry based on the access frequency and timeliness attributes of the memory entry, and to perform hierarchical storage management of memory data in the cloud memory storage cluster based on the importance score.

[0103] The memory aging unit periodically scans all memory entries in the cloud memory storage cluster, with a default scan cycle of 24 hours. For each memory entry, the memory aging unit reads access frequency, last access time, creation time, and semantic importance data from the knowledge association network. Semantic importance is determined by the weighted sum of the number of in-degree edges and the number of out-degree edges of a node in the knowledge association network, specifically calculated as: Semantic Importance = 0.6 × Number of In-Degree Edges + 0.4 × Number of Out-Degree Edges.

[0104] The memory aging unit calculates the overall importance score for each memory item using the following formula: Overall Importance Score = 0.4 × Normalized Access Frequency + 0.3 × Normalized Reciprocal Last Access Time + 0.2 × Normalized Semantic Importance + 0.1 × Normalized Creation Time Freshness The meaning and calculation method of each component are as follows: Normalized access frequency: The total number of accesses to a memory entry in the past 30 days, divided by the maximum number of accesses to all memory entries in this embodiment, with the result ranging from 0 to 1.

[0105] Normalized reciprocal of the most recent access time: Define the reciprocal of the most recent access time as 1 / current time minus the last access time, in days. Normalize the maximum value of the reciprocal of the most recent access time for all memory entries to obtain a score ranging from 0 to 1.

[0106] Normalized semantic importance: Divide the semantic importance value by the maximum semantic importance value of all memory entries in this embodiment to obtain a score ranging from 0 to 1.

[0107] Normalized Creation Time Freshness: Define Creation Time Freshness = 1 / Current Time minus Creation Time, in days. Normalize the maximum value of the creation time freshness for all memory entries to obtain a score ranging from 0 to 1.

[0108] The memory aging unit performs hierarchical storage management of memory data based on a comprehensive importance score, with the following hierarchical rules: High-performance storage tier: Memory entries with an overall importance score of 0.7 or higher are stored on solid-state drives. The high-performance storage tier provides millisecond-level read latency.

[0109] Low-cost storage tier: Memory entries with an overall importance score between 0.3 and 0.7 are stored on a hard disk drive. The low-cost storage tier provides read latency in the tens of milliseconds.

[0110] Archived layer: Memory entries with an overall importance score between 0.1 and 0.3 and a last access time of more than 90 days are migrated to object storage services, and the data is compressed and stored using the LZ4 compression algorithm.

[0111] Cold Storage Layer: Memory entries with an overall importance score below 0.1 and a last access time exceeding 365 days are migrated to a distributed file system or tape storage. When access to data in the cold storage layer is required, a data retrieval operation is triggered first to restore the data to the high-performance storage layer before providing service.

[0112] The migration strategy for memory aging units can be adjusted through configuration parameters. Users can set the threshold boundaries for the overall importance score and the data retention period for each storage layer according to actual business needs. In the default configuration, the retention period for the high-performance storage layer is 30 days, the retention period for the low-cost storage layer is 180 days, the retention period for the archive layer is 3 years, and the retention period for the cold storage layer is 10 years.

[0113] The aforementioned memory aging unit optimizes the use of storage resources by periodically calculating a comprehensive importance score, retaining high-frequency active data in the high-performance storage layer, and relegating low-frequency cold data to the low-cost storage layer.

[0114] In summary, the change detection unit captures local data changes and generates structured change records through hooking or polling mechanisms; the conflict arbitration unit detects concurrent conflicts through version numbers and provides two arbitration strategies: timestamp priority and authoritative node priority; the synchronization execution unit maintains a synchronization task queue, supports both uplink and downlink synchronization directions, and has two synchronization modes: exponential backoff retry and incremental full synchronization; the memory aging unit periodically calculates a comprehensive importance score and stores the memory data in layers based on the score, namely a high-performance layer, a low-cost layer, an archive layer, and a cold storage layer.

[0115] The intelligent synchronization and coordination module enables automatic synchronization and consistency assurance between local cache and cloud storage, providing a reliable bridge for data interaction between local running modules and cloud memory storage modules, and optimizing the storage cost and access efficiency of large-scale memory data.

[0116] The multi-agent collaboration module is an extended functional module in this embodiment, used to coordinate memory sharing and collaborative reasoning among multiple locally running instances. Each locally running instance shares public memory data in the cloud memory storage module through the intelligent synchronization coordination module, and completes differentiated task processing based on its own private memory data.

[0117] The multi-agent collaboration module maintains a collaborative task scheduler. The collaborative task scheduler receives collaboration requests from different local running instances, decomposes and allocates tasks according to the nature of the tasks and the capabilities of the participating instances, and distributes the decomposed subtasks to the corresponding local running instances for execution.

[0118] During task decomposition, the collaborative task scheduler breaks down the overall task into multiple subtasks based on the task dependency graph. For tasks that can be processed in parallel, they are divided into multiple data slices with equal input data volume, and each slice corresponds to a subtask. For tasks with sequential dependencies, they are divided into stages, with the next stage executed only after the previous stage is completed.

[0119] The capability assessment of participating instances is based on indicators such as the number of CPU cores, available memory, current CPU utilization, and network bandwidth reported by each instance. This embodiment uses a weighted summation method to calculate the capability score of each instance, with the weights adjusted according to the actual scenario. When allocating subtasks, instances with higher capability scores and shorter current task queues are prioritized for execution to achieve load balancing.

[0120] During collaborative inference, each locally running instance shares intermediate results and synchronizes its inference state through the multi-agent collaboration module. After completing a subtask, each instance uploads the generated intermediate data to the cloud memory storage module and notifies the collaborative task scheduler.

[0121] The inference state synchronization is implemented as follows: each instance sends a state update message to the collaborative task scheduler. The state update message includes the instance identifier, current execution progress, and a list of completed subtasks. The collaborative task scheduler maintains a global state table, which records the state snapshot and last update time of each instance using the collaborative task identifier as the key.

[0122] The multi-agent collaboration module triggers a conflict arbitration process when necessary. When multiple instances simultaneously perform write operations on the same memory data, causing version conflicts, the multi-agent collaboration module calls the conflict arbitration unit in the intelligent synchronization coordination module to arbitrate according to a preset conflict resolution strategy. When multiple instances produce different calculation results for the same intermediate result, a majority voting mechanism is used to select the result that appears most frequently as the final result.

[0123] The multi-agent collaboration module supports collaborative session management, enabling the establishment of persistent collaborative sessions between multiple locally running instances. Any locally running instance can initiate a session creation request, specifying participating instances and session parameters. After the session is established, shared memory data generated during the session is automatically synchronized to the cloud memory storage module for access by all session participants. Sessions support instance joining, leaving, and destroying operations, and have a timeout mechanism; after the timeout, the session is automatically destroyed to release resources.

[0124] The multi-agent collaboration module decomposes and allocates tasks through a collaborative task scheduler, supports multi-instance collaborative reasoning through intermediate result sharing and state synchronization, triggers an arbitration process to ensure data consistency in the event of conflicts, and supports long-term structured multi-agent collaborative work through collaborative session management. The multi-agent collaboration module enhances the processing capabilities of this embodiment in cross-instance collaboration scenarios, and works in conjunction with the intelligent synchronization and coordination module to achieve efficient collaboration in a multi-agent environment.

[0125] In terms of the system's workflow, when the agent starts running, the environmental perception unit first collects external environment and internal state data, and transmits the collected data to the inference and decision-making unit and the local cache unit. Upon receiving input data, the inference and decision-making unit first checks if relevant short-term memories exist in the local cache unit. If they do, it directly uses the local memory for inference and decision-making. If the local memory is insufficient to support the current inference task, it initiates a memory query request to the cloud memory storage module through the intelligent synchronization and coordination module. Upon receiving the query request, the memory index engine first performs keyword matching through the inverted index sub-engine to filter out a set of candidate memory entries, then sorts the candidate set by semantic similarity through the vector index sub-engine, and finally returns the most relevant memory data to the inference and decision-making unit. After obtaining the memory data from the cloud, the inference and decision-making unit combines it with the short-term memories in the local cache for comprehensive inference and generates a decision result. After the decision result is generated, the environmental perception unit updates the internal state data, and the inference and decision-making unit records the decision process as a new memory entry and stores it in the local cache unit. When the local cache unit occupancy rate exceeds the preset threshold, the intelligent synchronization coordination module will automatically trigger the synchronization process to synchronize the memory data in the local cache to the cloud memory storage module. If a data conflict is detected during the synchronization process, the conflict arbitration unit will arbitrate the data according to the preset strategy.

[0126] In terms of distributed storage architecture, each storage node in the distributed memory storage cluster is deployed with three core components: data storage service, index service, and management service. The data storage service is responsible for reading and writing physical data files. Its underlying storage engine can be implemented using different methods depending on performance requirements, such as LevelDB, RocksDB, or a self-developed distributed storage engine. The index service is responsible for maintaining the index structure of the data stored on its node, supporting fast data location and range queries. The management service is responsible for node status monitoring, heartbeat reporting, fault detection, and automatic recovery, among other operational functions. Data replication and status synchronization between storage nodes are performed via a high-speed network. The network protocol typically uses a custom binary protocol based on TCP or the gRPC protocol to ensure efficient and reliable data transmission.

[0127] In terms of lifecycle management of memory data, memory data goes through multiple stages from creation to destruction. During the creation stage, memory data is first written to the local cache unit, while corresponding index entries are generated and entity nodes in the knowledge association network are updated. During the active stage, memory data is frequently accessed and referenced, maintaining a high importance score, and the memory aging unit retains it in the high-performance storage layer. During the decay stage, as time progresses and access frequency decreases, the importance score of memory data gradually declines, and the memory aging unit migrates it to the low-cost storage layer. During the archiving stage, for memory data that has not been accessed for a long time, the system marks it as archived and compresses it for storage to save space. During the destruction stage, when memory data exceeds the set retention period or is actively deleted by the user, the system performs a secure erase operation to ensure the data is unrecoverable.

[0128] In terms of system security, in addition to the attribute-based encryption mechanism provided by the privacy protection unit, the system also implements security protection at multiple layers. At the network transmission layer, all cross-node data communication uses TLS encrypted channels to prevent data from being stolen or tampered with during transmission; at the identity authentication layer, the system adopts a token-based identity authentication mechanism, and each locally running instance needs to provide a valid authentication token when accessing cloud services. The authentication server verifies the token and issues access permissions; at the access control layer, the system supports both role-based access control and policy-based access control modes, allowing administrators to configure granular access permissions for different users and instances; at the audit log layer, the system records all data access operations and administrator operations, and the audit logs are persistently stored and can be queried and exported.

[0129] In terms of system reliability, the distributed memory storage cluster ensures data persistence through multi-replica redundant storage and data verification mechanisms. Each data shard is stored with three copies by default, distributed across different storage nodes. When a node fails, the system automatically uses other copies to provide services and rebuilds the copies on the new nodes through a data reconstruction mechanism. The memory index engine and knowledge association network also employ similar redundant backup strategies to ensure high availability of the index structure and graph database. Furthermore, the intelligent synchronization coordination module supports breakpoint resumption. When synchronization operations fail due to network interruptions or system failures, the system records the synchronization progress and resumes the synchronization task from the breakpoint after the fault is recovered, avoiding duplicate or missed synchronization.

[0130] In terms of system scalability, the cloud-based memory storage module adopts a horizontally scalable architecture, allowing for easy expansion of storage capacity and throughput by simply adding storage nodes. The memory index engine supports distributed index building, with multiple index sub-engines processing indexing tasks in parallel, and the performance of the indexing service improving linearly with the number of nodes. The knowledge association network employs a distributed graph database architecture, where graph data is sharded and stored across multiple nodes, enabling graph traversal and relationship query operations to be executed in parallel across these nodes. The multi-agent collaboration module supports dynamically adding and removing locally running instances, and the system automatically adjusts the allocation strategy for collaborative tasks based on the current load, ensuring system responsiveness and stability under high concurrency scenarios.

[0131] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0132] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An AI intelligent agent memory hierarchical management system based on distributed cloud storage, characterized in that, include: A local execution module, deployed on the terminal device, is used to capture environmental information, perform inference decisions, and maintain a short-term memory cache; A cloud-based memory storage module, deployed in a distributed cloud computing environment, is used for persistent storage of memory data, index building, and semantic association management. The cloud-based memory storage module includes a distributed memory storage cluster, which uses a consistent hashing algorithm for data distribution management and achieves data redundancy backup and load balancing through multiple storage nodes deployed across regions. An intelligent synchronization and coordination module is communicatively connected to both the local operation module and the cloud memory storage module. The intelligent synchronization and coordination module is used to monitor changes in the memory data in the local running module, and to perform data synchronization operations between the local running module and the cloud memory storage module when changes are detected. The intelligent synchronization and coordination module is also used to arbitrate when a concurrent update conflict across nodes is detected, based on a preset conflict resolution strategy. The intelligent synchronization and coordination module includes a memory aging unit; The memory aging unit is used to calculate a comprehensive importance score based on the access frequency and timeliness attributes of memory entries, and to store the memory data in the distributed memory storage cluster in different storage layers based on the comprehensive importance score.

2. The AI ​​intelligent agent memory hierarchical management system based on distributed cloud storage according to claim 1, characterized in that, The cloud-based memory storage module also includes a memory indexing engine; The memory index engine includes an inverted index sub-engine and a vector index sub-engine; The inverted index sub-engine is used to segment the text content of each memory entry into independent terms, and to establish a mapping relationship for each term to a list of memory entry identifiers containing that term, so as to support precise keyword-based retrieval. The vector indexing sub-engine is used to convert the semantic content of memory entries into high-dimensional vectors using a pre-trained language model, and to construct a multi-layer graph index structure for the semantic vectors of all memory entries using a hierarchical navigable small-world graph algorithm, so as to support approximate retrieval based on semantic similarity. The memory indexing engine also supports combined search modes; The combined retrieval mode first uses the inverted index sub-engine to filter out a set of candidate memory entries through keyword matching, and then uses the vector index sub-engine to sort the semantic vectors in the set of candidate memory entries by similarity and return the retrieval results.

3. The AI ​​intelligent agent memory hierarchical management system based on distributed cloud storage according to claim 1, characterized in that, The cloud-based memory storage module also includes a privacy protection unit; The privacy protection unit is used to encrypt sensitive memory data using a ciphertext policy attribute-based encryption algorithm before storing it in the distributed memory storage cluster. The workflow of the privacy protection unit performing encryption processing includes: determining an access strategy represented by an access tree structure based on the sensitivity level and access control requirements of the sensitive memory data; The privacy protection unit uses the system master public key and the access policy to encrypt the sensitive memory data to generate ciphertext, and stores the ciphertext in association with the access policy; When an access request carrying a set of requester attributes and a corresponding private key is received, the privacy protection unit determines whether the set of requester attributes satisfies the access policy embedded in the ciphertext. If the conditions are met, the privacy protection unit decrypts and returns the original memory data; If the conditions are not met, the privacy protection unit rejects the access request.

4. The AI ​​intelligent agent memory hierarchical management system based on distributed cloud storage according to claim 1, characterized in that, Each storage node in the distributed memory storage cluster is configured with a local data management subsystem and a replica synchronization subsystem; The local data management subsystem is used to manage the data shards stored on this node and maintain the data version information and access logs; The replica synchronization subsystem is used to synchronize data with other storage nodes; The default number of data replicas is 3, stored on one master node and two slave nodes; The master node is responsible for handling write operations, and the slave nodes synchronize the data changes of the master node through an asynchronous replication mechanism. The default time interval for asynchronous replication is 1 second. The distributed memory storage cluster uses the heartbeat mechanism in the Raft protocol for master node election; When the master node fails, the distributed memory storage cluster automatically elects a slave node as the new master node to continue providing services.

5. The AI ​​intelligent agent memory hierarchical management system based on distributed cloud storage according to claim 1, characterized in that, The intelligent synchronization and coordination module also includes a conflict arbitration unit; The conflict arbitration unit is used to determine that a conflict has occurred when multiple update requests with the same version number for the same data entry are received, and to select the winning version based on a preset conflict resolution strategy. The conflict resolution strategies include a timestamp priority strategy and an authority node priority strategy; The timestamp priority strategy is used to compare the timestamps of each concurrent update operation and select the data version with the latest timestamp as the final result; The authoritative node priority strategy is used to prioritize the data version of the higher-authority node as the conflict arbitration result based on the preset authority level of each storage node. After completing the arbitration, the conflict arbitration unit generates an arbitration result notification. The arbitration result notification includes a unique identifier for the conflicting data, the data content of the winning version, and a timestamp of the winning version; The conflict arbitration unit updates the data version number to the current maximum value plus 1.

6. The AI ​​intelligent agent memory hierarchical management system based on distributed cloud storage according to claim 1, characterized in that, The intelligent synchronization and coordination module also includes a change detection unit and a synchronization execution unit; The change detection unit is used to monitor data changes in the local cache unit of the local running module through a Hook mechanism or a polling mechanism. The change detection unit generates a change record when it detects data addition, data modification, or data deletion operations. The change record includes a unique data identifier, change type, data snapshot before the change, data snapshot after the change, and change timestamp; The synchronization execution unit is used to maintain the synchronization task queue and supports two types of data synchronization operations: uplink synchronization and downlink synchronization. The uplink synchronization is used to synchronize new or updated data in the local cache unit to the cloud memory storage module. The downlink synchronization is used to download the retrieval results or the winning version of the conflict arbitration results from the cloud memory storage module to the local cache unit; When the synchronization execution unit detects that the target node is unreachable while executing a synchronization task, it marks the synchronization task as waiting for retry and retryes it using an exponential backoff method. The synchronous execution unit verifies data integrity using an MD5 checksum after data transmission is completed.

7. The AI ​​intelligent agent memory hierarchical management system based on distributed cloud storage according to claim 1, characterized in that, The memory aging unit in the intelligent synchronization and coordination module periodically scans the memory entries in the distributed memory storage cluster. The formula for calculating the overall importance score of the memory aging unit is as follows: Overall importance score = 0.4 × Normalized access frequency + 0.3 × Normalized reciprocal recent access time + 0.2 × Normalized semantic importance + 0.1 × Normalized creation time freshness; The normalized semantic importance is determined by the weighted sum of the number of in-degree edges and the number of out-degree edges of a node in the knowledge association network; The rules for hierarchical storage management of memory data by the memory aging unit based on the comprehensive importance score are as follows: Memory entries with an overall importance score of 0.7 or higher are stored in a high-performance storage layer composed of solid-state drives; Memory entries with an overall importance score between 0.3 and 0.7 are stored in a low-cost storage layer consisting of mechanical hard drives; Memory entries with an overall importance score between 0.1 and 0.3 and a last access time of more than 90 days are compressed and migrated to the archive layer composed of object storage services; Memory entries with an overall importance score below 0.1 and a last access time exceeding 365 days are migrated to a cold storage layer consisting of a distributed file system or tape storage.

8. The AI ​​intelligent agent memory hierarchical management system based on distributed cloud storage according to claim 1, characterized in that, The local operation module includes an environment perception unit, a reasoning and decision-making unit, and a local cache unit; The environmental sensing unit is used to continuously capture external environmental information and internal state data, and to perform standardized preprocessing on the captured data; The standardized preprocessing includes data format unification, timestamp marking, priority marking, and noise reduction filtering; The reasoning decision unit is used to perform semantic understanding and knowledge matching based on the short-term memory data stored in the local cache unit, and to select the optimal action strategy based on the reasoning results. The semantic understanding uses a pre-trained language model to convert the input text into semantic vectors; The knowledge matching is achieved by calculating the cosine similarity between the semantic vector and the semantic vector of the memory entry in the local cache unit. Memory entries with a similarity exceeding a preset threshold of 0.7 are identified as matching items. The local cache unit is used to temporarily store the interaction data and state variables generated by the agent in the current running cycle, and maintain an elimination queue based on elimination priority scores. When the cache space is insufficient, the local cache unit prioritizes clearing the data entry with the lowest eviction priority score.

9. The AI ​​intelligent agent memory hierarchical management system based on distributed cloud storage according to claim 1, characterized in that, The system also includes a multi-agent collaboration module; The multi-agent collaboration module is used to coordinate memory sharing and collaborative reasoning among multiple locally running instances; The multi-agent collaboration module maintains a collaborative task scheduler; The collaborative task scheduler breaks down the overall task into multiple sub-tasks based on the task dependency graph; The collaborative task scheduler evaluates the capability score of each instance based on the number of CPU cores, available memory, CPU utilization, and network bandwidth reported by each locally running instance. The collaborative task scheduler prioritizes instances with higher capability scores and shorter current task queues to assign the sub-tasks. The multi-agent collaboration module also supports collaboration session management, which is used to establish persistent collaboration sessions between multiple locally running instances. Shared memory data generated during the session is automatically synchronized to the cloud memory storage module for access by all session participants.

10. The AI ​​intelligent agent memory hierarchical management system based on distributed cloud storage according to claim 1, characterized in that, In the distributed memory storage cluster, the specific implementation method of using the consistent hashing algorithm for data distribution management is as follows: The entire storage space is abstracted as a hash ring with a numerical range of 0 to 2^32-1; The hash value of the storage node identifier is calculated using the secure hash algorithm 256, and the first 32 bits are taken as the position of the node on the hash ring. When new memory data is written, the hash value of the unique identifier of the data is calculated and the first 32 bits are taken. The first storage node in the clockwise direction on the hash ring is then used as the storage node to which the data belongs. The consistent hashing algorithm also supports virtual node technology; Each physical storage node corresponds to multiple virtual nodes, and the number of virtual nodes is dynamically adjusted according to the hardware performance of the physical node and the business load. The adjustment rules are as follows: collect the CPU utilization and network bandwidth utilization of each physical node every 5 minutes; when the load score exceeds 60%, add 10 virtual nodes to that physical node; when the load score is below 30%, reduce 10 virtual nodes; and each physical node shall retain at least 20 virtual nodes. The load score is calculated as follows: Load score = 0.6 × CPU utilization + 0.4 × Network bandwidth utilization.

Images