Multi-mode space-time memory retrieval system for AI intelligent agent

Abstract

The invention discloses a multi-mode space-time memory retrieval system for an AI agent, and the system comprises a memory storage module which employs a ConceptNode structure to uniformly store heterogeneous memory content; the multi-modal retrieval module is used for realizing memory retrieval sorting through a comprehensive scoring function; the three retrieval modes specifically comprise a keyword matching degree retrieval mode, a semantic matching degree retrieval mode and a time-space matching degree retrieval mode; the timeliness calculation module adopts an exponential decay function; the importance evaluation module is used for automatically generating importance scores by combining a large language model with domain knowledge; and the forgetting calculation management module is used for dynamically distributing memory duration based on a nonlinear life cycle function, configuring differentiated parameters for different memory types and supporting life cycle extension of access triggering. According to the method, the adaptive learning ability and decision authenticity of the AI intelligent agent in a dynamic environment are remarkably improved, and the method is suitable for intelligent decision scenes such as urban traffic simulation and robot navigation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and machine learning, and particularly relates to a multi-mode spatio-temporal memory retrieval system for AI agents. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, AI agents play an increasingly important role in various complex application scenarios. In urban traffic simulation, robot navigation, intelligent assistants and other applications, agents need to have the ability to learn from historical experience and adapt to dynamic environmental changes. This requires agents to have an efficient memory system to store, retrieve and manage massive amounts of historical information.

[0003] The existing AI agent memory system mainly has the following technical problems:

[0004] Low efficiency in processing spatio-temporal correlation information: Traditional memory systems often store and process spatial information and temporal information separately, which cannot effectively capture spatio-temporal correlation, resulting in low retrieval efficiency in scenarios requiring spatio-temporal reasoning. For example, in traffic simulation, agents need to make decisions based on historical traffic conditions at a specific time and location, but existing systems cannot quickly retrieve relevant spatio-temporal correlation memories.

[0005] Insufficient precision in multi-mode information retrieval: Existing methods usually only support a single retrieval mode, such as keyword matching or semantic similarity, and cannot comprehensively consider multiple dimensions such as keyword exact match, semantic concept association, and spatio-temporal correlation, resulting in limited relevance and accuracy of retrieval results.

[0006] For example, in existing traffic simulation systems, agents can record historical travel experiences, but when faced with new traffic conditions, they often cannot quickly retrieve relevant historical experiences for reference, resulting in decreased decision-making quality. At the same time, the system accumulates a large amount of redundant and low-value memories, occupying valuable storage resources.

[0007] Therefore, there is an urgent need for an AI agent memory system that can efficiently process spatio-temporal correlation information, has an intelligent forgetting mechanism, and supports multi-mode retrieval. SUMMARY

[0008] In view of the deficiencies in the prior art, the present application aims to provide a multi-mode spatio-temporal memory retrieval system for AI agents, which solves the technical problems of low efficiency in processing spatio-temporal correlation information, lack of forgetting mechanism consistent with human cognitive rules, and insufficient precision in multi-mode information retrieval in the prior art, and realizes efficient storage, accurate retrieval and natural forgetting of agent memory. In order to achieve the above-mentioned purposes and other advantages according to the present application, a multi-mode spatio-temporal memory retrieval system for AI agents is provided, comprising:

[0009] a memory storage module, which uniformly stores heterogeneous memory content in a ConceptNode structure;

[0010] a multi-modal retrieval module, which realizes memory retrieval sorting through a comprehensive scoring function; specifically including three retrieval modes of keyword matching degree, semantic matching degree and space-time matching degree;

[0011] a timeliness calculation module, which adopts an exponential decay function;

[0012] an importance evaluation module, which automatically generates importance scores through a large language model combined with domain knowledge;

[0013] a forgetting calculation management module, which dynamically allocates memory survival time based on a nonlinear life cycle function, configures differentiated parameters for different memory types, and supports life cycle extension triggered by access;

[0014] The memory retrieval process of the system is as follows: for memories within the life cycle, different retrieval modes are selected, and the comprehensive score is calculated; then the comprehensive score is sorted; and the top-k memories with the highest comprehensive score are returned as the retrieval result.

[0015] Preferably, the three retrieval modes of keyword matching degree, semantic matching degree and space-time matching degree are specifically:

[0016] The keyword matching degree is calculated by Jaccard similarity coefficient;

[0017] The semantic matching degree is calculated by normalized vector distance based on pre-trained language model embedding;

[0018] The space-time matching degree is calculated by Overlap coefficient to obtain the overlap degree of space and time dimensions.

[0019] Preferably, the forgetting calculation management module calculates the memory survival time by using the following nonlinear life cycle function:

[0020] lifespan=lifespan min +(lifespan max -lifespan min )*importance(m) power

[0021] wherein,

[0022] lifespan min is the minimum life cycle (unit: hour);

[0023] lifespan max is the maximum life cycle (unit: hour);

[0024] importance(m) is the importance score of memory m;

[0025] Power is the forgetting index; a value of 1.5 to 3.0 is recommended, and different lifespans can be set according to the type of memory. min lifespan max Power parameters.

[0026] Preferably, the forgetting management module supports a lifecycle extension mechanism triggered by memory access, whereby the expiration time of a memory is extended to the initial lifecycle length when the memory is accessed.

[0027] Preferably, the forgetting management module configures differentiated Lifespan settings for different memory types, such as event memory, conversational memory, and reflective insight. min lifespan max The power parameter simulates the selective retention of human memory.

[0028] Preferably, the multi-mode retrieval module supports a weighted fusion strategy to obtain the top-k memories from each retrieval mode, and then re-scores them using mode-specific weights w_r and makes a final selection.

[0029] A memory retrieval method based on a multimodal spatiotemporal memory retrieval system for AI agents, characterized by the following steps:

[0030] (1) Receive query requests and extract the set of keywords, semantic embedding vectors and spatiotemporal information of the query;

[0031] (2) Traverse the memories in the memory bank that are within their life cycle and calculate the keyword matching degree, semantic matching degree and spatiotemporal matching degree respectively;

[0032] (3) Calculate the timeliness score and importance score of each memory;

[0033] (4) Calculate the final score for each memory under the specified retrieval mode according to the comprehensive scoring formula;

[0034] (5) Sort by comprehensive score in descending order and return the top-k relevant memories;

[0035] (6) Periodically perform memory cleanup tasks, determine expired memories based on the memory lifecycle function and perform deletion operations to free up storage space.

[0036] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the memory retrieval method of claim 7.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] (1) Significantly improve the efficiency of spatiotemporal information processing: Through the unified structure of ConceptNode and spatiotemporal matching algorithm, the integrated storage and retrieval of spatial and temporal information is realized, and the retrieval efficiency is improved.

[0039] (2) Achieve intelligent memory management: Based on the psychologically inspired differential forgetting mechanism, it can intelligently distinguish the importance of memories, automatically clear low-value memories, improve the efficiency of storage space utilization, and at the same time maintain the long-term availability of high-value memories.

[0040] (3) Significantly improves retrieval accuracy: The multi-mode fusion retrieval algorithm comprehensively considers three dimensions: keywords, semantics, and spatiotemporal, and the retrieval relevance accuracy is improved compared with the single-mode method.

[0041] (4) Enhance system adaptability: The system can automatically assess the importance of memory through a large language model, and can adapt to different fields and application scenarios without manual parameter tuning.

[0042] (5) Simulates human cognitive characteristics: The forgetting mechanism conforms to the laws of human memory, supports the modeling of the interval effect, and makes the behavior of AI agents more natural and credible. Attached Figure Description

[0043] Figure 1 This is a diagram illustrating the overall architecture of a multimodal spatiotemporal memory retrieval system for AI agents according to the present invention.

[0044] Figure 2 This is a schematic diagram of the ConceptNode memory storage structure for a multi-modal spatiotemporal memory retrieval system for AI agents according to the present invention;

[0045] Figure 3 The multi-mode retrieval flowchart of the multi-mode spatiotemporal memory retrieval system for AI intelligent agents according to the present invention is shown below.

[0046] Figure 4 A schematic diagram illustrating the calculation of three matching modes in the multi-modal spatiotemporal memory retrieval system for AI agents according to the present invention;

[0047] Figure 5 The forgetting mechanism lifecycle function curve of the multimodal spatiotemporal memory retrieval system for AI agents according to the present invention;

[0048] Figure 6 This is a comparison diagram of different memory type parameter configurations for the multi-modal spatiotemporal memory retrieval system for AI agents according to the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Reference Figure 1 A multimodal spatiotemporal memory retrieval system for AI agents, including:

[0051] The memory storage module adopts a ConceptNode structure to uniformly store heterogeneous memory content. Each ConceptNode node includes a text description, semantic vector embedding, keyword set, geospatial coverage area, effective time interval, importance quantification score, creation timestamp, lifecycle, and access history, demonstrating the unified data structure design of memory storage.

[0052] Furthermore, such as Figure 3 As shown, the complete algorithm process is illustrated in detail, starting from receiving a query request, parsing the query content to extract keywords (K), embeddings (E), spatial range (S), and time interval (T), then performing parallel processing of keyword matching calculation, semantic matching calculation, and spatiotemporal matching calculation through pattern selection, while simultaneously generating importance and calculating timeliness, and finally returning the search results through comprehensive score calculation and sorting.

[0053] The multi-mode retrieval module supports a weighted fusion strategy to obtain the top-k memories from each retrieval mode, then re-scores them using mode-specific weights w_r and makes the final selection.

[0054] A memory retrieval method includes the following steps:

[0055] (1) Receive query requests and extract the set of keywords, semantic embedding vectors and spatiotemporal information of the query;

[0056] (2) Traverse the memories in the memory bank that are within their life cycle and calculate the keyword matching degree, semantic matching degree and spatiotemporal matching degree respectively;

[0057] (3) Calculate the timeliness score and importance score of each memory;

[0058] (4) Calculate the final score for each memory under the specified retrieval mode according to the comprehensive scoring formula;

[0059] (5) Sort by comprehensive score in descending order and return the top-k relevant memories;

[0060] (6) Periodically perform memory cleanup tasks, determine expired memories based on the memory lifecycle function and perform deletion operations to free up storage space.

[0061] The multimodal retrieval module uses a comprehensive scoring function to achieve memory-based retrieval and ranking; specifically, it includes three retrieval modes: keyword matching degree, semantic matching degree, and spatiotemporal matching degree.

[0062] Specifically, a comprehensive scoring mechanism will be adopted.

[0063] score r (q,m)=w r *matching r (q,m)+δ*recency(m)+γ*importance(m)

[0064] in,

[0065] q (short for query) is for querying (the ConceptNode object);

[0066] m (short for memory) is a memory record (ConceptNode object) in the memory bank;

[0067] matching r (m) represents the matching degree; the matching mode (r) supports three retrieval modes: keyword matching, semantic matching, and spatiotemporal matching.

[0068] recency(m) represents timeliness;

[0069] importance(m) represents importance.

[0070] score r The overall score of memory m obtained under mode r.

[0071] matching r Regarding (q,m), the keyword matching degree is calculated using the Jaccard similarity coefficient:

[0072]

[0073] in,

[0074] K q and K m These are the sets of keywords for query (q) and memory (m), respectively.

[0075] ∩ and ∪ are the intersection and union operations of sets;

[0076] |·| is the cardinality of a set.

[0077] matching r Regarding (q,m), the semantic matching degree is based on the pre-trained language model embedding, and the matching degree is calculated using normalized vector distance:

[0078]

[0079] in,

[0080] E q and E m These are the semantic embedding vectors for the query and the memory content, respectively;

[0081] d(E q E m () is the vector distance; L2 Euclidean distance is recommended.

[0082] d max It is the maximum vector distance; it is recommended to take...

[0083] matching r In terms of (q,m), the spatiotemporal matching degree is calculated using the Overlap coefficient (Szymkiewicz–Simpson coefficient) to obtain the overlap between the spatial and temporal dimensions:

[0084] matching spatialtemporal (q,m)=overlap(S q ,S m )*overlap(T q ,T m )

[0085]

[0086] in,

[0087] S q S m The spatial coverage of query (q) and memory (m);

[0088] T q T m The time period coverage for query (q) and memory (m);

[0089] overlap(A,B) is the overlap coefficient of two sets A and B;

[0090] ∩ and ∪ are the intersection and union operations of sets;

[0091] |·| is the cardinality of a set.

[0092] Furthermore, such asFigure 4 As shown, the specific algorithm implementations and numerical examples of keyword matching (based on the Jaccard similarity coefficient, with an example showing the calculation process of the ratio between the intersection {Zhongshan Road, congestion} and the union {morning rush hour, Zhongshan Road, Chang'an Road, congestion, traffic accident}), semantic matching (based on the distance calculation between vectors, showing the normalization process of the distance between the query vector and the memory vector), and spatiotemporal matching (based on the overlap coefficient to calculate the overlap of spatial and temporal dimensions, with an example showing the calculation results of the spatial overlap of Zhongshan Road being 1.0 and the partial overlap of the time interval) are presented.

[0093] The timeliness calculation module uses an exponential decay function:

[0094]

[0095] in,

[0096] λ is the decay rate parameter; a value of 0.90 to 0.95 is recommended.

[0097] t m Duration of memory retention (unit: days).

[0098] The importance assessment module automatically generates importance scores by combining a large language model with domain knowledge.

[0099] Furthermore, such as Figure 5 As shown, with importance score (0-1) on the horizontal axis and lifespan days on the vertical axis, the nonlinear lifespan function curves of three memory types are displayed: event memory (blue solid line, parameters min=2h, max=4d, power=2.4), chat memory (red dashed line, parameters min=4h, max=2d, power=3.2), and thinking memory (green dotted line, parameters min=8h, max=8d, power=1.6). Specific numerical examples under different importance levels (trivial, minor, moderate, important, severe) are also marked. The forgetting calculation and management module dynamically allocates memory duration based on the nonlinear lifespan function, configures differentiated parameters for different memory types, and supports access-triggered lifespan extension. Each memory has a lifespan attribute; the calculation method uses the following nonlinear lifespan function to dynamically allocate memory duration:

[0100] lifespan = lifespan min +(lifespan max -lifespan min *importance(m) power

[0101] in,

[0102] lifespan min It is the minimum lifespan (unit: hours);

[0103] lifespan max This is the maximum lifespan (unit: hours);

[0104] importance(m) is the importance score of memory m;

[0105] Power is the forgetting index; a value of 1.5 to 3.0 is recommended.

[0106] like Figure 6 As shown, a table illustrates the differentiated configurations of three memory types—event memory, conversational memory, and reflective insight—in terms of minimum lifespan (2h, 4h, 8h), maximum lifespan (96h, 48h, 192h), and Power parameter (2.4, 3.2, 1.6). A bar chart visually compares the relative magnitudes of each parameter. Different lifespans can be set according to the memory type. min lifespan max The forgetting management module configures differentiated Lifespan parameters for different memory types, such as event memory, conversational memory, and reflective insight. min lifespan max The `power` parameter simulates the selective retention characteristics of human memory. The forgetting management module supports a lifespan extension mechanism triggered by memory access; when a memory is accessed, its expiration time is extended beyond its initial lifespan.

[0107] The memory retrieval process of this system is as follows: for memories within their life cycle, different retrieval modes are selected and a comprehensive score is calculated; then, the memories are sorted according to the comprehensive score; and the top-k memories with the highest comprehensive scores are returned as retrieval results.

[0108] This invention can be implemented in a standard computer hardware environment, and recommended configurations include:

[0109] CPU: Intel i7 or equivalent processor

[0110] Memory: 16GB or more RAM

[0111] Storage: 100GB or larger SSD hard drive

[0112] Software Architecture

[0113] The system adopts a modular design and is implemented based on a Python 3.8+ environment. Its main dependent libraries include:

[0114] NumPy and SciPy: Numerical computation

[0115] faiss: Efficient Vector Search

[0116] sentence-transformers: semantic embedding generation

[0117] Large language model API interfaces (such as OpenAI GPT, Qwen, etc.).

[0118] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0119] Example 1: ConceptNode Unified Memory Structure

[0120] 1.1 ConceptNode Class Definition

[0121] like Figure 2 As shown, this invention uses the ConceptNode class as a unified memory storage structure, and the specific implementation is as follows:

[0122]

[0123]

[0124] 1.2 The realization of a psychologically inspired forgetting mechanism

[0125] like Figure 5 and Figure 6 As shown, this invention implements a significance-based differential forgetting mechanism:

[0126]

[0127]

[0128] Example 2: Long-Term Memory Management System

[0129] 2.1 LongTermMemory Class Architecture

[0130] like Figure 1 As shown, the long-term memory system employs multiple index structures to support efficient retrieval:

[0131]

[0132]

[0133] 2.2 Memory Addition Mechanism

[0134]

[0135]

[0136] Example 3: Implementation of a Multi-Mode Retrieval Algorithm

[0137] 3.1 Keyword matching retrieval

[0138] like Figure 4 As shown, keyword matching is performed using the Jaccard similarity coefficient:

[0139]

[0140]

[0141] 3.2 Semantic Similarity Retrieval Based on FAISS Vector Index for Efficient Semantic Retrieval:

[0142]

[0143]

[0144] 3.3 Spatiotemporal Matching Retrieval

[0145] Spatiotemporal matching degree is calculated using the overlap coefficient (Szymkiewicz–Simpson coefficient):

[0146]

[0147]

[0148]

[0149] Example 4: Intelligent Forgetting Management Mechanism 4.1 Expired Memory Clearing

[0150]

[0151]

[0152] 4.2 Extended lifecycle triggered by access

[0153]

[0154] Example 5: Comprehensive Search Strategy

[0155] 5.1 Multi-mode fusion retrieval

[0156] like Figure 3 As shown, a weighted fusion of multiple retrieval modes is implemented:

[0157]

[0158]

[0159]

[0160] The number of devices and processing scale described herein are for simplification of the invention. Applications, modifications, and variations of this invention will be readily apparent to those skilled in the art. Although embodiments of the invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. It can be applied to various fields suitable for this invention, and further modifications can be readily implemented by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, this invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A multi-modal spatiotemporal memory retrieval system for AI intelligent agents, characterized in that, include: The memory storage module adopts a ConceptNode structure to uniformly store heterogeneous memory content; The multimodal retrieval module uses a comprehensive scoring function to achieve memory-based retrieval and ranking; specifically, it includes three retrieval modes: keyword matching degree, semantic matching degree, and spatiotemporal matching degree. The timeliness calculation module uses an exponential decay function; The importance assessment module automatically generates importance scores by combining a large language model with domain knowledge. The forgetting calculation and management module dynamically allocates memory duration based on a non-linear lifecycle function, configures differentiated parameters for different memory types, and supports access-triggered lifecycle extension. The memory retrieval process of this system is as follows: for memories within their life cycle, different retrieval modes are selected and a comprehensive score is calculated; then, the memories are sorted according to the comprehensive score; and the top-k memories with the highest comprehensive scores are returned as retrieval results.

2. The multi-modal spatiotemporal memory retrieval system for AI agents as described in claim 1, characterized in that, The three retrieval modes—keyword matching, semantic matching, and spatiotemporal matching—are as follows: Keyword matching accuracy was calculated using the Jaccard similarity coefficient. Semantic matching score is based on pre-trained language model embeddings and is calculated using normalized vector distance. The spatiotemporal matching degree is calculated using the overlap coefficient to obtain the degree of overlap in the spatial and temporal dimensions.

3. The multi-modal spatiotemporal memory retrieval system for AI agents as described in claim 1, characterized in that, The forgetting calculation management module uses the following non-linear lifecycle function to dynamically allocate memory duration: lifespan=lifespan min +(lifespan max -lifespan min )*importance(m) power in, lifespan min It is the minimum lifespan (unit: hours); lifespan max This is the maximum lifespan (unit: hours); importance(m) is the importance score of memory m; Power is the forgetting index; a value of 1.5 to 3.0 is recommended, and different lifespans can be set according to the type of memory. min lifespan max Power parameters.

4. The multi-modal spatiotemporal memory retrieval system for AI agents as described in claim 3, characterized in that, The forgetting management module supports a lifecycle extension mechanism triggered by memory access, which extends the expiration time of a memory by the initial lifecycle length when the memory is accessed.

5. The multi-modal spatiotemporal memory retrieval system for AI agents as described in claim 4, characterized in that, The forgetting management module configures differentiated Lifespan settings for different memory types, such as event memory, conversation memory, and reflective insight. min lifespan max The power parameter simulates the selective retention of human memory.

6. The multi-modal spatiotemporal memory retrieval system for AI agents as described in claim 1, characterized in that, The multi-mode retrieval module supports a weighted fusion strategy to obtain the top-k memories from each retrieval mode, then re-scores them using mode-specific weights w_r and makes the final selection.

7. A memory retrieval method based on the system according to any one of claims 1-6, characterized in that, Includes the following steps: (1) Receive query requests and extract the set of keywords, semantic embedding vectors and spatiotemporal information of the query; (2) Traverse the memories in the memory bank that are within their life cycle and calculate the keyword matching degree, semantic matching degree and spatiotemporal matching degree respectively; (3) Calculate the timeliness score and importance score of each memory; (4) Calculate the final score for each memory under the specified retrieval mode according to the comprehensive scoring formula; (5) Sort by comprehensive score in descending order and return the top-k relevant memories; (6) Periodically perform memory cleanup tasks, determine expired memories based on the memory lifecycle function and perform deletion operations to free up storage space.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the memory retrieval method as described in claim 7.

Images