Intelligent response method and device based on memory system drive, equipment and medium

Abstract

The invention discloses an intelligent response method and device based on memory system driving, equipment and a medium. The method comprises the following steps: in response to an input query statement, coding the query statement into a situation vector in combination with current situation information; based on the context vector, retrieving a group of memory units related to the context vector in a dynamic memory knowledge graph in multiple dimensions; based on a preset memory association analysis rule, constructing an association relationship between a group of memory units to obtain a memory sub-graph; and based on the memory sub-graph, performing multi-modal fusion on the memory fragments corresponding to the group of memory units to generate a natural language response. Through the mode, the memory can be represented as the memory unit and stored in the dynamic memory knowledge graph, so that the memory ability of the intelligent agent is greatly enhanced; according to the method, multi-dimensional retrieval is carried out in the dynamic knowledge graph, the most relevant memory unit is quickly and accurately retrieved, and accurate and coherent natural language response is generated through multi-modal fusion, so that the response efficiency and the response quality are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an intelligent response method, apparatus, device and medium based on a memory system. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have achieved remarkable results in fields such as natural language processing and question answering. However, existing AI agents generally lack persistent and structured memory capabilities. In application scenarios such as intelligent assistants, personalized services, education and tutoring, and medical diagnosis, AI agents need to be able to remember historical interaction content, learn user preferences, and accumulate domain knowledge, and provide more personalized and coherent services based on these memories.

[0003] Existing AI memory-related technologies mainly include: Context-window-based memory techniques rely on the input length limitations of the model to maintain short-term memory within a single conversation, but cannot persist across conversations.

[0004] External knowledge base enhancement technology: Historical information is stored through technologies such as vector databases, but it lacks a structured organization and dynamic update mechanism for the stored content.

[0005] The prompting engineering approach compresses historical information into prompt words and injects them into the model, but it is limited by the length of the model input and cannot realize the dynamic evolution of memory. Summary of the Invention

[0006] This application provides a memory system-driven intelligent response method, device, equipment, and medium to address the problems of insufficient memory capacity, low memory retrieval efficiency and accuracy, and difficulty in generating accurate responses in existing AI memory technologies.

[0007] To address the aforementioned technical problems, this application provides a technical solution: a memory system-driven intelligent response method. This method includes: In response to the input query statement, the query statement is encoded into a context vector in combination with the current context information; Based on the context vector, a set of memory units related to the context vector are retrieved in a multi-dimensional manner in the dynamic memory knowledge graph; Based on preset memory association analysis rules, a set of association relationships between the memory units is constructed to obtain a memory subgraph; Based on the memory subgraph, multimodal fusion is performed on the memory segments corresponding to a set of memory units to generate natural language responses.

[0008] In one optional embodiment of this application, the step of retrieving a set of memory units related to the context vector in a dynamic memory knowledge graph from multiple dimensions based on the context vector includes: For the memory unit in the dynamic memory knowledge graph, a comprehensive relevance score is calculated based on one or more preset relevance evaluation factors; Based on the comprehensive relevance score, and in combination with at least one retrieval dimension independent of the comprehensive relevance score, a set of memory units related to the context vector are retrieved from the dynamic memory knowledge graph.

[0009] In one optional embodiment of this application, the construction process of the memory unit includes the following: The system receives raw input information from multiple modalities and extracts features from the raw input information based on the encoders corresponding to each modality. The features from each modality are then fused and atomized into memory units that contain at least semantic labels and timestamps.

[0010] In an optional embodiment of this application, after constructing the memory unit, the method further includes: The memory units are managed based on a hierarchical structure containing multiple memory layers; the importance of the memory units is calculated based on an importance evaluation algorithm, and the memory units are dynamically migrated between the multiple memory layers based on the importance; based on an adaptive forgetting mechanism and the importance, memory units that have not reached the preset level standard corresponding to the memory layer are forgotten.

[0011] In an optional embodiment of this application, after constructing the memory unit, the method further includes: Based on the semantic tag and the timestamp, calculate the first association strength between the current memory unit and each memory unit in the dynamic memory knowledge graph; When the first association strength satisfies the first preset association condition, the association relationship between the current memory unit and the corresponding memory unit is established in the dynamic memory knowledge graph.

[0012] In one optional embodiment of this application, the step of constructing a set of association relationships between the memory units based on preset memory association analysis rules to obtain a memory subgraph includes: Based on the semantic tag, the timestamp, and the first association strength, calculate the second association strength between a group of memory units; When the second association strength satisfies the second preset association condition, the association relationship between the corresponding memory units in the dynamic memory knowledge graph is constructed to obtain the memory subgraph.

[0013] In one optional embodiment of this application, the step of performing multimodal fusion on memory segments corresponding to a group of memory units based on the memory subgraph to generate a natural language response includes: Based on a preset timeline reconstruction method and a preset semantic coherence processing method, the memory segments corresponding to each memory unit in the memory subgraph are sorted to obtain coherent memory segments. The coherent memory fragments are fused using multimodal methods, and the natural language response is generated based on a pre-trained language model.

[0014] In one optional embodiment of this application, the method further includes: Obtain the retrieval frequency and activation time corresponding to each memory unit, and perform memory conflict detection on the new original input information; In response to the memory cell's retrieval frequency being higher than a preset retrieval frequency, the importance of the memory cell is updated; In response to the fact that the activation times of the plurality of memory cells are the same, the first association strength between the plurality of memory cells is updated; In response to the detection of the memory conflict, the dynamic memory knowledge graph is updated based on a preset memory correction mechanism.

[0015] To solve the aforementioned technical problems, another technical solution adopted in this application is: to provide an intelligent response device driven by a memory system. The device includes: The encoding module is used to encode the input query statement into a context vector in response to the input query statement, combined with the current context information. The retrieval module is used to retrieve a set of memory units related to the context vector in a dynamic memory knowledge graph from multiple dimensions based on the context vector. The association module is used to construct a set of association relationships between the memory units based on preset memory association analysis rules, thereby obtaining a memory subgraph; The generation module is used to perform multimodal fusion on memory fragments corresponding to a group of memory units based on the memory subgraph to generate a natural language response.

[0016] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a computer device, including a memory, a processor and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the above-mentioned intelligent response method based on memory system drive.

[0017] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a storage medium on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the steps of the above-mentioned intelligent response method based on a memory system.

[0018] The beneficial effects of this application are as follows: Unlike existing technologies, this application discloses an intelligent response method, apparatus, device, and medium driven by a memory system. This method significantly enhances the memory capacity of the AI ​​agent by storing memories as memory units in a dynamic memory knowledge graph. Based on the context vector obtained from the encoded input query statement, multi-dimensional retrieval is performed in this dynamic knowledge graph, thereby quickly and accurately retrieving the most relevant memory units. Furthermore, accurate and coherent natural language responses are generated through multimodal fusion, simultaneously improving response efficiency and quality. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a flowchart illustrating an embodiment of the memory system-driven intelligent response method provided in this application; Figure 2 This is a schematic diagram of the structure of an embodiment of the memory system-driven intelligent response device provided in this application; Figure 3 This is a schematic diagram of the structure of an embodiment of the storage medium provided in this application; Figure 4 This is a schematic diagram of the structure of an embodiment of the computer device provided in this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] The terms "first," "second," and "third" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] This application provides an intelligent response method based on a memory system, see reference. Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the memory system-driven intelligent response method provided in this application. The memory system-driven intelligent response method includes: S10: In response to the input query statement, encode the query statement into a context vector by combining it with the current context information.

[0024] In this application, a large language model is encapsulated into an intelligent agent using preset prompts. This intelligent agent possesses at least strong language understanding and generation capabilities. The preset prompts can be flexibly adjusted in different application scenarios to guide the intelligent agent to generate responses that meet the needs of the current scenario. No limitation is placed on the type of intelligent agent here.

[0025] Upon receiving a user's query input on the interactive interface, the query language, combined with current contextual information, is input into a vector model for encoding, thereby converting multi-source heterogeneous information into a unified vector representation. The query can be natural language text or speech; when the device supports a camera, it can also be a video conversation or image / text input format, etc., without limitation here. Current contextual information includes at least the current dialogue context, user state, and environmental information.

[0026] The dialogue context is the historical record of the interaction between the user and the agent. Through the dialogue context, the agent can understand the coherence and continuity of the user's query. The user's state includes at least the user's emotions, preferences, and historical behavior patterns. This information helps the agent to more accurately grasp the user's needs and provide personalized responses. The environmental information includes at least external factors such as time, space, and device information. These external factors may affect the user's query intent. By combining environmental information, the agent can provide responses that are more in line with the actual situation.

[0027] Dialogue context and environment information can be obtained directly, while user state can be obtained through the following methods: User emotions can be obtained through text sentiment analysis technology, which performs semantic analysis on the natural language queries entered by users on the interactive interface to identify the emotional tendencies contained therein. In addition, if the user's input includes voice or video, voice sentiment analysis technology can be used to analyze the characteristics of the user's voice, such as tone, speed, and volume, to determine the user's current emotional state. Alternatively, facial expression recognition technology can be used to capture changes in the user's facial expressions to infer their emotions.

[0028] User preferences can be obtained through user feedback, such as ratings or comments, allowing the agent to more accurately grasp user needs and preferences and provide responses that meet user expectations. For example, if a user expects the agent to provide a "concise and clear answer," this feature can be stored as a user preference in the memory system.

[0029] Historical behavioral patterns can be obtained by analyzing the historical records of user interactions with the agent. Specifically, the agent can record data such as the types of questions asked by users in past interactions, query frequency, and usage of frequently used functions, and then use data analysis techniques to mine and organize this data. For example, by statistically analyzing the distribution of user query topics over different time periods, trends in user interests and preferences can be understood; by analyzing user satisfaction feedback with the agent's responses, subsequent response strategies can be optimized. This historical behavioral pattern data will be stored in a memory system to provide personalized response support for the agent.

[0030] The query statement, combined with the current context information, is input into the vector model. The vector model then performs feature extraction and semantic analysis on the query statement and the current context information, converting multi-source heterogeneous information into a unified vector representation. This enables a more accurate characterization of the semantic connotation of the query statement and the current context information, providing a solid data foundation for subsequent intelligent responses.

[0031] S20: Based on context vectors, retrieve a set of memory units related to context vectors in a dynamic memory knowledge graph from multiple dimensions.

[0032] In this application, the construction process of the memory unit is as follows, including: It receives raw input information from multiple modalities and extracts features from the raw input information based on the encoders corresponding to each modality; it then fuses the features from each modality and atomizes them into memory units that contain at least semantic labels and timestamps.

[0033] In this application, to transform multimodal heterogeneous memory fragments into a unified memory representation, a memory unit is designed to store important features within the memory fragments. Specifically, the multimodal raw input information input to the agent includes at least one or more of the following: text channel, image channel, audio channel, and video channel. Correspondingly, the agent internally has pre-built encoders corresponding to the raw input information of each modality. Semantic features are extracted from the raw input information of the text channel based on the text encoder (such as a pre-trained large language model); visual features are extracted from the raw input information of the image or video channel based on the image encoder (such as a visual model); and acoustic features are extracted from the raw input information of the audio or video channel based on the audio encoder (such as an audio processing model).

[0034] In some embodiments, the encoders corresponding to the above modalities can be implemented based on a pluggable encoder architecture. The pluggable encoder architecture enables hot-swapping of encoders through standardized interfaces and modular design, allowing access to pre-trained large models of different types or sizes. Before encoding, the most suitable encoder is dynamically selected based on indicators such as content matching degree and performance score by analyzing the characteristics of the original input information. Furthermore, it supports flexible switching between models of different sizes according to scenario requirements, balancing performance and resource consumption.

[0035] Based on the semantic, visual, and acoustic features extracted from the original input information of each modality, feature fusion is performed and atomized into memory units with unified semantic labels. For example, in an interaction where a user seeks technical advice related to product design, the original input information of the user interaction can be represented as a memory unit: {ID: "M_001", Type: "Multimodal Unified", Unified Feature: [0.12, 0.45, ...], Timestamp: "2023-10-15 14:30", Semantic Label: ["User Interaction", "Technical Consultation", "Product Design"], Original Content: {...}}.

[0036] In a memory unit, ID is a unique identifier assigned during its construction; type is the type of the original input information corresponding to the memory unit, including text, image, audio, and video types. When the original input information contains two or more modalities, its type is denoted as a multimodal unified type; unified feature is the unified feature obtained after feature fusion of the original input information corresponding to the memory unit; timestamp is the timestamp of the original input information corresponding to the memory unit being input to the agent; semantic label is a unified semantic label generated based on the semantic understanding of the multimodal original input information; and original content is the multimodal content of the original input information, which can be called a memory fragment.

[0037] Unlike existing memory systems primarily designed for textual information, which struggle to effectively process and associate memory content from multimodal channels such as images, audio, and video, this application presents a memory system-driven intelligent response method. By constructing a unified memory unit that integrates multimodal information, it achieves effective integration and association of various types of raw input information, including images, audio, and video. This multimodal fusion approach comprehensively and accurately captures and stores multimodal information, providing rich and reliable data support for intelligent responses, thereby effectively improving the accuracy and comprehensiveness of intelligent responses.

[0038] In this application, after constructing the memory unit, the following is also included: S21: Memory units are managed based on a hierarchical structure containing multiple memory layers.

[0039] In this application, a hierarchical structure containing multiple memory layers is constructed by simulating the hierarchical structure of human memory to manage memory units and raw input information. This application uses a three-level memory layer structure as an example for illustration. The three-level memory layer structure can be divided into a transient memory layer, a short-term memory layer, and a long-term memory layer according to capacity and memory storage duration.

[0040] Specifically, the transient memory layer stores the latest received raw input information (referred to as new information), that is, the immediate information in the current dialogue. The transient memory layer has a limited capacity but a fast response speed. The short-term memory layer stores information that has undergone preliminary processing and encoding, that is, memory units. These memory units have a certain degree of coherence and contextual relevance in the dialogue. The short-term memory layer has a larger capacity than the transient memory layer, but a slightly slower response speed. The long-term memory layer stores important memory units that have undergone importance assessment and structured organization. These memory units play a key role in the accuracy and depth of intelligent responses. The long-term memory layer has the largest capacity, but a relatively slow response speed, and is used to persist knowledge and experience.

[0041] In some embodiments, the hierarchical structure of the memory system can be flexibly adjusted according to the application scenario. For example, a simple dialogue system can adopt a two-layer structure (such as a working memory layer and a persistent memory layer) to reduce complexity; a professional medical diagnostic system can use a four-layer structure (such as a sensory memory layer, a working memory layer, a medium-term memory layer, and a long-term memory layer) to achieve fine-grained management; a personalized education system can adopt a five-layer structure (such as a transient memory layer, a short-term memory layer, a recent memory layer, a medium-term memory layer, and a long-term memory layer) to achieve ultra-fine time granularity. Special fields such as real-time monitoring, creative assistance, and research assistants can also customize exclusive hierarchical structures, and even support dynamic adjustment and user customization to ensure that the system meets functional requirements while optimizing resource utilization efficiency.

[0042] This hierarchical structure enables more efficient management and utilization of memory units, improving the quality and efficiency of responses. S22: The importance of memory units is calculated based on an importance assessment algorithm, and based on this importance, memory units are dynamically migrated between multiple memory layers.

[0043] Once new information enters the transient memory layer, the importance of the new information (or the memory unit encoded by that new information) is calculated in real time based on an importance assessment algorithm. Importance assessment primarily includes a comprehensive evaluation of emotional intensity and user attention. Emotional intensity is obtained through sentiment analysis of user interaction content. Specifically, this can be achieved through text sentiment analysis models, speech emotion recognition models, or multimodal emotion recognition technology, which quantify and analyze user emotional responses to obtain emotional intensity. User attention is obtained through behavioral analysis of user interactions. Specifically, this can be achieved by statistically analyzing or calculating behavioral parameters such as user dwell time, frequency of repeated inquiries, and frequency of proactive questioning during the interaction process, and then weighting these parameters to obtain user attention.

[0044] Importance is calculated through quantified emotional intensity and user attention. The calculation method can be a weighted combination, such as weighted addition or weighted multiplication, or a direct combination such as addition or multiplication; there are no specific limitations on the calculation method. Simultaneously, the importance calculated for each memory unit can be stored within that unit. For example, a memory unit can be represented as {ID: "M_001", Type: "Multimodal Unified", Unified Feature: [0.12, 0.45, ...], Timestamp: "2023-10-15 14:30", Importance: 0.78, Semantic Tags: ["User Interaction", "Technical Consultation", "Product Design"], Original Content: {...}}.

[0045] A dynamic migration mechanism is set up so that when new information enters the short-term memory layer and its importance is calculated, a decision is made based on the importance to determine whether to encode the new information as a memory unit and migrate it to the short-term memory layer. Similarly, for the memory units in the short-term memory layer, the importance is calculated in real time, and a decision is made based on the importance to determine whether to migrate the memory unit to the long-term memory layer. During the decision-making process, when the importance reaches the preset level standard of a higher level, the memory unit (or new information) is migrated to that higher level.

[0046] Unlike existing technologies where memory systems cannot automatically assess the importance of different memory contents, leading to the potential neglect of important memories and the consumption of excessive storage space by redundant information, this application provides a memory system-driven intelligent response method. This method constructs a dynamic importance assessment system to perform real-time importance calculations on memory units. Based on the calculation results, it intelligently manages the migration of memory units between different memory layers, ensuring that important memories are effectively retained and migrated to long-term memory layers in a timely manner. Redundant information is promptly cleared or compressed through an adaptive forgetting mechanism, thereby optimizing storage space utilization efficiency and improving the accuracy and response speed of intelligent responses.

[0047] S23: Based on the adaptive forgetting mechanism and importance, memory units that have not reached the preset level standard corresponding to the corresponding memory layer are forgotten.

[0048] An adaptive forgetting mechanism is implemented. When a memory unit (or new information) stored in a memory layer fails to meet the preset level standard corresponding to that memory layer, the memory unit is forgotten (i.e., deleted). Specifically, the preset level standard includes not only thresholds set for importance, but also thresholds corresponding to evaluation criteria such as the memory unit's historical usage frequency, time freshness, closeness of association with other memory units, and utility value in practical applications. If any evaluation criterion fails to meet the corresponding threshold, it indicates that the memory unit has not met the preset level standard corresponding to that memory layer, and the memory unit is chosen to be forgotten. Furthermore, in addition to forgetting a memory unit when its importance does not meet the preset level standard of its memory layer, it can also be forgotten when the memory unit is proven to be erroneous or outdated, when the memory unit contains highly repetitive and redundant content, or when the memory unit is temporary information with no relevant value.

[0049] In the preset hierarchical standards corresponding to each memory layer, the thresholds for the importance evaluation criterion can be different for the transfer and forgetting scenarios. For example, the importance value can be 0~1. In the preset hierarchical standards of the short-term memory layer, the importance is set to no less than 0.6 in the transfer scenario and to less than 0.2 in the forgetting scenario. In this case, when the importance of the original input information stored in the transient memory layer is 0.62, the original input information can be encoded into a memory unit and transferred to the short-term memory layer. At the same time, for memory units in the short-term memory layer with a real-time importance of 0.18, they are directly forgotten.

[0050] It should be noted that the above steps S21 to S23 are all performed in real time throughout the entire memory system, and there is no specific order of execution.

[0051] This application also includes: Obtain the retrieval frequency and activation time corresponding to each memory unit, and perform memory conflict detection on new original input information; The importance of the memory cell is updated in response to the memory cell's retrieval frequency being higher than the preset retrieval frequency; In response to the same activation time of multiple memory units, update the first association strength between the multiple memory units; In response to the detection of memory conflicts, the dynamic memory knowledge graph is updated based on a preset memory correction mechanism.

[0052] In this application, a dynamic memory update mechanism is set up for each memory unit in the dynamic memory knowledge graph. The memory unit is dynamically updated according to new information and the usage of the memory unit. The dynamic update can include memory reinforcement, memory correction and memory generalization.

[0053] In this application, memory enhancement is achieved by updating the importance and connection strength of memory units, including: updating the importance of memory units based on the retrieval frequency of memory units and their relevance to new information; and updating the first association strength of multiple memory units based on the fact that multiple memory units have the same activation time.

[0054] Specifically, when a memory unit is frequently retrieved or highly relevant to new information, its importance is automatically increased. The new importance is calculated by combining the original importance of the memory unit, the increment of the memory unit's access frequency, and the comprehensive relevance score between the memory unit and the new information. The calculation method for the comprehensive relevance score is described in step S21 above and will not be repeated here; the increment of access frequency is calculated based on the change in the number of times the memory unit is accessed within a specific time window. The combination calculation method can be a weighted combination, such as weighted addition or weighted multiplication, where the weights can be dynamically adjusted according to the application scenario. Alternatively, direct combination, such as addition or multiplication, can be used; there are no specific limitations on the calculation method.

[0055] When two memory units are activated at the same time, i.e., co-activated, the initial association strength between them increases. The new initial association strength can be calculated by combining the original initial association strength between the two memory units and the co-activation frequency. The co-activation frequency refers to the number of times the two memory units are simultaneously retrieved or activated within a specific time window. The calculation method can be a weighted combination, such as weighted addition or weighted multiplication, where the weights can be dynamically adjusted according to the application scenario. Alternatively, a direct combination such as addition or multiplication can be used; there are no specific limitations on the calculation method. For example, the new initial association strength = original initial association strength + γ × co-activation frequency; γ is the learning rate, which can be adjusted according to the application scenario to control the speed at which the initial association strength increases. In this application, memory correction includes: detecting memory conflicts between new information and existing memory units; and updating the corresponding memory units based on a preset memory correction mechanism in response to the detection of a memory conflict. The preset memory correction mechanism may involve determining whether to retain, correct, or delete existing memory units through a confidence level assessment.

[0056] Memory conflict refers to a contradiction or inconsistency between new information and already stored memory units in terms of facts, time, relationships, or attributes. Memory conflict detection methods can be one or more of the following: Semantic similarity analysis: Based on pre-trained language models (such as BERT or RoBERTa), the text content of new information and memory units are encoded into semantic vectors. The semantic relevance is measured by calculating the similarity between vectors. When the similarity exceeds a preset similarity threshold and there are differences between the new information and memory units in key facts, it is determined that a potential memory conflict has been detected. Entity alignment detection: Named entities are extracted using sequence labeling models (such as Bi-LSTM-CRF), and entity standardization is performed using knowledge graphs. New information is identified as different descriptions of the same entity in memory units. When the entity sets highly overlap but the attributes or relationships are contradictory, it is determined that a potential memory conflict has been detected. Logical consistency check: Based on the rule engine and external knowledge graph, by constructing fact triples and verifying their logical rationality, when a memory unit is found to have an inherent contradiction in the dimensions of time logic, spatial logic, and quantitative logic, it is determined that a potential memory conflict has been detected.

[0057] When a memory conflict is detected, a confidence assessment is performed on the new information and the memory unit, and the confidence scores for the new information and the memory unit are calculated separately. The confidence score is calculated based on a comprehensive evaluation of multiple dimensions, including the credibility level of the information source, the timeliness of the information, the sufficiency of supporting evidence, the consistency of the content logic, and the degree of matching with the external knowledge base.

[0058] Specifically, for new information, it is necessary to trace its original source and assess its credibility level, while also considering the time of its publication to determine its timeliness. Various supporting evidence should be collected and its sufficiency assessed. The logical consistency of the information content should be analyzed, and its matching degree with relevant content in external knowledge bases should be calculated. For memory units, similar evaluation from the above dimensions is required, including the credibility of the source when the memory unit was initially entered, the impact of storage time on timeliness, supporting evidence accumulated in past applications, internal logical consistency verification, and continuous matching with external knowledge bases. Finally, the confidence levels of new information and memory units are derived based on the comprehensive evaluation results.

[0059] Based on the confidence level of the new information and the memory unit, a decision is made to retain, modify, or delete the original memory unit. Specifically, when the confidence level of the memory unit > the confidence level of the new information + threshold 1, it indicates that the memory unit has abundant supporting evidence, the new information is isolated, and the memory unit has been verified multiple times, while the new information lacks verification; therefore, the decision is made to retain the original memory unit. At threshold 2, it indicates that the new information has been sufficiently verified, while the memory unit has been proven incorrect, and the memory unit has been unused for a long time and has low importance; therefore, it is decided to delete the original memory unit. When the confidence level of the new information differs from that of the memory unit within the range defined by thresholds 1 and 2, it indicates that both the new information and the memory unit have a certain degree of credibility, but it is impossible to clearly determine which one is more advantageous. At this time, further analysis of the content relevance and complementarity between the two is needed. If the new information and the memory unit have a complementary relationship in content and can jointly constitute a more complete and accurate knowledge system, it is decided to revise the original memory unit and integrate the new information into it to improve the accuracy and completeness of the memory unit. If the new information and the memory unit have a low content relevance and cannot form an effective complementarity, then both the original memory unit and the new information are retained. The settings of thresholds 1 and 2 can be dynamically adjusted according to specific application scenarios and needs, and are not specifically limited here.

[0060] In this application, memory generalization includes: identifying common patterns in multiple memory units through cluster analysis or frequent pattern mining; and forming higher-level abstract memory units based on the common patterns and establishing their association with multiple memory units.

[0061] Common patterns refer to recurring structures, rules, or features within multiple memory units. These patterns reflect the essential connections and commonalities between memory units. By identifying common patterns in multiple memory units through techniques such as cluster analysis and frequent pattern mining, these recurring key features are extracted to form higher-level abstract memory units. Furthermore, the relationships between these abstract memory units and the multiple memory units are established in a knowledge graph. By abstracting general knowledge from concrete experience, the intelligence level and reasoning ability of the intelligent agent are improved.

[0062] In some embodiments, the adaptive forgetting mechanism and the dynamic memory update mechanism can be adjusted according to different application scenarios, such as strengthening memory consolidation in educational scenarios and strengthening rapid forgetting in real-time interactive scenarios.

[0063] Unlike existing memory systems that use static storage, which cannot update, correct, or forget existing memories based on new information and lack a dynamic evolution mechanism similar to human memory, this application provides a memory system-driven intelligent response method. By setting up an adaptive forgetting mechanism and a dynamic memory update mechanism, it flexibly chooses to update, correct, forget, or simultaneously retain new information in existing memory units based on the correlation between new information and memory units, thereby simulating the dynamic evolution process of human memory. This mechanism not only enhances the flexibility and adaptability of the memory system but also makes intelligent responses more closely resemble human thought patterns, improving the accuracy and relevance of responses.

[0064] In this application, after constructing the memory unit, the following is also included: Based on semantic tags and timestamps, calculate the first association strength between the current memory unit and each memory unit in the dynamic memory knowledge graph; When the first association strength satisfies the first preset association condition, an association relationship is established between the current memory unit and the corresponding memory unit in the dynamic memory knowledge graph.

[0065] For memory units that have entered the short-term memory layer, i.e., newly constructed memory units, association analysis is performed. Specifically, based on the semantic label and timestamp of the memory unit, the semantic similarity and temporal proximity between the memory unit and other memory units are calculated, thereby establishing the association relationship between the memory unit and other memory units.

[0066] Semantic similarity refers to the degree of similarity between two memory units at the semantic level. This similarity can be calculated by analyzing and comparing the semantic labels of the memory units. For example, semantic feature vectors of memory units can be extracted using pre-trained semantic vector models (such as BERT and Word2Vec), and then the similarity between the corresponding semantic feature vectors of two memory units can be calculated. The calculation method can be cosine similarity or Euclidean distance, etc., to quantify the semantic similarity between the two memory units. Semantic similarity can be further refined into similarities or relationships that can be obtained based on semantic features, such as content similarity, causal relationship, and topic relevance.

[0067] Temporal proximity refers to the degree of closeness between two memory units in the time dimension. This proximity can be determined by comparing the timestamps of the memory units; the shorter the time interval, the stronger the association.

[0068] By calculating semantic similarity and temporal proximity, a first association strength between two memory units is obtained, and it is determined whether the first association strength meets a first preset association condition. The first preset association condition includes a semantic similarity threshold and a temporal proximity threshold, and these two thresholds can be dynamically set according to the application scenario. When the semantic similarity between two memory units reaches the semantic similarity threshold and the temporal proximity reaches the temporal proximity threshold, an association relationship is established between the two memory units.

[0069] By combining semantic similarity and temporal proximity, the relationships between memory units can be established more accurately, thereby improving the performance and accuracy of intelligent responses.

[0070] Furthermore, this association can be stored in a memory unit. For example, a memory unit can be represented as {ID:"M_001", Type: "Multimodal Unified", Unified Feature: [0.12, 0.45, ...], Timestamp: "2023-10-15 14:30", Importance: 0.78, Associative Memory: ["M_045", "M_123"], Semantic Tag: ["User Interaction", "Technical Consultation", "Product Design"], Original Content: {...}}, where the associative memory is the ID of the memory unit that has an association with this memory unit.

[0071] Furthermore, the memory units with related relationships in the short-term memory layer are organized in a structured manner. This structured organization is primarily achieved through memory association and the construction of a dynamic memory knowledge graph. Independent memory units are stored in the dynamically updated knowledge graph through these relationships. In the dynamic memory knowledge graph, connecting edges are established for related memory units, forming a structured memory network. Nodes represent memory units and conceptual entities, while edges represent various relationships between memory units.

[0072] The aforementioned importance assessment and structured organization primarily target the raw input information and memory units in the immediate and short-term memory layers. Through continuous importance assessment, the raw input information is encoded into memory units, and these memory units are transferred between different memory layers. Through continuous association analysis, important memory units in the short-term memory layer are gradually integrated into the dynamic memory knowledge graph corresponding to the long-term memory layer.

[0073] The dynamic memory knowledge graph in the long-term memory layer possesses greater stability and persistence, capable of storing selected and integrated key memory units for extended periods. These memory units are arranged and stored in the long-term memory layer according to specific logic and relationships, providing the agent with a rich and reliable knowledge base. When faced with new user interactions, technical consultations, or product design challenges, the agent can quickly retrieve and call upon relevant memory units from the dynamic memory knowledge graph in the long-term memory layer, thereby effectively improving the quality and efficiency of intelligent responses.

[0074] Unlike existing technologies that store historical interaction content as independent fragments, lacking semantic association and structured organization of the memory content, which prevents the agent from forming a coherent knowledge system and memory network, this application provides a memory system-driven intelligent response method. By constructing a multi-level memory system, it achieves refined processing and efficient organization of memory units. It not only focuses on the immediate storage of new information but also emphasizes the semantic association and logical integration between memory units, thereby constructing a dynamic memory knowledge graph. This enables the agent to form a structured memory network similar to that of humans, realizing semantic association and knowledge reasoning between memory content. It can transcend single interaction scenarios, forming a coherent knowledge system across domains and time sequences, providing more comprehensive and accurate knowledge support for intelligent response.

[0075] In this application, based on context vectors, a set of memory units related to context vectors are retrieved from a dynamic memory knowledge graph in multiple dimensions, including: S24: For memory units in a dynamic memory knowledge graph, calculate a comprehensive relevance score based on one or more preset relevance assessment factors.

[0076] After obtaining the context vector in step S10, a multi-dimensional retrieval is performed on the dynamic knowledge graph based on the context vector. During the retrieval process, the comprehensive correlation between the context vector and the memory units in the dynamic knowledge graph is evaluated to determine the set of memory units most relevant to the context vector.

[0077] During this process, you can choose to evaluate the relevance of each context vector to all memory units in the dynamic knowledge graph, or you can choose to pre-screen the memory units in the dynamic knowledge graph using preset filtering conditions to remove obviously irrelevant memory units, thereby reducing the computational load of subsequent relevance evaluations. Then, you can perform a relevance evaluation on the memory units that have undergone the pre-screening. Pre-screening conditions can include semantic tag matching, time range limitations, etc., and can be flexibly set according to specific application scenarios; no specific limitations are specified here. For example, if the context vector corresponds to technical consultation about product design, you can first filter out memory units whose semantic tags contain "product design" and "technical consultation," as well as memory units whose timestamps are within a reasonable time range (e.g., within 10 days).

[0078] For a context vector and multiple memory units to be relevance evaluated, a comprehensive relevance score between the context vector and each memory unit is calculated based on one or more preset relevance evaluation factors. Specifically, the preset relevance evaluation factors can be semantic similarity, time decay factor, and importance, etc. Semantic similarity is the same as calculating the degree of similarity between two memory units at the semantic level, except that it is used here to calculate the semantic similarity between the context vector and the memory unit, and can also be obtained by calculating cosine similarity, Jaccard similarity, or Euclidean distance, etc.; the time decay factor reflects the forgetting trend of the memory unit over time and is calculated based on the time difference between the creation time of the memory unit and the current time; importance is calculated based on importance evaluation. It should be noted that the semantic relevance calculation operates on the context vector and memory unit, while the time decay factor and importance are attributes of the memory unit itself.

[0079] Based on the quantitative values ​​of one or more preset relevance assessment factors mentioned above, a comprehensive relevance score is calculated. The calculation method can be a weighted combination, such as weighted addition or weighted multiplication, and the weights can be dynamically adjusted according to the application scenario. Alternatively, they can be directly combined, such as addition or multiplication, without specific limitations on the calculation method.

[0080] S25: Based on the comprehensive relevance score and combined with at least one retrieval dimension independent of the comprehensive relevance score, retrieve a set of memory units related to the context vector from the dynamic memory knowledge graph.

[0081] After obtaining the overall relevance score, you can also combine at least one retrieval dimension independent of the overall relevance score, such as time proximate cause or sentiment consistency, to retrieve memory units in a multi-dimensional way.

[0082] In this application, at least one retrieval dimension independent of the overall relevance score can be time recency. Time recency refers to the degree of proximity of a memory unit to the current context in the time dimension, reflecting the "freshness" of the memory unit. In multi-dimensional retrieval, time recency participates in the ranking of memory units as an independent retrieval dimension. It is related to the time decay factor in the overall relevance score but has different functions. Among them, time recency directly reflects the timeliness value of memory, while the time decay factor is a parameter in the calculation of the overall relevance score.

[0083] The proximate time factor is calculated using an exponential decay function based on the time difference between the creation time of the memory cell and the current time. The formula is: Proximate Time Factor = exp(-λ × Time Difference / Base Time Unit), where λ is a decay coefficient dynamically adjusted according to the type of memory cell, the base time unit is a preset value set according to the application scenario, and the time difference is obtained by calculating the difference between the timestamp in the memory cell and the current time.

[0084] In this application, at least one retrieval dimension independent of the comprehensive relevance score can be sentiment consistency. Sentiment consistency refers to the degree of fit between the memory unit and the current context in terms of sentiment, reflecting the "emotional fit" of the memory unit. In multi-dimensional retrieval, sentiment consistency participates in the ranking of memory units as an independent retrieval dimension, directly reflecting the degree of matching between the memory and the current emotional atmosphere. Sentiment consistency is calculated based on matching the sentiment intensity in the memory unit with the user's emotion in the context vector. This calculation can be achieved by constructing an sentiment vector space, mapping sentiment intensity and user emotion to this space respectively, and measuring sentiment consistency by calculating the similarity between the two vectors; the higher the similarity, the stronger the sentiment consistency.

[0085] Based on the comprehensive relevance score, and combined with at least one retrieval dimension independent of the comprehensive relevance score, a set of memory units related to the context vector is retrieved from the dynamic memory knowledge graph. During this process, unique retrieval thresholds can be set for each of these retrieval dimensions, and memory units exceeding the threshold in any retrieval dimension can be considered as memory units related to the context vector; alternatively, retrieval thresholds can be set by combining the retrieval dimensions, and memory units that simultaneously meet the conditions corresponding to multiple dimensions can be considered as memory units related to the context vector. Here, the retrieval thresholds are not specifically limited and can be flexibly adjusted according to the application scenario.

[0086] In some embodiments, the retrieval dimensions may also include importance or graph structure, and the retrieval strategy may also include adaptive retrieval optimized by reinforcement learning, or these retrieval strategies may be used in combination to dynamically adjust the weights, so as to flexibly select the best retrieval method according to different scenarios.

[0087] Unlike existing technologies where the accumulation of memory content makes it difficult to efficiently and accurately retrieve the most relevant memory fragments for the current context, leading to a decline in response quality and efficiency, this application provides a memory system-driven intelligent response method. By introducing a comprehensive relevance score and a multi-dimensional retrieval mechanism, the method offers diverse and dynamically adjustable retrieval methods. It accurately selects memory units highly relevant to the context vector from a dynamic memory knowledge graph, ensuring that even as memory content accumulates, the most suitable memory fragments for the current context can still be retrieved efficiently and accurately, significantly improving the response quality and efficiency of intelligent responses.

[0088] S30: Based on the preset memory association analysis rules, construct a set of association relationships between memory units to obtain a memory subgraph.

[0089] In this application, based on preset memory association analysis rules, a set of association relationships between memory units is constructed to obtain a memory subgraph, including: S31: Calculate the second association strength between a group of memory units based on semantic labels, timestamps, and the first association strength.

[0090] For the set of memory units related to the context vector obtained in step S20 above, an association analysis is performed between each memory unit. Similar to the association analysis method of calculating the first association strength based on the semantic tags and timestamps of memory units, the second association strength is calculated here. In addition to calculating the semantic similarity and temporal proximity between two memory units, the original first association strength between the two memory units is also used as a new dimension for association analysis. This can be understood as the association structure before the update also need to be considered during the dynamic update of the dynamic memory knowledge graph.

[0091] By calculating semantic similarity and temporal proximity, and combining them with the first association strength, a second association strength between two memory units is obtained, and it is determined whether the second association strength satisfies the second preset association condition. The second preset association condition includes not only semantic similarity thresholds and temporal proximity thresholds, but also the first association strength threshold, and these three thresholds can be dynamically set according to the application scenario. The criterion for determining whether the second association strength satisfies the second preset association condition can be: when the semantic similarity between two memory units reaches the semantic similarity threshold, the temporal proximity reaches the temporal proximity threshold, and the first association strength reaches the first association strength threshold, an association relationship is established between the two memory units; or when any one of semantic similarity, temporal proximity, and the first association strength reaches the corresponding threshold, an association relationship is established between the two memory units. The settings of the semantic similarity threshold and temporal proximity threshold in the second preset association condition can be different from those in the first preset association condition.

[0092] S32: In response to the second association strength satisfying the second preset association condition, construct the association relationship between the corresponding memory units in the dynamic memory knowledge graph to obtain the memory subgraph.

[0093] When the second association strength of two memory units satisfies the second preset association condition, an association relationship is established between the two memory units. The second association strength is calculated for each pair of memory units obtained in step S20, and an association relationship is established accordingly. Establishing an association relationship means building or updating edges between two nodes in the dynamic memory knowledge graph to obtain a memory subgraph with this set of memory units as nodes from the dynamic memory knowledge graph.

[0094] The memory subgraph obtained through the above steps can more accurately reflect the relationship between memory units, providing a richer and more accurate information basis for subsequent intelligent responses, enabling intelligent responses to better meet user needs and provide more personalized and intelligent services.

[0095] S40: Based on the memory subgraph, multimodal fusion is performed on the memory fragments corresponding to a set of memory units to generate natural language responses.

[0096] In this application, based on a memory subgraph, multimodal fusion is performed on memory segments corresponding to a set of memory units to generate natural language responses, including: S41: Based on the preset timeline reconstruction method and the preset semantic coherence processing method, sort the memory segments corresponding to each memory unit in the memory subgraph to obtain coherent memory segments.

[0097] For each memory unit in the memory subgraph, the memory segments are sorted based on a pre-defined timeline reconstruction method. Specifically, time analysis techniques can be used to process the time information in each memory segment, semantic analysis can be used to identify the causal relationships between memory segments, and the memory segments can be sorted according to time order and causal relationship order.

[0098] For each sorted memory fragment, the fragments are integrated based on a pre-defined semantic coherence processing method. Specifically, entity recognition and referential resolution techniques are used to unify the entity references in each memory fragment, knowledge graph reasoning techniques are used to ensure the logical consistency of each memory fragment, content contradictions are detected and eliminated, and the fragments are integrated into a coherent memory fragment.

[0099] S42: Perform multimodal fusion on coherent memory fragments and generate natural language responses based on a pre-trained language model.

[0100] Multimodal fusion is performed on the integrated coherent memory fragments. Specifically, for memory fragments of different modalities (text, images, audio, and video, etc.), key features of each modality are extracted and transformed into a unified feature representation by employing corresponding feature extraction techniques and representation learning methods for each modality. Then, a multimodal fusion algorithm is used to effectively fuse these features, enabling information from different modalities to complement and enhance each other, thereby obtaining fused features containing rich information and achieving effective integration of cross-modal information.

[0101] Finally, natural language descriptions are generated based on the integrated features from the pre-trained language model. A coherent and accurate memorized text is output as context through domain fine-tuning and generation control strategies. The agent then generates natural language responses to the input query based on this context. Domain fine-tuning involves further training the pre-trained language model for a specific domain or task to better adapt it to the language understanding and generation needs of that particular scenario. Generation control strategies control and optimize the text generation process of the pre-trained language model, such as controlling the length, style, and logic of the generated text. This ensures that the output memorized text is not only coherent and accurate but also meets the diverse needs of users. This allows the agent to more accurately understand user intent and provide more personalized and intelligent response services after receiving this contextual information.

[0102] During user interaction, user behavior data and feedback can be recorded. Preference features can be extracted from the user's language style and selection patterns, and encoded into preference memory units to record the user's personalized tendencies during use. Like general memory units, preference memory units are persistently stored in the memory system. They adopt the same structural representation as general memory units, while adding specific attributes such as preference category, preference content, and confidence level. For example, a preference memory unit can be represented as {ID: "P_001", Type: "User Preference", Uniform Feature: [0.23, 0.67, ...], Time: "2023-10-15 14:30", Importance: 0.85, Associative Memory: ["M_045", "M_123"], Preference Category: "Interaction Method", Preference Content: "Concise and Clear Answer", Confidence Level: 0.9}.

[0103] In subsequent interactions, the agent can adjust and optimize the generated responses based on these preference memory units. For example, it can reply using the user's preferred language style or anticipate user needs based on the user's selection pattern, thereby further improving the personalization and intelligence of the intelligent response and providing users with a more considerate and personalized interactive experience that suits their habits.

[0104] In one specific embodiment, the user inputs a natural language query through the interactive interface: "Help me recall the progress of the machine learning project we discussed last time." In response to this query, the query is encoded into a context vector based on current context information, and memory units related to key features such as "machine learning" and "project progress" are retrieved from the dynamic memory knowledge graph. The retrieval reveals a set of memory units including project discussion records, code submission history, and performance evaluation reports. Using preset memory association analysis rules, the relationships between these memory units are constructed, resulting in a memory subgraph. Further, based on the memory subgraph, multimodal fusion is performed on the memory fragments corresponding to these memory units, integrating them into a coherent summary of project progress as a memory context provided to the agent for response. The agent generates a detailed and coherent natural language response: "According to our records, the machine learning project we discussed last time has achieved the following progress: 1. Model accuracy improved by 15%... 2. Overfitting problem was solved... 3. Data processing flow was optimized...". Record the query and response process, add the information as a new memory unit to the memory system, and update the importance of the relevant memory units.

[0105] This application provides an intelligent response device driven by a memory system, see reference. Figure 2 , Figure 2 This is a schematic diagram of an embodiment of the memory system-driven intelligent response device provided in this application. The memory system-driven intelligent response device includes: Encoding module 10 is used to encode the query statement into a context vector in response to the input query statement and the current context information; The retrieval module 20 is used to retrieve a set of memory units related to the context vector in a dynamic memory knowledge graph based on the context vector in multiple dimensions. The association module 30 is used to construct a set of association relationships between memory units based on preset memory association analysis rules to obtain a memory subgraph; The generation module 40 is used to perform multimodal fusion on memory fragments corresponding to a set of memory units based on a memory subgraph to generate a natural language response.

[0106] The above-mentioned encoding module 10, retrieval module 20, association module 30 and generation module 40 interact to realize the intelligent response based on the memory system. You can refer to the specific description of steps S10 to S40 above. The repeated parts will not be repeated here.

[0107] See Figure 3 , Figure 3 This is a schematic diagram of an embodiment of the storage medium provided in this application.

[0108] The storage medium 300 stores program data 310, which, when executed by the processor, implements, as follows: Figure 1 The steps of the memory system-driven intelligent response method described herein.

[0109] The program data 310 is stored in a storage medium 300 and includes several instructions for causing a network device (which may be a router, personal computer, server, or other network device) or processor to execute all or part of the steps of the methods described in the various embodiments of this application.

[0110] Optionally, the storage medium 300 can be any medium capable of storing program data, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), disk, or optical disc.

[0111] See Figure 4 , Figure 4 This is a schematic diagram of the structure of an embodiment of the computer device provided in this application.

[0112] The device 400 includes a processor 420 and a memory 410 connected to each other. The memory 410 stores a computer program. When the processor 420 executes the computer program, it implements the steps of the memory system-driven intelligent response method described above.

[0113] Unlike existing technologies, this application discloses a memory system-driven intelligent response method, apparatus, device, and medium. This method significantly enhances the memory capabilities of AI agents by storing memories as memory units in a dynamic memory knowledge graph. Based on the context vector encoded from the input query statement, multi-dimensional retrieval is performed within this dynamic knowledge graph to quickly and accurately retrieve the most relevant memory units. Furthermore, multimodal fusion generates accurate and coherent natural language responses, thereby improving both response efficiency and quality.

[0114] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the storage medium embodiments and computer device embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0115] This application can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. For example: personal computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputers, distributed computing environments including any of the above systems or devices, etc.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative; multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed.

[0117] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0118] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0119] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A memory-based intelligent response method, characterized in that, include: In response to the input query statement, the query statement is encoded into a context vector in combination with the current context information; Based on the context vector, a set of memory units related to the context vector are retrieved in a multi-dimensional manner in the dynamic memory knowledge graph; Based on preset memory association analysis rules, a set of association relationships between the memory units is constructed to obtain a memory subgraph; Based on the memory subgraph, multimodal fusion is performed on the memory segments corresponding to a set of memory units to generate natural language responses.

2. The intelligent response method based on a memory system as described in claim 1, characterized in that, The step of retrieving a set of memory units related to the context vector in a dynamic memory knowledge graph from multiple dimensions based on the context vector includes: For the memory unit in the dynamic memory knowledge graph, a comprehensive relevance score is calculated based on one or more preset relevance evaluation factors; Based on the comprehensive relevance score, and in combination with at least one retrieval dimension independent of the comprehensive relevance score, a set of memory units related to the context vector are retrieved from the dynamic memory knowledge graph.

3. The intelligent response method based on a memory system as described in claim 1, characterized in that, The construction process of the memory unit is as follows, including: The system receives raw input information from multiple modalities and extracts features from the raw input information based on the encoders corresponding to each modality. The features from each modality are then fused and atomized into memory units that contain at least semantic labels and timestamps.

4. The intelligent response method based on a memory system as described in claim 3, characterized in that, After constructing the memory unit, the following is also included: The memory units are managed based on a hierarchical structure containing multiple memory layers; the importance of the memory units is calculated based on an importance evaluation algorithm, and the memory units are dynamically migrated between the multiple memory layers based on the importance; based on an adaptive forgetting mechanism and the importance, memory units that have not reached the preset level standard corresponding to the memory layer are forgotten.

5. The intelligent response method based on a memory system as described in claim 3, characterized in that, After constructing the memory unit, the following is also included: Based on the semantic tag and the timestamp, calculate the first association strength between the current memory unit and each memory unit in the dynamic memory knowledge graph; When the first association strength satisfies the first preset association condition, the association relationship between the current memory unit and the corresponding memory unit is established in the dynamic memory knowledge graph.

6. The intelligent response method based on a memory system as described in claim 5, characterized in that, Based on preset memory association analysis rules, a set of association relationships between the memory units is constructed to obtain a memory subgraph, including: Based on the semantic tag, the timestamp, and the first association strength, calculate the second association strength between a group of memory units; When the second association strength satisfies the second preset association condition, the association relationship between the corresponding memory units in the dynamic memory knowledge graph is constructed to obtain the memory subgraph.

7. The intelligent response method based on a memory system as described in claim 1, characterized in that, The step of performing multimodal fusion on memory segments corresponding to a set of memory units based on the memory subgraph to generate a natural language response includes: Based on a preset timeline reconstruction method and a preset semantic coherence processing method, the memory segments corresponding to each memory unit in the memory subgraph are sorted to obtain coherent memory segments. The coherent memory fragments are fused using multimodal methods, and the natural language response is generated based on a pre-trained language model.

8. The intelligent response method based on a memory system as described in claim 5, characterized in that, Also includes: Obtain the retrieval frequency and activation time corresponding to each memory unit, and perform memory conflict detection on the new original input information; In response to the memory cell's retrieval frequency being higher than a preset retrieval frequency, the importance of the memory cell is updated; In response to the fact that the activation times of the plurality of memory cells are the same, the first association strength between the plurality of memory cells is updated; In response to the detection of the memory conflict, the dynamic memory knowledge graph is updated based on a preset memory correction mechanism.

9. A memory-based intelligent response device, characterized in that, include: The encoding module is used to encode the input query statement into a context vector in response to the input query statement, combined with the current context information. The retrieval module is used to retrieve a set of memory units related to the context vector in a dynamic memory knowledge graph from multiple dimensions based on the context vector. The association module is used to construct a set of association relationships between the memory units based on preset memory association analysis rules, thereby obtaining a memory subgraph; The generation module is used to perform multimodal fusion on memory fragments corresponding to a group of memory units based on the memory subgraph to generate a natural language response.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the memory system-driven intelligent response method according to any one of claims 1-8.

11. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the memory system-driven intelligent response method as described in any one of claims 1-8.

Images