Large language model intelligent interaction method and system with memory management capability

Abstract

The invention provides a large language model intelligent interaction method and system with memory management capability in the technical field of artificial intelligence. The method comprises the following steps: S1, creating a work memory module, a long-term memory module, a memory fusion module and an intelligent interaction module; s2, acquiring external knowledge and integrating the external knowledge into a long-term memory module; s3, the intelligent interaction module obtains an intention sequence based on a user query statement and sends the intention sequence to a work memory module; s4, the memory fusion module retrieves context information from the long-term memory module based on the user query statement and sends the context information to the working memory module; s5, the work memory module constructs a work memory sending intelligent interaction module based on the intention sequence, the context information and the system instruction; s6, the intelligent interaction module outputs a query result based on the working memory; s7, a memory fusion module extracts semantic memory and plot memory from user query statements and query results and stores the semantic memory and plot memory in a long-term memory module. The method has the advantage that the man-machine interaction performance is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large language model intelligent interaction method and system with memory management capability. Background Art

[0002] Intelligent assistants are a type of software used for intelligent human-computer interaction. In the early stages of development of intelligent assistants, fixed rules and templated response mechanisms were mainly used to process user queries. Although easy to implement, they have obvious limitations, mainly manifested in the inflexibility of processing capabilities and excessive reliance on specific question types, which limits the application potential of intelligent assistants in diverse and complex interaction scenarios.

[0003] With the rapid evolution of artificial intelligence technology, machine learning has become the key to promoting the development of intelligent assistants. In particular, large language models (LLMs), with their ability to handle complex language tasks, have become the core of building advanced intelligent assistants and are widely used in a variety of interactive scenarios. These large language models use their advanced algorithms and huge parameter sets to achieve efficient processing of complex language tasks, bringing users an unprecedented natural and intelligent interactive experience.

[0004] The core goal of designing intelligent assistants based on large language models is to achieve natural communication capabilities similar to those of humans. Therefore, it is a natural and necessary choice to design intelligent assistants in accordance with human working mechanisms. To achieve this goal, it is particularly important to imitate human communication and memory capabilities. The human brain has the unique ability to store, process, and update a large amount of knowledge in a dynamic environment. If intelligent assistants want to achieve a similar level of communication naturalness and adaptability, they must have effective memory management functions to support complex cognitive processes and continuous knowledge accumulation.

[0005] However, limited by the complexity of the attention mechanism of large language models, the computational complexity and resource requirements increase exponentially as the length of the input sequence increases, which limits the ability of intelligent assistants to store and recall information obtained during interactions with users, thereby limiting the interactive performance.

[0006] Therefore, how to provide a large language model intelligent interaction method and system with memory management capabilities to improve human-computer interaction performance has become a technical problem that needs to be solved urgently. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a large language model intelligent interaction method and system with memory management capabilities to improve human-computer interaction performance.

[0008] In a first aspect, the present invention provides a large language model intelligent interaction method with memory management capability, comprising the following steps:

[0009] Step S1, creating a working memory module, a long-term memory module, a memory fusion module and an intelligent interaction module with a large language model;

[0010] Step S2: Acquire external knowledge for intelligent interaction, and integrate the external knowledge into the long-term memory module;

[0011] Step S3, the intelligent interaction module identifies the user query statement through the large language model to obtain the intention sequence, and sends the intention sequence to the working memory module;

[0012] Step S4, the memory fusion module retrieves relevant context information from the external knowledge in the long-term memory module based on the user query statement, and sends the context information to the working memory module;

[0013] Step S5, the working memory module constructs a dynamic working memory based on the intention sequence, context information and preset system instructions, stores the working memory and sends it to the intelligent interaction module;

[0014] Step S6: The intelligent interaction module outputs the query result through the large language model based on the working memory to perform intelligent interaction;

[0015] Step S7: During the intelligent interaction process, the memory fusion module extracts semantic memory and episodic memory from the user query statement and query results, and stores the semantic memory and episodic memory in the long-term memory module.

[0016] Furthermore, in step S1, the working memory module is used for short-term storage and processing of information, and is provided with an instruction management unit, a context association unit, and a user intention input unit;

[0017] The instruction management unit is used to define system instructions that guide the interaction process, namely static rules; the context association unit is used to dynamically store interaction fact information, user preferences and role information; and the user intention input unit is used to input intention sequences.

[0018] Furthermore, in step S1, the long-term memory module is used for long-term storage and processing of information, and is provided with an injection memory unit, a semantic memory unit, and an episodic memory unit;

[0019] The injection memory unit is used to perform text vectorization processing on external knowledge through the bge-base-zh text embedding model to obtain vectorized knowledge, and after embedding the vectorized knowledge, store it in the Milvus database in the form of high-dimensional vectors; the semantic memory unit is used to embed organized knowledge and store it in the Milvus database in the form of high-dimensional vectors; the plot memory unit is used to encode and store the characters involved in the interaction, the time of the event, the place of the event and the environment in which the event occurs.

[0020] Furthermore, in step S1, the memory fusion module is used for data flow between the working memory module and the long-term memory module, and is provided with a dynamic working memory construction unit, a semantic memory extraction unit, and an episodic memory extraction unit;

[0021] The dynamic working memory construction unit is used to retrieve context information from the long-term memory module based on the user query statement, and then use the BCE-Reranker model to rearrange the context information, combine the rearranged context information with the system instructions of the instruction management unit, and format it according to the input specifications preset by the large language model, so as to construct a complete working memory for processing the current user query statement; the semantic memory extraction unit is used to interact based on the current working memory, read the semantic knowledge carried by the context information generated during the interaction, and store the semantic knowledge in the long-term memory module; the episodic memory extraction unit is used to extract episodic memory including characters, event time, event location and event environment from the context information, generate a high-level summary based on the episodic memory and store it in the long-term memory module.

[0022] Furthermore, in step S1, the interaction module is used to mine surface meanings and deep needs from user query statements, generate an intention sequence related to user intentions based on the surface meanings and deep needs, i.e., a keyword sequence, and perform intelligent interaction based on the intention sequence.

[0023] In a second aspect, the present invention provides a large language model intelligent interaction system with memory management capability, comprising the following modules:

[0024] An initialization module, used to create a working memory module, a long-term memory module, a memory fusion module, and an intelligent interaction module with a large language model;

[0025] An external knowledge integration module, used to obtain external knowledge for intelligent interaction and integrate the external knowledge into the long-term memory module;

[0026] An intention recognition module is used for the intelligent interaction module to recognize the user query statement through a large language model to obtain an intention sequence, and send the intention sequence to the working memory module;

[0027] A context information retrieval module, used for the memory fusion module to retrieve relevant context information from the external knowledge in the long-term memory module based on the user query statement, and send the context information to the working memory module;

[0028] A working memory construction module, which is used for the working memory module to construct a dynamic working memory based on the intention sequence, context information and preset system instructions, store the working memory and send it to the intelligent interaction module;

[0029] An interaction module, used for the intelligent interaction module to output query results through the large language model based on working memory to perform intelligent interaction;

[0030] The long-term memory optimization module is used in the intelligent interaction process. The memory fusion module extracts semantic memory and episodic memory from user query statements and query results, and stores the semantic memory and episodic memory in the long-term memory module.

[0031] Further, in the initialization module, the working memory module is used for short-term storage and processing of information, and is provided with an instruction management unit, a context association unit, and a user intention input unit;

[0032] The instruction management unit is used to define system instructions that guide the interaction process, namely static rules; the context association unit is used to dynamically store interaction fact information, user preferences and role information; and the user intention input unit is used to input intention sequences.

[0033] Furthermore, in the initialization module, the long-term memory module is used for long-term storage and processing of information, and is provided with an injection memory unit, a semantic memory unit, and an episodic memory unit;

[0034] The injection memory unit is used to perform text vectorization processing on external knowledge through the bge-base-zh text embedding model to obtain vectorized knowledge, and after embedding the vectorized knowledge, store it in the Milvus database in the form of high-dimensional vectors; the semantic memory unit is used to embed organized knowledge and store it in the Milvus database in the form of high-dimensional vectors; the plot memory unit is used to encode and store the characters involved in the interaction, the time of the event, the place of the event and the environment in which the event occurs.

[0035] Furthermore, in the initialization module, the memory fusion module is used for data flow between the working memory module and the long-term memory module, and is provided with a dynamic working memory construction unit, a semantic memory extraction unit, and an episodic memory extraction unit;

[0036] The dynamic working memory construction unit is used to retrieve context information from the long-term memory module based on the user query statement, and then use the BCE-Reranker model to rearrange the context information, combine the rearranged context information with the system instructions of the instruction management unit, and format it according to the input specifications preset by the large language model, so as to construct a complete working memory for processing the current user query statement; the semantic memory extraction unit is used to interact based on the current working memory, read the semantic knowledge carried by the context information generated during the interaction, and store the semantic knowledge in the long-term memory module; the episodic memory extraction unit is used to extract episodic memory including characters, event time, event location and event environment from the context information, generate a high-level summary based on the episodic memory and store it in the long-term memory module.

[0037] Furthermore, in the initialization module, the interaction module is used to mine surface meanings and deep needs from user query statements, generate an intention sequence related to user intentions based on the surface meanings and deep needs, i.e., a keyword sequence, and perform intelligent interaction based on the intention sequence.

[0038] The advantages of the present invention are:

[0039] 1. By creating a working memory module, a long-term memory module, a memory fusion module and an intelligent interaction module with a large language model, external knowledge for intelligent interaction is obtained and integrated into the long-term memory module. The intelligent interaction module recognizes the user query sentence output by the large language model to obtain the intention sequence and sends it to the working memory module. The memory fusion module retrieves relevant context information from the external knowledge in the long-term memory module based on the user query sentence and sends it to the working memory module. Then the working memory module builds a dynamic working memory based on the intention sequence, context information and preset system instructions, stores the working memory and sends it to the intelligent interaction module. The intelligent interaction module is based on the working memory, inputs the large language model, and obtains the intention sequence. The query results are output for intelligent interaction. During the intelligent interaction process, the memory fusion module extracts semantic memory and episodic memory from the user query statements and query results and stores them in the long-term memory module; that is, it creates two memory mechanisms including working memory and long-term memory, and supports self-optimization. The working memory module is used for short-term storage and processing of information, and the long-term memory module is used for long-term storage and processing of information. The two memory mechanisms work together to achieve efficient retrieval and application of user query statements, promote deep integration of knowledge through the memory fusion module, and interact with the intention sequence related to the user's intention through the intelligent interaction module, which significantly improves the performance of the large language model in continuous dialogue and execution of complex tasks, and ultimately greatly improves the human-computer interaction performance.

[0040] 2. By deeply analyzing the specific goals that users hope to achieve through dialogue, generating intent sequences closely related to user intentions for interaction, supporting multiple rounds of in-depth dialogues, and gradually guiding users to their ultimate goals, we can achieve a deep understanding of user intentions and respond in a timely manner, greatly improving the user experience and greatly enhancing user satisfaction.

[0041] 3. By drawing on the declarative memory mechanism of the human brain, the semantic memory and episodic memory in the interaction process are innovatively extracted to achieve self-optimization of long-term memory; the memory enhancement strategy of the memory fusion module realizes dynamic storage and accumulation of experiential knowledge, forming an in-depth understanding of user preferences and needs, thereby providing more accurate and personalized services.

[0042] 4. Adopting the core concept of Retrieval Augmented Generation (RAG) and combining it with advanced Chinese text embedding technology (bge-base-zh text embedding model), key external knowledge is converted into vectorized knowledge and directly integrated into the long-term memory module, effectively improving the efficiency of knowledge storage and the speed of retrieval, and enhancing the processing capabilities in complex interactive scenarios, so as to more accurately understand and respond to user needs.

[0043] 5. Retrieve closely related contextual information from long-term memory based on user query statements, and then combine it with system instructions and intent sequences to effectively enhance the naturalness and accuracy of the interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0045] Figure 1 It is a flow chart of a large language model intelligent interaction method with memory management capability of the present invention.

[0046] Figure 2 It is a structural schematic diagram of a large language model intelligent interaction system with memory management capability according to the present invention.

[0047] Figure 3 It is a schematic diagram of the interaction process of the present invention. DETAILED DESCRIPTION

[0048] The technical solution in the embodiments of the present application has the following overall idea: based on the hippocampus index theory in human long-term memory (the hippocampus is responsible for quickly indexing and storing new experiences and knowledge in the brain, and the neocortex is responsible for long-term storage and stabilization of this information), two memory mechanisms including working memory and long-term memory are created for intelligent interaction. The working memory module is used for short-term storage and processing of information, and the long-term memory module is used for long-term storage and processing of information. The two memory mechanisms work together to achieve efficient retrieval and application of user query statements, promote deep integration of knowledge through the memory fusion module, and interact with the intention sequence related to the user's intention through the intelligent interaction module, which significantly improves the performance of the large language model in continuous dialogue and execution of complex tasks, thereby improving the human-computer interaction performance.

[0049] Please refer to Figures 1 to 3 As shown, a preferred embodiment of the large language model intelligent interaction method with memory management capability of the present invention comprises the following steps:

[0050] Step S1, creating a working memory module, a long-term memory module, a memory fusion module and an intelligent interaction module with a large language model;

[0051] Step S2: Acquire external knowledge for intelligent interaction, and integrate the external knowledge into the long-term memory module;

[0052] Step S3, the intelligent interaction module identifies the user query statement through the large language model to obtain the intention sequence, and sends the intention sequence to the working memory module;

[0053] Step S4, the memory fusion module retrieves relevant context information from the external knowledge in the long-term memory module based on the user query statement, and sends the context information to the working memory module;

[0054] Step S5, the working memory module constructs a dynamic working memory based on the intention sequence, context information and preset system instructions, stores the working memory and sends it to the intelligent interaction module;

[0055] Step S6: The intelligent interaction module outputs the query result through the large language model based on the working memory to perform intelligent interaction;

[0056] Step S7: During the intelligent interaction process, the memory fusion module extracts semantic memory and episodic memory from the user query statement and query results, and stores the semantic memory and episodic memory in the long-term memory module.

[0057] In step S1, the working memory module is used for short-term storage and processing of information, and is provided with an instruction management unit, a context association unit, and a user intention input unit;

[0058] The instruction management unit is used to define system instructions that guide the interaction process, namely static rules; the context association unit is used to dynamically store interaction fact information, user preferences and role information; and the user intention input unit is used to input intention sequences.

[0059] That is, the working memory module imitates the short-term memory mechanism of the human brain to provide instant data support for our thinking and decision-making, ensuring that the large language model can deeply understand user needs and their situational background, more accurately build communication with users, support dynamic adjustment of communication strategies based on working memory, and select appropriate language and expressions to achieve more natural and coherent dialogues; the static rules clarify how to process various retrieved knowledge information to ensure the consistency and accuracy of the interaction process; the context association unit updates and maintains factual information, user preferences and role information in real time, so that the large language model can have more natural, coherent and personalized dialogues with users; the user intention recognition unit is used to ensure that each communication can accurately capture and reflect the user's expectations, achieve efficient and targeted communication, and gradually guide the user to approach their ultimate goal. It is composed of a series of intention sequences (keyword sequences) closely linked to the goal. The intention sequence not only indicates the logical progression from the current dialogue to the achievement of the goal, but also ensures the directionality and coherence of the communication.

[0060] In step S1, the long-term memory module is used for long-term storage and processing of information, and is provided with an injection memory unit, a semantic memory unit and an episodic memory unit;

[0061] The injection memory unit is used to perform text vectorization processing on external knowledge through the bge-base-zh text embedding model to obtain vectorized knowledge, and after embedding the vectorized knowledge, store it in the Milvus database in the form of high-dimensional vectors; the semantic memory unit is used to embed organized knowledge and store it in the Milvus database in the form of high-dimensional vectors; the plot memory unit is used to encode and store the characters involved in the interaction, the time of the event, the place of the event and the environment in which the event occurs.

[0062] The long-term memory module is similar to the neocortex of the human brain, responsible for long-term storage and stabilization of this information. It is a key component of the large language model, which not only ensures the persistence of information, but also enables the large language model to quickly retrieve and use this information when needed; the injected memory unit is used to improve the context understanding ability when communicating with users, and directly integrates important external knowledge into the long-term memory module. This process draws on the core concept of Retrieval-Augmented Generation (RAG), and embeds the vectorized knowledge and stores it in the form of high-dimensional vectors in Mil vus database, not only the persistence of data is guaranteed, but also it is convenient for subsequent rapid retrieval and update; semantic memory is an important part of the human memory system, which is responsible for storing organized knowledge, such as words, concepts, the relationship between words and concepts, rules and formulas, etc. The semantic memory unit plays a similar role, imitating the way humans extract key information when recalling experiences, and through long-term interaction, abstracting the observations in multiple rounds of dialogue into semantic memory for storage, thereby continuously improving its own knowledge system; for example, by analyzing the patterns and trends in the dialogue to gain insights into personality traits, it can more accurately predict and respond to user needs; episodic memory refers to the memory of events (situations) that individuals have personally experienced and occurred at a certain time and place. This memory not only involves the storage of the event itself, but also includes the encoding of the environment and time of the event. The episodic memory unit plays a vital role, which can process and refine dialogues with users and generate high-level summaries of daily events, thereby helping users better understand and review their interactive experiences. In this way, not only the user experience is enhanced, but also a basis for personalized services is provided, which can more deeply understand user needs and preferences, and thus provide more accurate and considerate services.

[0063] In step S1, the memory fusion module is used for data flow between the working memory module and the long-term memory module, and is provided with a dynamic working memory construction unit, a semantic memory extraction unit, and an episodic memory extraction unit;

[0064] The dynamic working memory construction unit is used to retrieve context information from the long-term memory module based on the user query statement, and then use the BCE-Reranker model to rearrange the context information, combine the rearranged context information with the system instructions of the instruction management unit, and format it according to the input specifications preset by the large language model, so as to construct a complete working memory for processing the current user query statement; the semantic memory extraction unit is used to interact based on the current working memory, read the semantic knowledge carried by the context information generated during the interaction, and store the semantic knowledge in the long-term memory module; the episodic memory extraction unit is used to extract episodic memory including characters, event time, event location and event environment from the context information, generate a high-level summary based on the episodic memory and store it in the long-term memory module.

[0065] The memory fusion module is similar to the hippocampus in the human brain, which is responsible for quickly indexing and storing new experiences and knowledge. In the present invention, the memory fusion module is responsible for coordinating the data flow between working memory and long-term memory, ensuring that working memory can be dynamically updated according to current tasks and environmental changes, and integrating important information into long-term memory in a timely manner, thereby maintaining a flexible and efficient memory system. The dynamic working memory construction unit optimizes the relevance of information by rearranging the information of each context, and can more accurately understand and respond to user needs. The semantic memory extraction unit is used to read semantic knowledge in the fields of language, culture, science, and art. Once the semantic knowledge is found, it is extracted and stored in the long-term memory module in a timely manner, thereby enhancing the depth and breadth of the long-term memory module. This dynamic learning process not only improves the large language model's ability to understand complex contexts, but also strengthens its knowledge accumulation and application capabilities in different fields. Once the episodic memory is found, the episodic memory extraction unit generates a high-level summary of daily events, thereby helping users better understand and review their interactive experiences, which not only improves the user experience, but also can more accurately capture and reflect the needs and preferences of users, thereby providing more personalized and considerate services.

[0066] In step S1, the interaction module is used to mine surface meanings and deep needs from user query statements, generate an intention sequence related to user intentions based on the surface meanings and deep needs, i.e., a keyword sequence, and perform intelligent interaction based on the intention sequence.

[0067] The intelligent interaction module is a bridge for users to interact with the large language model. Its core task is to accurately identify the user's intentions and purposes. The intelligent interaction module can deeply understand the specific goals that users hope to achieve through dialogue, support multiple rounds of in-depth dialogues, and ensure that each communication can accurately reflect the user's expectations and achieve efficient and targeted communication. This deep understanding and responsiveness has greatly improved the user experience and greatly enhanced user satisfaction.

[0068] A preferred embodiment of a large language model intelligent interaction system with memory management capability of the present invention includes the following modules:

[0069] An initialization module, used to create a working memory module, a long-term memory module, a memory fusion module, and an intelligent interaction module with a large language model;

[0070] An external knowledge integration module, used to obtain external knowledge for intelligent interaction and integrate the external knowledge into the long-term memory module;

[0071] An intention recognition module is used for the intelligent interaction module to recognize the user query statement through a large language model to obtain an intention sequence, and send the intention sequence to the working memory module;

[0072] A context information retrieval module, used for the memory fusion module to retrieve relevant context information from the external knowledge in the long-term memory module based on the user query statement, and send the context information to the working memory module;

[0073] A working memory construction module, which is used for the working memory module to construct a dynamic working memory based on the intention sequence, context information and preset system instructions, store the working memory and send it to the intelligent interaction module;

[0074] An interaction module, used for the intelligent interaction module to output query results through the large language model based on working memory to perform intelligent interaction;

[0075] The long-term memory optimization module is used in the intelligent interaction process. The memory fusion module extracts semantic memory and episodic memory from user query statements and query results, and stores the semantic memory and episodic memory in the long-term memory module.

[0076] In the initialization module, the working memory module is used for short-term storage and processing of information, and is provided with an instruction management unit, a context association unit, and a user intention input unit;

[0077] The instruction management unit is used to define system instructions that guide the interaction process, namely static rules; the context association unit is used to dynamically store interaction fact information, user preferences and role information; and the user intention input unit is used to input intention sequences.

[0078] That is, the working memory module imitates the short-term memory mechanism of the human brain to provide instant data support for our thinking and decision-making, ensuring that the large language model can deeply understand user needs and their situational background, more accurately build communication with users, support dynamic adjustment of communication strategies based on working memory, and select appropriate language and expressions to achieve more natural and coherent dialogues; the static rules clarify how to process various retrieved knowledge information to ensure the consistency and accuracy of the interaction process; the context association unit updates and maintains factual information, user preferences and role information in real time, so that the large language model can have more natural, coherent and personalized dialogues with users; the user intention recognition unit is used to ensure that each communication can accurately capture and reflect the user's expectations, achieve efficient and targeted communication, and gradually guide the user to approach their ultimate goal. It is composed of a series of intention sequences (keyword sequences) closely linked to the goal. The intention sequence not only indicates the logical progression from the current dialogue to the achievement of the goal, but also ensures the directionality and coherence of the communication.

[0079] In the initialization module, the long-term memory module is used for long-term storage and processing of information, and is provided with an injection memory unit, a semantic memory unit and an episodic memory unit;

[0080] The injection memory unit is used to perform text vectorization processing on external knowledge through the bge-base-zh text embedding model to obtain vectorized knowledge, and after embedding the vectorized knowledge, store it in the Milvus database in the form of high-dimensional vectors; the semantic memory unit is used to embed organized knowledge and store it in the Milvus database in the form of high-dimensional vectors; the plot memory unit is used to encode and store the characters involved in the interaction, the time of the event, the place of the event and the environment in which the event occurs.

[0081] The long-term memory module is similar to the neocortex of the human brain, responsible for long-term storage and stabilization of this information. It is a key component of the large language model, which not only ensures the persistence of information, but also enables the large language model to quickly retrieve and use this information when needed; the injected memory unit is used to improve the context understanding ability when communicating with users, and directly integrates important external knowledge into the long-term memory module. This process draws on the core concept of Retrieval-Augmented Generation (RAG), and embeds the vectorized knowledge and stores it in the form of high-dimensional vectors in Mil vus database, not only the persistence of data is guaranteed, but also it is convenient for subsequent rapid retrieval and update; semantic memory is an important part of the human memory system, which is responsible for storing organized knowledge, such as words, concepts, the relationship between words and concepts, rules and formulas, etc. The semantic memory unit plays a similar role, imitating the way humans extract key information when recalling experiences, and through long-term interaction, abstracting the observations in multiple rounds of dialogue into semantic memory for storage, thereby continuously improving its own knowledge system; for example, by analyzing the patterns and trends in the dialogue to gain insights into personality traits, it can more accurately predict and respond to user needs; episodic memory refers to the memory of events (situations) that individuals have personally experienced and occurred at a certain time and place. This memory not only involves the storage of the event itself, but also includes the encoding of the environment and time of the event. The episodic memory unit plays a vital role, which can process and refine dialogues with users and generate high-level summaries of daily events, thereby helping users better understand and review their interactive experiences. In this way, not only the user experience is enhanced, but also a basis for personalized services is provided, which can more deeply understand user needs and preferences, and thus provide more accurate and considerate services.

[0082] In the initialization module, the memory fusion module is used for data flow between the working memory module and the long-term memory module, and is provided with a dynamic working memory construction unit, a semantic memory extraction unit, and an episodic memory extraction unit;

[0083] The dynamic working memory construction unit is used to retrieve context information from the long-term memory module based on the user query statement, and then use the BCE-Reranker model to rearrange the context information, combine the rearranged context information with the system instructions of the instruction management unit, and format it according to the input specifications preset by the large language model, so as to construct a complete working memory for processing the current user query statement; the semantic memory extraction unit is used to interact based on the current working memory, read the semantic knowledge carried by the context information generated during the interaction, and store the semantic knowledge in the long-term memory module; the episodic memory extraction unit is used to extract episodic memory including characters, event time, event location and event environment from the context information, generate a high-level summary based on the episodic memory and store it in the long-term memory module.

[0084] The memory fusion module is similar to the hippocampus in the human brain, which is responsible for quickly indexing and storing new experiences and knowledge. In the present invention, the memory fusion module is responsible for coordinating the data flow between working memory and long-term memory, ensuring that working memory can be dynamically updated according to current tasks and environmental changes, and integrating important information into long-term memory in a timely manner, thereby maintaining a flexible and efficient memory system. The dynamic working memory construction unit optimizes the relevance of information by rearranging the information of each context, and can more accurately understand and respond to user needs. The semantic memory extraction unit is used to read semantic knowledge in the fields of language, culture, science, and art. Once the semantic knowledge is found, it is extracted and stored in the long-term memory module in a timely manner, thereby enhancing the depth and breadth of the long-term memory module. This dynamic learning process not only improves the large language model's ability to understand complex contexts, but also strengthens its knowledge accumulation and application capabilities in different fields. Once the episodic memory is found, the episodic memory extraction unit generates a high-level summary of daily events, thereby helping users better understand and review their interactive experiences, which not only improves the user experience, but also can more accurately capture and reflect the needs and preferences of users, thereby providing more personalized and considerate services.

[0085] In the initialization module, the interaction module is used to mine surface meanings and deep needs from user query statements, generate an intention sequence related to user intentions based on the surface meanings and deep needs, i.e., a keyword sequence, and perform intelligent interaction based on the intention sequence.

[0086] The intelligent interaction module is a bridge for users to interact with the large language model. Its core task is to accurately identify the user's intentions and purposes. The intelligent interaction module can deeply understand the specific goals that users hope to achieve through dialogue, support multiple rounds of in-depth dialogues, and ensure that each communication can accurately reflect the user's expectations and achieve efficient and targeted communication. This deep understanding and responsiveness has greatly improved the user experience and greatly enhanced user satisfaction.

[0087] In summary, the advantages of the present invention are:

[0088] 1. By creating a working memory module, a long-term memory module, a memory fusion module and an intelligent interaction module with a large language model, external knowledge for intelligent interaction is obtained and integrated into the long-term memory module. The intelligent interaction module recognizes the user query sentence output by the large language model to obtain the intention sequence and sends it to the working memory module. The memory fusion module retrieves relevant context information from the external knowledge in the long-term memory module based on the user query sentence and sends it to the working memory module. Then the working memory module builds a dynamic working memory based on the intention sequence, context information and preset system instructions, stores the working memory and sends it to the intelligent interaction module. The intelligent interaction module is based on the working memory, inputs the large language model, and obtains the intention sequence. The query results are output for intelligent interaction. During the intelligent interaction process, the memory fusion module extracts semantic memory and episodic memory from the user query statements and query results and stores them in the long-term memory module; that is, it creates two memory mechanisms including working memory and long-term memory, and supports self-optimization. The working memory module is used for short-term storage and processing of information, and the long-term memory module is used for long-term storage and processing of information. The two memory mechanisms work together to achieve efficient retrieval and application of user query statements, promote deep integration of knowledge through the memory fusion module, and interact with the intention sequence related to the user's intention through the intelligent interaction module, which significantly improves the performance of the large language model in continuous dialogue and execution of complex tasks, and ultimately greatly improves the human-computer interaction performance.

[0089] 2. By deeply analyzing the specific goals that users hope to achieve through dialogue, generating intent sequences closely related to user intentions for interaction, supporting multiple rounds of in-depth dialogues, and gradually guiding users to their ultimate goals, we can achieve a deep understanding of user intentions and respond in a timely manner, greatly improving the user experience and greatly enhancing user satisfaction.

[0090] 3. By drawing on the declarative memory mechanism of the human brain, the semantic memory and episodic memory in the interaction process are innovatively extracted to achieve self-optimization of long-term memory; the memory enhancement strategy of the memory fusion module realizes dynamic storage and accumulation of experiential knowledge, forming an in-depth understanding of user preferences and needs, thereby providing more accurate and personalized services.

[0091] 4. Adopting the core concept of Retrieval Augmented Generation (RAG) and combining it with advanced Chinese text embedding technology (bge-base-zh text embedding model), key external knowledge is converted into vectorized knowledge and directly integrated into the long-term memory module, effectively improving the efficiency of knowledge storage and the speed of retrieval, and enhancing the processing capabilities in complex interactive scenarios, so as to more accurately understand and respond to user needs.

[0092] 5. Retrieve closely related contextual information from long-term memory based on user query statements, and then combine it with system instructions and intent sequences to effectively enhance the naturalness and accuracy of the interaction.

[0093] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A large language model intelligent interaction method with memory management capability, characterized by: The steps include: Step S1, creating a working memory module, a long-term memory module, a memory fusion module and an intelligent interaction module with a large language model; Step S2: Acquire external knowledge for intelligent interaction, and integrate the external knowledge into the long-term memory module; Step S3, the intelligent interaction module identifies the user query statement through the large language model to obtain the intention sequence, and sends the intention sequence to the working memory module; Step S4, the memory fusion module retrieves relevant context information from the external knowledge in the long-term memory module based on the user query statement, and sends the context information to the working memory module; Step S5, the working memory module constructs a dynamic working memory based on the intention sequence, context information and preset system instructions, stores the working memory and sends it to the intelligent interaction module; Step S6: The intelligent interaction module outputs the query result through the large language model based on the working memory to perform intelligent interaction; Step S7: During the intelligent interaction process, the memory fusion module extracts semantic memory and episodic memory from the user query statement and query results, and stores the semantic memory and episodic memory in the long-term memory module.

2. A large language model intelligent interaction method with memory management capability as claimed in claim 1, characterized in that: In step S1, the working memory module is used for short-term storage and processing of information, and is provided with an instruction management unit, a context association unit, and a user intention input unit; The instruction management unit is used to define system instructions that guide the interaction process, namely static rules; the context association unit is used to dynamically store interaction fact information, user preferences and role information; and the user intention input unit is used to input intention sequences.

3. The large language model intelligent interaction method with memory management capability as claimed in claim 1, characterized in that: In step S1, the long-term memory module is used for long-term storage and processing of information, and is provided with an injection memory unit, a semantic memory unit, and an episodic memory unit; The injection memory unit is used to perform text vectorization processing on external knowledge through the bge-base-zh text embedding model to obtain vectorized knowledge, and after embedding the vectorized knowledge, it is stored in the Milvus database in the form of a high-dimensional vector; the semantic memory unit is used to embed organized knowledge and store it in the Milvus database in the form of a high-dimensional vector; the plot memory unit is used to encode and store the characters involved in the interaction, the time of the event, the place of the event, and the environment in which the event occurs.

4. The large language model intelligent interaction method with memory management capability as claimed in claim 1, characterized in that: In step S1, the memory fusion module is used for data flow between the working memory module and the long-term memory module, and is provided with a dynamic working memory construction unit, a semantic memory extraction unit, and an episodic memory extraction unit; The dynamic working memory construction unit is used to retrieve context information from the long-term memory module according to the user query statement, and then use the BCE-Reranker model to rearrange the context information, combine the rearranged context information with the system instructions of the instruction management unit, and format it according to the input specification preset by the large language model, so as to construct a complete working memory for processing the current user query statement; The semantic memory extraction unit is used to interact according to the current working memory, read the semantic knowledge carried by the context information generated during the interaction, and store the semantic knowledge in the long-term memory module; The episodic memory extraction unit is used to extract episodic memories including characters, event time, event location, and event environment from context information, generate a high-level summary based on the episodic memory, and store it in a long-term memory module.

5. The large language model intelligent interaction method with memory management capability as claimed in claim 1, characterized in that: In step S1, the interaction module is used to mine surface meanings and deep needs from user query statements, generate an intention sequence related to user intentions based on the surface meanings and deep needs, i.e., a keyword sequence, and perform intelligent interaction based on the intention sequence.

6. A large language model intelligent interaction system with memory management capability, characterized by: Includes the following modules: An initialization module, used to create a working memory module, a long-term memory module, a memory fusion module, and an intelligent interaction module with a large language model; An external knowledge integration module, used to obtain external knowledge for intelligent interaction and integrate the external knowledge into the long-term memory module; An intention recognition module is used for the intelligent interaction module to recognize the user query statement through a large language model to obtain an intention sequence, and send the intention sequence to the working memory module; A context information retrieval module, used for the memory fusion module to retrieve relevant context information from the external knowledge in the long-term memory module based on the user query statement, and send the context information to the working memory module; A working memory construction module, which is used for the working memory module to construct a dynamic working memory based on the intention sequence, context information and preset system instructions, store the working memory and send it to the intelligent interaction module; An interaction module, used for the intelligent interaction module to output query results through the large language model based on working memory to perform intelligent interaction; The long-term memory optimization module is used in the intelligent interaction process. The memory fusion module extracts semantic memory and episodic memory from user query statements and query results, and stores the semantic memory and episodic memory in the long-term memory module.

7. A large language model intelligent interaction system with memory management capability as claimed in claim 6, characterized in that: In the initialization module, the working memory module is used for short-term storage and processing of information, and is provided with an instruction management unit, a context association unit, and a user intention input unit; The instruction management unit is used to define system instructions that guide the interaction process, namely static rules; the context association unit is used to dynamically store interaction fact information, user preferences and role information; and the user intention input unit is used to input intention sequences.

8. A large language model intelligent interaction system with memory management capability as claimed in claim 6, characterized in that: In the initialization module, the long-term memory module is used for long-term storage and processing of information, and is provided with an injection memory unit, a semantic memory unit and an episodic memory unit; The injection memory unit is used to perform text vectorization processing on external knowledge through the bge-base-zh text embedding model to obtain vectorized knowledge, and after embedding the vectorized knowledge, it is stored in the Milvus database in the form of a high-dimensional vector; the semantic memory unit is used to embed organized knowledge and store it in the Milvus database in the form of a high-dimensional vector; the plot memory unit is used to encode and store the characters involved in the interaction, the time of the event, the place of the event, and the environment in which the event occurs.

9. A large language model intelligent interaction system with memory management capability as claimed in claim 6, characterized in that: In the initialization module, the memory fusion module is used for data flow between the working memory module and the long-term memory module, and is provided with a dynamic working memory construction unit, a semantic memory extraction unit, and an episodic memory extraction unit; The dynamic working memory construction unit is used to retrieve context information from the long-term memory module according to the user query statement, and then use the BCE-Reranker model to rearrange the context information, combine the rearranged context information with the system instructions of the instruction management unit, and format it according to the input specification preset by the large language model, so as to construct a complete working memory for processing the current user query statement; The semantic memory extraction unit is used to interact according to the current working memory, read the semantic knowledge carried by the context information generated during the interaction, and store the semantic knowledge in the long-term memory module; The episodic memory extraction unit is used to extract episodic memories including characters, event time, event location, and event environment from context information, generate a high-level summary based on the episodic memory, and store it in a long-term memory module.

10. The large language model intelligent interaction system with memory management capability as claimed in claim 6, characterized in that: In the initialization module, the interaction module is used to mine surface meanings and deep needs from user query statements, generate an intention sequence related to user intentions based on the surface meanings and deep needs, i.e., a keyword sequence, and perform intelligent interaction based on the intention sequence.

Images