Memory retrieval method based on large language model and related device

Abstract

The invention relates to the field of data processing, in particular to a memory retrieval method based on a large language model and a related device. The method comprises the following steps: in response to an interaction instruction sent by a target user, performing context analysis processing on the interaction instruction to obtain a real-time interaction intention of the target user; performing memory retrieval on a short-term memory module and a long-term memory module based on the real-time interaction intention to obtain target retrieval information; and in combination with a personalized knowledge base of the target user, carrying out personalized processing on the target retrieval information to obtain response information matched with the use habit of the target user, and using the response information to respond to the interaction instruction. According to the method, the memory capacity of the large language model in the interaction process is improved, the information forgetting rate is reduced, the information retrieval efficiency is improved, effective classification and organization of personalized information are achieved, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a memory retrieval method based on a large language model and related devices. Background Art

[0002] With the rapid development of information technology, the amount of information that people need to process and remember every day is exploding, which brings many challenges to the application of large language models (LLMs).

[0003] In the related art, large language models mainly rely on traditional context storage methods, which have significant defects. First, this storage method results in limited memory capacity of the model, which cannot carry a large amount of personalized information accumulated by users for a long time, and it is difficult to meet their actual needs. Secondly, the model has a high forgetting rate, and it is difficult for the model to store specific user information persistently. The frequent "forgetfulness" problem causes the interaction to lack coherence, affecting the user experience. Furthermore, the information retrieval efficiency is low, and it is quite difficult to accurately and quickly find the personalized information required by users in massive information, which greatly affects the response speed and accuracy of the model. At the same time, there is a lack of effective information classification mechanism in the related art, which makes it difficult to intelligently classify and organize user personalized information, making information management chaotic and the retrieval process hindered. In addition, the information update mechanism is imperfect, and it is impossible to efficiently integrate and update the new and old information of users, which is easy to cause information redundancy or contradictions.

[0004] The above problems are particularly obvious in the interaction between large language models and users, which seriously hinders the improvement of user experience and the optimization of information processing efficiency. Therefore, it is urgent to propose a new technical solution to solve at least one of the above technical problems and promote the development and improvement of large models in practical applications. Summary of the invention

[0005] In view of the technical problems existing in the prior art, the present application provides a memory retrieval method based on a large language model and related devices to solve at least one technical problem existing in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a memory retrieval method based on a large language model, the method comprising:

[0007] In response to an interaction instruction issued by a target user, context parsing is performed on the interaction instruction to obtain a real-time interaction intention of the target user;

[0008] Based on the real-time interaction intention, the short-term memory module and the long-term memory module are memory-searched to obtain target retrieval information; wherein the short-term memory module and the long-term memory module support bidirectional transmission of stored information; the short-term memory module is used to temporarily store the real-time interaction information and real-time retrieval information between the target user within a preset period of time; the long-term memory module is used to store the key information in the real-time interaction information and the real-time retrieval information that conforms to the preset dynamic transfer strategy;

[0009] The target search information is personalized in combination with the personalized knowledge base of the target user to obtain response information that matches the target user's usage habits to respond to the interactive instruction.

[0010] In a second aspect, an embodiment of the present application provides a memory retrieval device based on a large language model, the device comprising at least the following units:

[0011] An interaction unit is configured to respond to an interaction instruction of a target user, perform context analysis on the interaction instruction, and obtain a real-time interaction intention of the target user;

[0012] A retrieval unit is configured to perform memory retrieval on a short-term memory module and a long-term memory module based on the real-time interaction intention to obtain target retrieval information; wherein the short-term memory module and the long-term memory module support bidirectional transmission of stored information; the short-term memory module is used to temporarily store the real-time interaction information and real-time retrieval information with the target user within a preset period of time; the long-term memory module is used to store key information in the real-time interaction information and the real-time retrieval information that complies with the preset dynamic transfer strategy;

[0013] The generating unit is configured to perform personalized processing on the target search information in combination with the personalized knowledge base of the target user, and obtain response information matching the usage habits of the target user to respond to the interactive instruction.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0015] at least one processor, memory, and input-output unit;

[0016] The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the memory retrieval method based on a large language model of the first aspect.

[0017] In a fourth aspect, a computer-readable storage medium is provided, which includes instructions, and when the instructions are executed on a computer, the computer executes the memory retrieval method based on a large language model of the first aspect.

[0018] The beneficial effect of the present application is: providing a memory retrieval method based on a large language model and a related device. In the technical scheme, first, in response to the interactive instruction issued by the target user, the interactive instruction is context-parsed and processed to obtain the real-time interactive intention of the target user. Then, based on the real-time interactive intention, the short-term memory module and the long-term memory module are memory-retrieved to obtain the target retrieval information. Among them, the short-term memory module and the long-term memory module support two-way transmission of stored information; the short-term memory module is used to temporarily store the real-time interactive information and real-time retrieval information between the target user within a preset period of time; the long-term memory module is used to store the key information in the real-time interactive information and the real-time retrieval information that conforms to the preset dynamic transfer strategy. Finally, combined with the personalized knowledge base of the target user, the target retrieval information is personalized to obtain the response information that matches the target user's usage habits to respond to the interactive instruction. In summary, the scheme can improve the information retrieval efficiency of the large language model, improve the performance of the large language model in the interactive scenario, and improve the user experience through multiple effects such as accurately understanding the user's intention, efficiently and comprehensively retrieving memory information, and realizing personalized responses. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of a memory retrieval method based on a large language model according to an embodiment of the present application;

[0020] Figure 2 It is a structural schematic diagram of a memory retrieval device based on a large language model according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0022] Figure 4 It is a structural schematic diagram of a medium device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0024] In order to solve at least one technical problem in the related art, the embodiment of the present application provides a memory retrieval method based on a large language model and a related device. In this technical solution, in response to the interactive instruction issued by the target user, the interactive instruction is context-parsed and processed to obtain the real-time interactive intention of the target user. Then, based on the real-time interactive intention, the short-term memory module and the long-term memory module are memory-retrieved to obtain the target retrieval information. Among them, the short-term memory module and the long-term memory module support two-way transmission of stored information; the short-term memory module is used to temporarily store the real-time interactive information and real-time retrieval information between the target user within a preset period of time; the long-term memory module is used to store the key information in the real-time interactive information and the real-time retrieval information that conforms to the preset dynamic transfer strategy. Finally, combined with the personalized knowledge base of the target user, the target retrieval information is personalized to obtain the response information that matches the target user's usage habits to respond to the interactive instruction. In summary, the scheme can improve the information retrieval efficiency of the large language model, improve the performance of the large language model in the interactive scenario, and improve the user experience by accurately understanding the user's intention, efficiently and comprehensively retrieving memory information, and realizing personalized responses.

[0025] Specifically, the embodiment of the present application proposes a memory retrieval scheme based on a large language model, which is composed of multiple specially trained large language models. For example, the memory retrieval system based on the large language model includes the following functions: multimodal information perception, information understanding and extraction, short-term and long-term memory, memory integration and updating, forgetting mechanism, information classification and indexing, context understanding, personalized adaptation, memory retrieval and reasoning, output generation and central coordination modules. These modules simulate the working principle of the human memory system through a sophisticated collaboration mechanism, and realize efficient processing, storage and retrieval of multimodal information. The workflow of the system begins with the reception and preliminary processing of multimodal information, which is stored in short-term memory after deep understanding and extraction. At the same time, the embodiment of the present application analyzes the current context, activates relevant long-term memory, and performs information retrieval and reasoning according to demand. The output generation module generates a personalized response based on the retrieved memory and the current context. The embodiment of the present application also continuously optimizes the memory management strategy through modules such as memory integration and updating, forgetting mechanism and personalized adaptation, and improves the long-term operation efficiency and personalization of the system.

[0026] It is understandable that the improvements of the technical solution of this application over the related art are analyzed from multiple perspectives below:

[0027] The short-term memory module focuses on real-time interaction and information retrieval with users within a preset period of time, and can quickly locate recent related content; the long-term memory module stores key information that conforms to the dynamic transfer strategy, and can tap into important knowledge accumulated in the past. The two support two-way transmission of stored information, which means that they cooperate and complement each other, and can not only quickly call up fresh content in short-term memory based on the current situation, but also trace back deep knowledge in long-term memory, making memory retrieval more comprehensive and efficient, covering as much information as possible that is valuable for responding to interactive instructions, and increasing the probability of retrieving accurate target retrieval information.

[0028] The target search information is personalized in combination with the personalized knowledge base of the target user, and finally the response information that matches the user's usage habits is obtained. This makes the response no longer a stereotyped universal answer, but fully considers the user's unique preferences, past operating habits and other personalized factors. For example, for users who are accustomed to concise expressions, the response will be as concise and clear as possible; for users who prefer detailed and in-depth explanations, richer and more detailed content can be given, which greatly enhances the user experience and improves the user's satisfaction with the interaction with the system and their willingness to continue using it.

[0029] The short-term memory module can temporarily store real-time interaction information and real-time retrieval information within a preset time period, which helps to quickly grasp the latest situation within a certain time range, and can update or clean up these temporary information according to certain time rules to avoid excessive redundant data accumulation affecting retrieval efficiency, so that the system can focus on the most relevant interaction content at the moment to retrieve information.

[0030] The long-term memory module filters and stores key information according to the preset dynamic transfer strategy, ensuring that the data retained in the long-term memory are of important reference value for subsequent interactions. It effectively refines and integrates the information, so that the core content can be located more quickly during long-term memory retrieval, thereby improving the resource utilization efficiency and operational performance of the entire memory retrieval system.

[0031] In general, the embodiments of the present application significantly improve the memory capacity of the large language model during the interaction process, reduce the information forgetting rate, improve the information retrieval efficiency, and realize the effective classification and organization of personalized information. At the same time, through the improved context understanding ability and personalized adaptation mechanism, the embodiments of the present application can maintain a high degree of coherence and personalized characteristics in long-term interactions, thereby greatly improving the user experience and interaction quality.

[0032] The technical solution of the present application, the memory retrieval scheme based on the large language model provided in the embodiment of the present application, can also be executed by an electronic device, which can be a server, a server cluster, or a cloud server. The electronic device can also be a terminal device such as a mobile phone, a computer, a tablet computer, a wearable device, or a special device (such as a special terminal device with a memory retrieval method system based on a large language model, etc.). These electronic devices can also be equipped with the chips or other hardware processing units introduced in the above embodiments. Alternatively, these electronic devices can also be installed with a service program for executing a memory retrieval scheme based on a large language model.

[0033] Figure 1 A flowchart of a memory retrieval method based on a large language model provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0034] 101, in response to an interaction instruction issued by a target user, performing context analysis on the interaction instruction to obtain a real-time interaction intention of the target user;

[0035] 102, performing memory retrieval on a short-term memory module and a long-term memory module based on the real-time interaction intention to obtain target retrieval information;

[0036] 103 , combining with the personalized knowledge base of the target user, the target search information is personalized to obtain response information matching the target user's usage habits to respond to the interactive instruction.

[0037] In an embodiment of the present application, the short-term memory module and the long-term memory module support bidirectional transmission of stored information. This bidirectional transmission mechanism enhances the synergy between the two memory modules. For example, when the real-time interactive information temporarily stored in the short-term memory module is precipitated for a period of time and meets the key information standard set by the dynamic transfer strategy of the long-term memory module, it can be transferred to the long-term memory for long-term preservation, so as to facilitate the subsequent more in-depth and comprehensive excavation of the user's past related situations. Conversely, when processing the current interactive instruction for memory retrieval, if there is historical key information related to the current real-time interactive intention in the long-term memory, it can be transmitted to the short-term memory module in time, and participate in the retrieval process together with the temporary information in the short-term memory, so that the retrieved target retrieval information is more accurate and comprehensive, and fully integrates the recent and past valuable data.

[0038] In an embodiment of the present application, the short-term memory module is used to temporarily store real-time interaction information and real-time retrieval information with the target user within a preset time period. The short-term memory module is mainly responsible for temporarily storing real-time interaction information and real-time retrieval information with the target user within a preset time period. The preset time period here is a pre-set time range, such as the past 1 hour, 1 day, etc. (the specific duration is set according to the actual application scenario and needs). Real-time interaction information covers various texts, data, etc. generated by the interaction process such as the command content entered by the user and the corresponding feedback made by the system; real-time retrieval information is all kinds of relevant information involved in the retrieval operation for the user interaction command, such as some candidate results retrieved in the intermediate process.

[0039] The temporary storage function of the short-term memory module helps to focus on the current interaction situation. During this time period, the system can quickly obtain the freshest and most directly related information to the current interaction, avoiding the need to sift through massive historical data from scratch when searching, improving the timeliness of information retrieval, and allowing the system to quickly locate recent key situations when responding to recent related interaction instructions, ensuring the timeliness of the response and its fit with the current scene. At the same time, by setting a time limit for temporary storage of information, unlimited data accumulation is avoided, which helps to maintain the rational use of system storage resources and stable retrieval efficiency.

[0040] In an embodiment of the present application, the long-term memory module is used to store the key information in the real-time interaction information and the real-time retrieval information that complies with the preset dynamic transfer strategy. The long-term memory module is used to store the key information in the real-time interaction information and the real-time retrieval information that complies with the preset dynamic transfer strategy. The dynamic transfer strategy is a set of pre-established rule systems for determining which information is worth transferring from the short-term memory module and storing for a long time. For example, these rules may involve considerations of multiple dimensions such as the importance of information (such as involving the core needs of users, frequently occurring key knowledge points, etc.), the strength of relevance (content closely related to the user's long-term focus areas), and the long-term value in the time dimension (information that may be of reference significance for subsequent interactions over a longer period of time).

[0041] By filtering and storing key information according to the dynamic transfer strategy, the long-term memory module is equivalent to building a refined knowledge base that is deeply valuable to user interactions. When faced with complex and diverse user interaction instructions, especially those that require tracing past history and digging out deep-level related content, the long-term memory module can provide reliable and targeted key data support, ensuring that the system can more accurately understand the intention and retrieve appropriate target retrieval information based on the user's long-term interaction picture, thereby improving the quality and effectiveness of the entire system response, while also making the storage and management of user-related information more scientific and orderly, focusing on the truly valuable parts.

[0042] As an optional embodiment, in 101, in response to the interaction instruction issued by the target user, context analysis is performed on the interaction instruction to obtain the real-time interaction intention of the target user.

[0043] As an optional embodiment, in 102, before performing memory retrieval on the short-term memory module and the long-term memory module based on the real-time interaction intention and obtaining the target retrieval information, the following steps may also be implemented:

[0044] 201. Collect multimodal information of target users;

[0045] 202. Extracting information to be stored from the multimodal information;

[0046] 203. Storing the information to be stored in a short-term memory module according to the importance score and the timeliness score;

[0047] 204. Transfer the key information stored in the short-term memory module to the long-term memory module according to the dynamic transfer strategy.

[0048] In the embodiment of the present application, the multimodal information at least includes: image information, audio information, and text information. Multimodal information refers to a collection of information containing multiple different types of data modes, and in the embodiment of the present application, at least image information, audio information, and text information are covered. Image information can be pictures uploaded by users, visual image content displayed in the system interaction interface, etc., which convey corresponding meanings through elements such as color, shape, and object layout; audio information, such as user voice input, background sound in the environment, etc., contains voice intonation, timbre, and various sound event characteristics; text information is text instructions input by users, text content fed back by the system, etc., which convey information through the meaning of text. These different modal information reflect the various situations of the target user in the process of interacting with the system from multiple angles, and together constitute a rich and three-dimensional source of information, which provides a basis for the subsequent comprehensive understanding of users and the realization of personalized services.

[0049] From the perspective of visual information processing, the Vision Transformer (ViT) model is used as the backbone network, and pre-trained models such as CLIP are used to process image information. They can accurately capture the key features of the image and convert them into vector representations that can interact with the language model, allowing the image information to be better integrated into the subsequent process. In addition, the integrated object detection and scene segmentation algorithms can obtain more fine-grained visual information, such as distinguishing different objects and scene areas in the image, helping to grasp the user's intention from a visual perspective. From the perspective of audio processing, self-supervised learning models such as Wav2Vec are used for speech recognition to convert speech into an analyzable form. At the same time, voiceprint recognition technology is combined to lock the speaker's identity features, which is conducive to distinguishing different users. For environmental sounds, the Sound Event Detection algorithm is used to classify and extract features, and audio information is analyzed in all directions to supplement key audio evidence for understanding user interaction intentions. From the perspective of text processing, pre-trained language models such as multilingual BERT or RoBERTa are used, combined with named entity recognition (NER) and relation extraction (RE) technologies to dig deep into text information, clarify key entities and their relationships, and achieve deep understanding. Then, the sentiment analysis algorithm is used to capture the emotional characteristics of the text, because emotions can reflect the user's potential intentions, further improve the understanding of the meaning conveyed by the text, and help clarify the interaction intentions.

[0050] In the embodiment of the present application, with the help of cross-modal attention mechanism and contrastive learning method, different modal information such as vision, audio, and text are mapped to the same semantic space, eliminating modal barriers, facilitating association and integration, laying a solid foundation for subsequent memory storage and retrieval, and also helping to analyze the embodiment and association of user intentions in various modal information. In addition, the fault-tolerant mechanism of modal missing ensures that even if some modal information is missing, the system can stably process information to avoid affecting the analysis of user intentions.

[0051] In order to further improve the efficiency of multimodal processing, a pipeline parallel processing architecture is adopted in the embodiment of the present application, and each modal information can be processed synchronously, making full use of resources and speeding up the processing speed. Finally, the feature fusion network is relied on to integrate all information into a unified representation, organically combining multimodal information. This not only significantly enhances the real-time performance of the system, enabling it to respond quickly to interactive instructions, but also ensures the integrity and accuracy of information processing, avoids information problems, and ensures accurate analysis of the user's real-time interactive intentions.

[0052] In actual applications, relying on such a multimodal collaborative processing mechanism, the embodiments of the present application can synchronously perceive and understand multiple information such as images, videos, audio, text, etc., provide high-quality input data for personalized memory processing, thereby more comprehensively understanding and remembering user input, and then more accurately providing personalized services in line with the user's real-time interaction intentions.

[0053] In an embodiment of the present application, the information to be stored includes at least: key content information associated with the target user in the multimodal information. The information to be stored is key content information that is further extracted from the collected multimodal information, has a certain value and is associated with the target user. It focuses on those parts that are important for subsequent memory storage and serving users, such as object features that may be closely related to the user's current needs in image information, key voice content that can reflect the user's identity or current intentions in audio information, and parts of text information involving user core demands and key entity relationships. After screening and extraction, these contents will be used as the main objects stored in the memory module later. They are the key link in the entire information processing process, which determines which information can be retained for a long time for further analysis and response to user interaction instructions.

[0054] Further optionally, the information to be stored is extracted based on the information extraction model using a multi-task learning mode. The multi-task learning mode enables the information extraction model to process multiple related tasks at the same time (for example, performing named entity recognition, relationship extraction and other tasks in text information at the same time), making full use of the correlation between different tasks to improve model performance, and then more accurately identifying and extracting key content from complex multimodal information, ensuring the quality and effectiveness of the information to be stored.

[0055] In step 201, multimodal information of the target user is collected through various channels such as corresponding input interfaces and sensors. For image information, visual image content may be obtained by using image acquisition devices such as cameras; for audio information, sound data is recorded by using audio acquisition devices such as microphones; and the collection of text information mainly comes from the text content entered by the user in the interactive interface through keyboard input, voice-to-text conversion, etc. At the same time, it is also necessary to ensure the synchronization and relevance of the collection of these different modal information, such as knowing which image a certain audio segment corresponds to, which text segment is related to it, etc., so as to carry out unified processing and analysis in the future, and comprehensively capture the complete information of the user at the same time or in a specific interactive scenario. Only by comprehensively and accurately collecting multimodal information can valuable content be mined to serve users.

[0056] In step 202, first, the collected multimodal information of different modes is input into the information extraction model. The model works based on a multi-task learning mode. For example, when processing the text mode, it simultaneously performs tasks such as named entity recognition (recognizing key entities such as names of people, places, and institutions in the text), relationship extraction tasks (analyzing the association between different entities), and sentiment analysis tasks (judging the emotional tendency contained in the text); for the image mode, the features of key objects in the image are extracted through computer vision related technologies, and specific scenes are identified; the audio mode extracts the key acoustic features of the speech, the speaker's voiceprint features, and the categories of environmental sounds. Then, the results of the analysis of each mode are combined to screen out the key content information that is closely related to the target user and has important value for subsequent services, and output as the information to be stored. In this way, the massive and complex multimodal information is refined to avoid storing too much redundant data in the memory module, improve the efficiency of subsequent storage and retrieval and the effectiveness of information utilization, and ensure that the stored information is the key content that serves to accurately understand the user and respond to interactive instructions.

[0057] In step 203, for the extracted information to be stored, the system will calculate its importance score and timeliness score respectively through a preset evaluation mechanism. The determining factors of the importance score may include the degree of relevance of the information to the user's current interaction intention, the value of understanding the user's long-term behavior pattern, etc. For example, the importance score of information directly related to the core needs clearly put forward by the user will be higher; the timeliness score considers the time sensitivity of the information, for example, the information that is frequently appeared recently and is closely related to the current interaction scene has a higher timeliness score, while some outdated or long-ago and not strongly related information has a lower timeliness score. Then, based on these two scores, the information to be stored is stored in the short-term memory module according to certain rules, for example, information with high importance scores and timeliness scores is stored first, and information with low scores can be selectively stored or temporarily not stored according to the capacity of the short-term memory module, etc., to ensure that the information stored in the short-term memory module is currently more critical and time-sensitive information, so as to quickly respond to subsequent interactive retrieval needs. In this way, the information storage of the short-term memory module is managed reasonably, allowing the short-term memory module to focus on the information that is most helpful for understanding user interactions and retrieving related content in the near future, so that valuable content can be quickly located when memory retrieval is based on real-time interaction intentions. At the same time, the storage situation is dynamically adjusted through the scoring mechanism to maintain the effectiveness and practicality of the short-term memory module information.

[0058] In step 204, the dynamic transfer strategy is a set of rules that comprehensively considers various factors, and is used to determine which information in the short-term memory module should be transferred to the long-term memory module to become key information for long-term storage. These factors may include the frequency of occurrence of information (such as information that frequently appears and is always related to the key needs of users), the representativeness of the overall user behavior pattern (information that can reflect the user's long-term preferences and habits), and the versatility in different interaction scenarios (information applicable to a variety of common interaction scenarios). According to this dynamic transfer strategy, the system regularly or when specific trigger conditions are met (such as the accumulation of a certain type of information in the short-term memory module reaches a certain threshold, etc.), screens and evaluates the information stored in the short-term memory module, extracts the key information that meets the conditions, and transfers it to the long-term memory module, so as to achieve a reasonable transition and effective retention of information from the short-term to the long-term, and provide a deep and valuable data foundation for long-term memory retrieval. In this way, through dynamic transfer, the long-term memory module can continuously accumulate key information that has long-term value to users, and build a more complete user interaction knowledge base, which is convenient for providing reliable and rich historical data support when it is necessary to deeply explore the user's past situation, fully understand the user's long-term behavior characteristics, and respond to complex interaction instructions in the future, thereby improving the quality and accuracy of the system's response to users based on memory retrieval.

[0059] In summary, 201 to 204 above, the collection of multimodal information and a series of subsequent steps such as extraction and storage of stored information work closely together to serve the memory retrieval based on real-time interactive intentions and the realization of personalized services for users by the entire system.

[0060] For example, the following are some detailed introductions to the above multimodal information processing scheme:

[0061] Text information processing: Use the improved BERT model for semantic understanding, and use the multi-head self-attention mechanism to capture the contextual relationship in the text, so as to accurately grasp the meaning of the text and provide basic text semantic analysis for subsequent processing.

[0062] Non-text information processing: For image information, use specific modality encoders (such as ResNet, ViT, etc.) to extract features and convert them into a form that can be integrated with other modal information so that they can interact and process in a unified semantic space, thereby achieving effective feature extraction and preliminary understanding of different types of non-text information.

[0063] Multi-task learning framework: Build a multi-task learning framework that integrates named entity recognition (NER), relation extraction (RE), event extraction (EE) and other functions. In this framework, conditional random fields (CRF) and bidirectional LSTM networks are used to work together to identify key entities, graph neural networks (GNN) are used to model the relationship between entities, and a hierarchical attention mechanism is used to capture event structures. Through these advanced technology combinations, various key information elements are comprehensively and accurately extracted from user interaction information. Moreover, these extractors have undergone adaptive training in specific fields, which can better meet the information extraction needs in different fields and improve the accuracy and effectiveness of extraction.

[0064] For example, pre-built knowledge graphs and external knowledge bases are used to inject knowledge into language models, so that the models can better understand professional terms and common sense knowledge, make up for the lack of knowledge reserves of simple language models, and improve the ability to understand complex semantics. At the same time, a contrastive learning mechanism is introduced to construct positive and negative sample pairs for comparative analysis, so that the model can keenly capture subtle semantic differences, further optimize the accuracy and depth of semantic understanding, and thus more accurately grasp the deep meaning and potential intentions of user interaction information.

[0065] The embodiment of the present application can further implement an adaptive quality control mechanism, which assigns reliability scores to the extraction results through a confidence assessment module, and combines a multi-round verification mechanism to strictly control the accuracy of the extracted information. For information with low confidence and uncertainty, an interactive clarification mechanism is activated to obtain supplementary information by asking users precise questions, ensuring that the information that ultimately enters the subsequent processing flow has high quality and reliability, reducing the interference of erroneous information and ambiguous information on the system.

[0066] Here, the extracted information will be organized into a structured knowledge representation form, using a hierarchical semantic framework to encode entity, relationship and event information into vector form, and attach meta-information such as timestamp and source. This makes the information not only have a clear semantic structure, but also contains rich context and source information, providing a solid, orderly and easy-to-operate foundation for subsequent memory storage and retrieval, and facilitating the system to efficiently manage and utilize this information, better serving personalized memory processing and subsequent user interaction responses.

[0067] Through the close coordination and organic combination of the above-mentioned technologies, it is possible to accurately understand and extract key information from multimodal user interaction information, and continuously optimize and improve this process. As time goes by, continuous model optimization and knowledge accumulation will further enhance the system's understanding ability, provide users with higher quality and more personalized service experience, meet users' diverse needs in different scenarios, play a key role in the interaction process between users and systems, and improve the overall interaction effect and user satisfaction.

[0068] In the embodiment of the present application, the dynamic transfer strategy is dynamically adjusted using a reinforcement learning algorithm. Reinforcement learning is a machine learning method, the core of which is that the agent (in this context, it can be understood as the relevant module or mechanism for executing the dynamic transfer strategy) interacts with the environment and learns what actions to take according to the reward signal fed back by the environment to maximize the long-term accumulated rewards. In the embodiment of the present application, the dynamic transfer strategy uses a reinforcement learning algorithm for dynamic adjustment, which means that the system will regard the process of transferring information from the short-term memory module to the long-term memory module as the behavior decision-making process of the agent. The environment is a comprehensive situation such as the scene of the entire user interaction and the existing information storage status of the system. For example, when a piece of information in a short-term memory is transferred to the long-term memory module, if the subsequent response to the user interaction instruction based on this information achieves a better effect (such as improving the accuracy of the response, user satisfaction, etc.), then the system will give a corresponding "reward" feedback, otherwise if the effect is not good, a lower "reward" or even "penalty" will be given.

[0069] In this way, the dynamic transfer strategy can continuously optimize and adjust itself according to actual interaction feedback. It does not decide which information is transferred from short-term memory to long-term memory based on fixed rules. Instead, it can adaptively discover which types of information are more valuable for the accumulation of long-term memory as the system and user interaction practices increase, and better meet the diverse and dynamically changing interaction needs of users. For example, at first, you may think that a certain type of information is not very important and does not need to be stored for a long time, but as the reinforcement learning process finds that it frequently plays a key role in different interaction scenarios, the strategy will be adjusted to include it in the long-term memory module, so that the key information stored in the long-term memory module always remains in the most valuable state for user interaction, improving the flexibility and effectiveness of the entire memory retrieval system.

[0070] In an embodiment of the present application, the storage mechanism in the short-term memory module is a temporary storage that depends on the scoring priority sorting method. It is understandable that the storage mechanism adopted by the short-term memory module temporarily stores information based on the scoring priority. As mentioned above, the importance score and timeliness score will be calculated for the stored information. Here, the priority of each piece of information is determined based on these two scores or other relevant scoring indicators. For example, information with high importance and strong timeliness will be prioritized, and then the information will be stored in the short-term memory module in order from high to low priority. And this storage is temporary and will be limited by a preset time period. Within a certain time range, this information is retained to facilitate the system to quickly retrieve recent content closely related to the current interaction. Once this time period is exceeded or due to the continuous influx of new information, the capacity and other limitations may be gradually replaced or deleted.

[0071] The temporary storage mechanism based on scoring priority allows the short-term memory module to focus on the most valuable information at the moment, ensuring the timeliness of the information and its relevance to the current user interaction. It can efficiently manage the storage space of the short-term memory module, avoid unlimited data accumulation, and enable the system to quickly locate the most critical content in the near future when performing memory retrieval in response to real-time interaction intentions, thereby improving retrieval efficiency and timeliness of response, and better serving user needs in the current interaction scenario.

[0072] In the embodiment of the present application, the storage mechanism in the long-term memory module is a classified compressed storage based on the knowledge graph structure. The long-term memory module stores information in a manner based on the knowledge graph structure. The knowledge graph is a structured representation form that constructs various types of knowledge into a network through nodes and edges. In the present application, the system will classify the key information transferred from the short-term memory module according to its inherent semantic relationship, category attributes, etc., for example, information related to user interests and hobbies is classified into one category, and information related to the user's work field is classified into another category. Then the nodes in the knowledge graph are used to represent these different categories of key information, and the edges are used to describe the associations between them, such as causal relationships, belonging relationships, etc. At the same time, in order to store and manage this information more efficiently, a certain degree of compression processing will be performed to remove some redundant data representations or use a more compact data structure to store the information in the knowledge graph, saving storage space while ensuring the integrity of the information.

[0073] Storage based on the knowledge graph structure helps to sort out the logical relationship between complex user interaction information, and can clearly show how different information is related to each other, making it easier for the system to grasp the user's overall behavior patterns, long-term interest preferences, etc. from a macro level. In addition, classified storage can make memory retrieval more targeted. When searching based on real-time interaction intentions, relevant information can be quickly located in the corresponding category. Compressed storage solves the storage resource pressure problem that may be caused by the growing amount of long-term memory data, ensuring that the long-term memory module can continuously and stably accumulate and save key information that has long-term value to users, and provide strong knowledge support for the subsequent comprehensive and in-depth understanding of users and response to interaction instructions.

[0074] For example, the long-term memory (LTM) subsystem uses a knowledge graph structure based on a graph neural network (GNN) and cooperates with a vector database to achieve efficient information storage and retrieval. The system uses the BERT model to encode input information, compresses information dimensions through knowledge distillation technology, and then establishes the relationship between information through the graph attention network (GAT). In order to process large-scale information, the system adopts a distributed storage architecture and combines the LSH (local sensitive hashing) algorithm to achieve fast approximate nearest neighbor search.

[0075] In general, the reinforcement learning adjustment mechanism of the dynamic transfer strategy and the respective storage mechanisms of the short-term and long-term memory modules ensure the rational flow, efficient storage and effective utilization of information in the system from different angles, and jointly help to achieve accurate memory retrieval and high-quality user interaction services.

[0076] As an optional embodiment, in 203, storing the information to be stored in the short-term memory module according to the importance score and the timeliness score can be implemented as follows:

[0077] A short-term memory model is used to determine the information importance and timeliness of the information to be stored, so as to obtain a real-time importance score of the information to be stored in the current interactive session; the information to be stored is prioritized based on the real-time importance score to obtain a real-time sorting result; based on the real-time sorting result, a dynamic storage space corresponding to the information to be stored in the short-term memory module is adaptively configured; and the information to be stored is stored in the corresponding dynamic storage space in the short-term memory module as short-term storage information.

[0078] Specifically, the importance assessment can be that the short-term memory model analyzes the degree of relevance between the information to be stored and the current user interaction intention. For example, if the information to be stored directly involves the question, demand or task currently clearly raised by the user, then its importance score will be increased accordingly. At the same time, the potential value of the information to the user's long-term behavior pattern will also be considered. For example, some recurring information related to the user's core business or interests will be given a higher importance score even if it is not at the core of the current interaction due to its long-term value.

[0079] Timeliness can be evaluated based on the time when the information was generated and its timeliness in the current interaction context. Information that was generated recently and is closely related to the current interaction has a higher timeliness score, while information that is older and less relevant to the current interaction has a lower timeliness score. For example, in a real-time conversation scenario, a key noun or event-related information that the user just mentioned has a strong timeliness; on the contrary, an insignificant interaction detail from a few days ago has a weaker timeliness. By combining the evaluation of these two dimensions, a real-time importance score in the current interactive session is calculated for each piece of information to be stored. This score can quantitatively reflect the value of the information at the moment.

[0080] Then, according to the calculated real-time importance score, all the information to be stored is arranged in order from high to low scores to form a priority sequence. Information with high scores is placed in front, which means that they have a higher priority in the current interaction and need to be processed and stored first so that they can be quickly obtained in subsequent retrieval and use; while information with low scores is placed at the back, which is relatively less urgent and important in the current interaction. Such sorting results provide guidance for subsequent storage space allocation and information storage, ensuring that the short-term memory module can focus on the most important information first.

[0081] Next, the storage space of the short-term memory module is not fixed, but is dynamically adjusted according to the priority of the information. For the information to be stored that is ranked in front and has a high priority, a relatively large and more accessible storage space will be allocated to ensure that these key information can be quickly stored and retrieved. For example, it may be stored near the front of the storage area or stored using a more efficient storage structure. As the priority decreases, the storage space allocated to the information to be stored will be reduced accordingly, and the storage location may also be relatively backward or a more compact storage method may be used. This adaptive storage space configuration mechanism can make full use of limited short-term memory resources to ensure that at any time, the short-term memory module stores the information most needed for the current interaction, while also avoiding low-priority information from occupying too much valuable space, improving space utilization and the rationality of information storage.

[0082] Finally, according to the dynamic storage space allocation scheme determined earlier, the information to be stored is stored one by one in the corresponding position of the short-term memory module to complete the short-term storage process of the information. At this point, this information officially becomes short-term storage information, which can be quickly retrieved and used by the system in the current interactive session and subsequent possible related interactions, providing timely and targeted data support for memory retrieval based on real-time interactive intentions. In addition, as new information to be stored continues to arrive and the interaction process progresses, the short-term memory module will continue to update, replace and reorder the information according to the above steps, always maintaining the timeliness and importance of its stored information to adapt to the ever-changing user interaction needs.

[0083] Through this series of steps, the short-term memory module can efficiently manage the information to be stored, maximize its support for the current interaction under limited space and time conditions, and provide strong guarantees for the efficient operation of the entire system and accurate response to user interaction instructions.

[0084] Exemplarily, in order to improve processing efficiency, the above steps can use a hierarchical attention network (HAN) to preliminarily screen and score the importance of information, and use an LSTM network to capture temporal dependencies. The capacity of the short-term memory module is dynamically adjusted through an adaptive mechanism in the short-term memory model, automatically optimizing the storage space according to the importance and timeliness of the information.

[0085] Further optionally, the corresponding dynamic storage space in the short-term memory module belongs to a dynamic buffer; the dynamic storage space in the short-term memory module belongs to a dynamic buffer, which means that the size of the buffer, the content of the stored information, etc. are not fixed, but will be dynamically adjusted according to the operating status of the system and the relevant characteristics of the information. It is like a flexible information temporary storage area, which is specially used to store the information to be stored that is extracted from multimodal information and needs to be retained in the short term after importance and timeliness evaluation, so as to facilitate the subsequent rapid memory retrieval based on real-time interactive intentions.

[0086] The short-term memory module is provided with an interactive window mechanism for maintaining and managing the dynamic buffer. This mechanism is used to determine which information in the dynamic buffer needs to be focused on, retained, and when to update or replace the information. For example, it can be set to focus only on the interactive information in the recent period of time. As new information continues to enter the dynamic buffer, the window will slide accordingly, and the earlier, relatively less important or outdated information will be removed from the window range (that is, removed from the dynamic buffer), while incorporating new recent interactive information into the window range (that is, stored in the dynamic buffer), so as to achieve effective management of the dynamic buffer information, and always keep the information in the buffer in the state with the strongest correlation with the current interaction and the best timeliness.

[0087] Exemplarily, the short-term memory (STM) subsystem adopts an improved Transformer architecture and uses a multi-head attention mechanism to process instant information. The subsystem maintains a dynamic buffer and manages the most recent interactive information through a sliding window mechanism. Specifically, the STM subsystem adopts an improved Transformer architecture, which has the advantages of strong parallel computing capabilities and the ability to effectively capture long-distance dependencies. On this basis, it is improved and applied to the short-term memory subsystem to make it more adaptable to the needs of the short-term memory module for rapid information processing and dynamic management. For example, it can process instant information more efficiently, quickly encode and integrate the information newly entering the dynamic buffer, so that this information can be quickly converted into a form that can be used for subsequent retrieval and analysis, and provide powerful computing architecture support for the system to respond to user interaction instructions in a timely manner. In this improved Transformer architecture, a multi-head attention mechanism is used to process instant information. The multi-head attention mechanism can simultaneously focus on different parts of the information and their relationships from multiple different "perspectives", which is very suitable for situations such as instant information, which is often complicated and requires rapid extraction of key content. It can dig out the degree of correlation between different elements in instant information, distinguish which are important and closely related to the current interaction, and which are relatively minor parts, so as to better assist the system to understand the key intentions of the current user interaction, and reasonably store the processed information in the dynamic buffer, so as to facilitate the subsequent flexible retrieval and utilization under the management of the interactive window mechanism, and further improve the processing efficiency and management level of the short-term memory subsystem for instant information. Specifically, the sliding window mechanism is used to manage the recent interactive information, which is similar to the interactive window mechanism mentioned above. It determines which recent interactive information is within the window range (that is, to be retained in the dynamic buffer) in a way similar to the sliding window moving on the timeline. For example, the window size can be set to the past 10 minutes or 20 interactive information (determined according to factors such as actual application scenarios and system performance). As time goes by and new interactive information is continuously generated, the window will slide accordingly, and the information in the dynamic buffer will be continuously updated to ensure that the key information stored therein is always recent and most likely to be related to the current interaction, so that the short-term memory subsystem can quickly locate the current useful data when performing operations such as memory retrieval, and enhance the timeliness and accuracy of the system's response to user interaction.

[0088] In summary, the dynamic buffer in the short-term memory module and the corresponding interactive window (sliding window) mechanism, together with the improved Transformer architecture and multi-head attention mechanism, jointly build an efficient, flexible and dynamically adaptable short-term memory management system that can dynamically adapt to changes in user interactions. It plays a vital role in the entire system to accurately understand user intentions, quickly retrieve memory information and provide appropriate responses.

[0089] Specifically, the interaction window mechanism first needs to be initialized. This includes determining the size and initial position of the window. The window size can be set based on factors such as system performance, frequency and characteristics of user interactions. For example, in a high-frequency interaction scenario, such as a real-time customer service chat system, the window size may be set to cover the most recent 10-20 interaction messages to ensure that enough recent context can be captured. The initial position usually starts when the system starts processing user interaction information, at which time the window covers the initial interaction information.

[0090] When new interactive information is generated, the system will determine whether this information should enter the interactive window. The judgment is mainly based on the timeliness and importance of the information. In terms of timeliness, newly generated information that meets the "recent" standard in time will usually be considered for inclusion in the window. Importance is measured by the information importance score mentioned earlier. For example, if the new information is related to the core needs of the current user (such as product details that the user is asking about), its importance score is higher and it will be included in the window first. This information will be stored in the dynamic buffer of the short-term memory module. The dynamic buffer serves as the actual storage area of ​​the interactive window mechanism and provides storage space for the information included in the window. For example, after the new text information is processed, it is stored in the corresponding position of the dynamic buffer in the form of a vector, and the key features extracted from the image or audio information are also stored here.

[0091] As new information is continuously added to the window, the window will slide. The sliding method is similar to an observation box moving on a timeline. For example, if the window size is set to the most recent 10 interactive messages, when the 11th message is generated and included in the window, the oldest message will be moved out of the window. This process is dynamic and automatic, ensuring that the window is always focused on the most recent interactive information. During the sliding process, the system will re-evaluate the priority of the information in the window based on the importance and timeliness of the information. For example, a piece of information that was originally at the edge of the window and had low importance may increase its priority as new information is added and time passes because of its increased relevance to the new information (such as the new information mentions related content), making its storage location in the dynamic buffer more conducive to retrieval.

[0092] When the system needs to perform memory retrieval based on real-time interaction intent, the information in the dynamic buffer managed by the interaction window mechanism will come into play. The system will prioritize retrieving information in the window because this information is recent and filtered, and is more likely to be relevant to the current interaction intent. For example, when answering a user's follow-up questions about a product, previous discussions about the product stored in the window will be quickly retrieved as a reference for answering the question.

[0093] If the information in the window cannot meet the retrieval requirements, the system may expand the retrieval scope according to the dynamic transfer strategy and consider obtaining relevant information from the long-term memory module. However, the interactive window mechanism ensures that the system can first use the most timely and relevant short-term information for a quick response in most cases.

[0094] The interactive window mechanism can also adjust the window size according to the system's operating status and user interaction. For example, in scenarios where user interaction is relatively simple and information relevance is not strong, the window can be appropriately reduced to reduce unnecessary information storage and improve the system's operating efficiency. On the contrary, in complex interactive scenarios, such as in-depth discussions involving multiple topics, the window may be expanded to ensure that sufficient contextual information is retained and provide more data support for the system to fully understand user intent. This ability to dynamically adjust the window size allows the interactive window mechanism to better adapt to different user interaction modes.

[0095] As an optional embodiment, in 204, transferring the key information stored in the short-term memory module to the long-term memory module according to the dynamic transfer strategy can be implemented as follows:

[0096] A dynamic priority queue is established based on the real-time sorting result through a memory integration model; short-term storage information whose priority meets the transfer condition indicated in the dynamic transfer strategy is obtained in the dynamic priority queue according to the memory update cycle or update event indicated in the dynamic transfer strategy; and the short-term storage information that meets the transfer condition is transferred from the short-term memory module to the long-term memory module.

[0097] Exemplarily, the embodiment of the present application introduces a transformer-based attention mechanism to achieve two-way information flow between the above two memory systems, ensuring the coherence and integrity of memory.

[0098] For example, in order to prevent catastrophic forgetting, the system also integrates the elastic weight consolidation (EWC) algorithm and the progressive neural network (PNN) technology to maintain the existing knowledge while maintaining the plasticity of the model. Through this complex combination of technologies, the memory storage module can achieve efficient information management and long-term knowledge accumulation.

[0099] Specifically, the memory integration and update module is one of the core components of the entire personalized memory system, which mainly adopts a multi-level neural network architecture and innovative algorithm mechanism to achieve dynamic memory management.

[0100] Specifically, firstly, an attention mechanism based on the Transformer architecture is used to evaluate the relevance and importance between new and old memories. Through the multi-head self-attention mechanism, the system can automatically identify the semantic associations between different memory fragments and establish a dynamic association weight matrix. This mechanism enables the system to accurately capture the intrinsic connections between memories and provide a basis for subsequent integration. Secondly, an improved memory compression algorithm is used in the memory integration process. The algorithm merges semantically similar memory information through a learnable compression matrix while retaining key information features. This compression mechanism can not only effectively reduce storage space occupancy, but also improve the efficiency of memory retrieval. The system adopts a structure based on variational autoencoder (VAE) to achieve efficient encoding and decoding of memory information.

[0101] In terms of update strategy, the priority-based memory update mechanism implemented by the module comprehensively considers the characteristics of multiple dimensions to determine the priority of memory, including the frequency of use, timeliness and importance of memory. For example, frequently retrieved memories are more critical in the current interaction scenario, and the frequency of use dimension will score higher; memories that are closely related to the current user interaction and are relatively recent have high timeliness scores; and those involving user core needs and key topics have advantages in terms of importance. Based on the evaluation results of these dimensions, the system builds a dynamic priority queue. The order of memories in this queue changes in real time and will be dynamically adjusted according to the changes in the performance of memories in various dimensions, reflecting the importance ranking of memories for the current system operation at all times. In this mechanism, higher priority memories will be retained and updated first because they are more important for the system to accurately understand user intentions and respond to interaction instructions. Low-priority memories face the possibility of being integrated or eliminated to free up space and resources for more valuable memory information. In order to better track the temporal characteristics of memory, the system uses an improved LSTM network. The LSTM network is good at processing data with time series properties, and can capture the order and changes of memory at different time points, providing support for accurately judging the timeliness and other characteristics of memory. At the same time, it also combines reinforcement learning algorithms to optimize the update strategy. Reinforcement learning adjusts the update strategy based on the actual effect of memory update (such as whether it improves retrieval efficiency, whether it is more conducive to responding to users, etc.) through continuous interactive feedback between the system and the environment (which can be understood as user interaction scenarios, etc.), so that the priority judgment and memory retention, integration, elimination and other operations are more in line with actual application needs, and the rationality and effectiveness of memory updates are continuously optimized.

[0102] In order to ensure the consistency and integrity of memory, the module also implements a memory conflict detection and resolution mechanism (Memory Conflict Resolution). When new memory information conflicts with existing memory, the system will start the corresponding processing flow. First, the semantic similarity calculation is used to measure the similarity between the new memory and the existing memory at the semantic level to determine whether they have contradictions or inconsistencies in meaning. At the same time, combined with context analysis, the specific interaction scenarios, previous and subsequent related content, etc. in which these memory information are located are considered to more comprehensively determine whether a conflict exists. In this process, a BERT-based semantic understanding model is used. BERT's powerful semantic representation ability can accurately analyze the semantics of memory content and accurately identify possible semantic conflicts. In addition, in conjunction with knowledge graph technology, the existing knowledge structure and entity relationship information in the knowledge graph are used to assist in judging the rationality of memory information in the entire knowledge system, ensure the coherence and accuracy of the memory system, and avoid deviations in system understanding and response due to conflicting memory information.

[0103] In this way, the memory integration and update module can achieve dynamic management and optimization of memory, which not only ensures the efficient operation of the system, but also maintains the quality and availability of memory. This multi-level technical architecture enables the system to flexibly process and organize memory information like the human brain, providing reliable memory management support for the entire personalized memory system.

[0104] As an optional embodiment, in the above steps, transferring the short-term storage information satisfying the transfer condition from the short-term memory module to the long-term memory module may be implemented as follows:

[0105] receiving short-term storage information satisfying the transfer condition from the short-term memory module;

[0106] Through the long-term memory model, the received short-term storage information is embedded into the long-term memory map related to the target user, and based on the embedding result, the received short-term storage information is stored in the corresponding long-term storage space in the vector database as long-term storage information.

[0107] In the above steps, first of all, it is necessary to clarify that short-term storage information is information that has been screened, evaluated, and stored according to certain rules by the short-term memory module. This information has undergone considerations such as importance and timeliness in the short-term memory module. When they meet specific transfer conditions (for example, according to the dynamic transfer strategy, the frequency of occurrence of information reaches a certain threshold, and it is important to represent the user's long-term behavior pattern, etc.), they will be selected and transferred from the short-term memory module to prepare for transfer to the long-term memory module. This receiving process is to obtain these short-term storage information that have met the transfer requirements, and prepare for their subsequent integration into the long-term memory system.

[0108] Next, the long-term memory model is used to process the received short-term storage information. The long-term memory model here plays a key role. It embeds the short-term storage information into the long-term memory map related to the target user. The long-term memory map is a structured knowledge representation that can present the complex associations between different information. Like a network, it clearly shows the various knowledge nodes surrounding the target user and their connections. In the embedding process, through specific algorithms and techniques, the short-term storage information is "placed" into the map structure in a suitable form, making it part of the map and establishing an intrinsic connection with other existing related information.

[0109] Then, based on the embedding results, the received short-term storage information is stored in the corresponding long-term storage space in the vector database, making it formally long-term storage information. A vector database is a database suitable for storing vector data, which can efficiently manage and retrieve the stored data. By storing information in the form of vectors in the corresponding long-term storage space, on the one hand, it is convenient for subsequent rapid retrieval based on vector features, and on the other hand, it also ensures the orderly storage of long-term memory information, making the entire long-term memory module more reasonable in storage structure, and providing a good foundation for subsequent memory retrieval and other operations.

[0110] For example, the long-term memory (LTM) subsystem uses a knowledge graph structure based on a graph neural network (GNN) and cooperates with a vector database to achieve efficient information storage and retrieval. The system uses the BERT model to encode input information, compresses information dimensions through knowledge distillation technology, and then establishes the relationship between information through the graph attention network (GAT).

[0111] Specifically, the long-term memory (LTM) subsystem adopts a knowledge graph structure based on a graph neural network (GNN), which means that it uses the characteristics of GNN to process and analyze information in the knowledge graph. GNN can automatically learn the complex relationships between nodes (representing different information elements) in the knowledge graph through the information transmission mechanism between nodes and edges, and mine deep-level association patterns. For example, for different types of nodes such as users' long-term interests and hobbies information, past interaction history, etc., GNN can analyze which interests and hobbies have mutually reinforcing or related relationships, which interaction history has an important impact on the current user's behavior pattern, etc., so that the information in the knowledge graph is no longer isolated, but an interconnected and logical whole, providing strong structural support for the system to fully and deeply understand users and accurately retrieve memory information.

[0112] For example, the BERT model is first used to encode the input information. With its powerful language understanding ability, the BERT model can convert information in various text forms into vector representations with rich semantic features, laying the foundation for subsequent processing. Then, the information dimension is compressed through knowledge distillation technology. Knowledge distillation can refine the high-dimensional vector information encoded by BERT, remove some redundant information dimensions, and reduce the amount of data while retaining key semantic features as much as possible, improve the efficiency of information processing and storage, and avoid the storage and computing burden caused by excessive data dimensions.

[0113] In order to process large-scale information, the system adopts a distributed storage architecture, combined with the local sensitive hashing (LSH) algorithm to achieve fast approximate nearest neighbor search. In order to process large-scale information, the system adopts a distributed storage architecture. In practical applications, with the increasing number of user interactions, the amount of information that the long-term memory module needs to store will become very large. If the traditional centralized storage method is adopted, it is easy to face problems such as insufficient storage capacity and slow retrieval speed. The distributed storage architecture stores a large amount of information in multiple different storage nodes, and through a reasonable distributed management strategy, it can achieve effective storage and management of large-scale information. Each storage node can work in parallel to share the pressure of storage and retrieval, just like dividing a large library into multiple small branches, each branch is responsible for managing a part of the books, so that when looking for books (retrieval information), you can search in multiple branches at the same time, which improves the overall processing efficiency and ensures that the system can still run stably when facing massive long-term memory information.

[0114] Combined with the Locality Sensitive Hashing (LSH) algorithm, its main purpose is to achieve fast approximate nearest neighbor search. In the long-term memory vector database, when it is necessary to find other information that is most similar to a given information (i.e., the nearest neighbor information), the traditional precise search method will be very computationally intensive and inefficient in the case of large-scale data. The LSH algorithm cleverly hashes the vector space so that similar vectors are divided into the same or similar hash buckets with a high probability. In this way, when searching, only a few related hash buckets need to be searched to quickly find the approximate nearest neighbor vector, which greatly improves the retrieval speed and provides an efficient search method for the system to quickly locate relevant information in the long-term memory module, further improving the operating efficiency and practicality of the entire long-term memory subsystem.

[0115] As an optional embodiment, determining the timing of transferring short-term stored information is a complex process involving comprehensive consideration of multiple factors and strategies. If certain short-term stored information appears frequently within a certain time frame, this indicates that this information may be of high importance to the user or is closely related to the user's core needs. For example, in a shopping recommendation system, if a user mentions his interest in a certain brand of electronic products many times in a short period of time, such as mobile phone models, computer configurations and other related information, the system will consider transferring this frequently appearing information from the short-term memory module to the long-term memory module. Because this high-frequency information is likely to be content that users pay attention to for a long time, it is of great value for understanding users' long-term preferences and intentions.

[0116] In order to quantify this frequency of occurrence, the system can set a frequency threshold. When the number of occurrences of short-term storage information reaches or exceeds this threshold, the transfer operation is triggered. The setting of the threshold can be determined based on factors such as the specific application scenario of the system, user behavior patterns, and storage resources. For example, in a simple question-and-answer system, if information related to a certain topic appears 6 times in 10 interactions, the transfer threshold is reached and it is transferred to the long-term memory module so that it can be better utilized in subsequent questions and answers to provide more accurate answers.

[0117] Short-term stored information has timeliness, but as time goes by, some information, although its timeliness is decreasing, shows long-term value. For example, a user mentioned a new technical requirement in a product consultation. Although the timeliness of this requirement may gradually decrease as the consultation ends, if the system determines that this technical requirement may still play a key role in the user's future product selection or use, then it has long-term value. The system will determine the transfer timing based on dynamic monitoring of the timeliness of information and the estimation of long-term value.

[0118] There are many factors that can be considered to determine the long-term value of information. For example, whether the information is related to the user's long-term goals, such as a grammatical point that the user repeatedly mentions in the process of learning a foreign language, which is related to the long-term goal of learning a foreign language; whether the information involves the user's basic preferences and habits, such as reading preferences and exercise habits that users often mention; or whether the information is related to the core service functions of the system. For a financial service system, information related to a certain financial management method that users frequently ask about has long-term value. When the system determines that the short-term stored information has sufficient long-term value, it will transfer it to the long-term memory module in a timely manner.

[0119] The timing of the transfer can be determined by observing changes in user interaction patterns. For example, when the user's interaction changes from the initial information collection stage to the in-depth comparison and selection stage, some key information previously obtained in the information collection stage, such as the basic categories of products and basic functional parameters that users care about, can be transferred from short-term memory to long-term memory. This is because this information will still play an important supporting role in the subsequent interaction process, and this change in interaction pattern implies the long-term usefulness of this information.

[0120] Determine the timing of transfer based on the system's own tasks and functional requirements. If the system's main task is to provide users with personalized services, such as personalized recommendations, personalized learning path planning, etc., then when short-term storage information is important for long-term planning to complete these tasks, it should be transferred. Taking the personalized learning system as an example, when the short-term stored user learning progress information and knowledge mastery reaches a certain level and can provide a reference for the formulation of long-term learning plans, this information will be transferred to the long-term memory module so that the system can better track the user's long-term learning trajectory and provide more effective learning suggestions.

[0121] The reward feedback mechanism uses a reinforcement learning algorithm to dynamically adjust the transfer strategy. The system can give reward or penalty feedback based on the actual effect of the information transfer. For example, if a piece of short-term storage information is transferred to the long-term memory module, and in subsequent user interactions, the information is successfully used to provide more accurate services or answers, such as improving the accuracy of recommendations, enhancing the satisfaction of answering questions, etc., the system will give a positive reward signal. On the contrary, if the transferred information does not play a positive role, or causes the system performance to deteriorate, such as increasing the retrieval time but not bringing actual value, a negative feedback will be given.

[0122] By continuously accumulating these feedback signals, the reinforcement learning algorithm can optimize the dynamic transfer strategy and adjust the transfer timing of short-term stored information. For example, the transfer timing may be set initially based on simple frequency rules, but as the reinforcement learning process finds that some high-frequency but ineffective information takes up too many resources after transfer, the strategy can be adjusted to increase consideration of long-term value and actual effects, and the transfer timing can be determined more reasonably, so that the information transferred to the long-term memory module is truly helpful for the long-term operation of the system and the improvement of service quality.

[0123] As an optional embodiment, the storage information of the short-term memory module and the long-term memory module carries a memory code; the memory code is used to identify the information content type of the corresponding storage information.

[0124] Memory encoding plays a key role in identifying the type of information content corresponding to the stored information.

[0125] For the short-term memory module, since it temporarily stores real-time interaction and retrieval information with users within a preset period of time, these information sources are diverse and complicated. Memory encoding can quickly distinguish whether the information is interactive discourse in text form, key voice content in audio, or key feature descriptions related to images, such as using "T" to represent text, "A" to represent audio, and "I" to represent image. In this way, when performing retrieval based on real-time interactive intent, the system can quickly locate the corresponding type of information based on the encoding, thereby improving retrieval efficiency.

[0126] The same is true for key information stored in the long-term memory module. It covers important content from many past interactions, and memory encoding facilitates the classification and management of key information of different content types. For example, when building a knowledge graph based on user interests, encoding can easily filter out information nodes that belong to the same interest content type, and better sort out the relationship between information. Moreover, when information needs to be integrated or updated, memory encoding can help the system accurately determine which types of information can be merged and which require special processing, ensuring the orderliness of memory module information and the convenience of subsequent use, helping the system to achieve more accurate and efficient memory management and retrieval utilization.

[0127] As an optional embodiment, in the above steps, the received short-term storage information is embedded into the long-term memory map related to the target user through the long-term memory model, and the received short-term storage information is stored in the corresponding long-term storage space in the vector database based on the embedding result, as the long-term storage information, which can be implemented as follows:

[0128] The received short-term storage information is encoded to obtain an initial information code; the information dimension of the initial information code is compressed through knowledge distillation processing to obtain an optimized information dimension corresponding to the initial information code; according to the optimized information dimension, the correlation between the initial information codes is established; according to the correlation and the optimized information dimension, the graph attribute corresponding to the initial information code in the long-term memory graph is determined; wherein the graph attribute includes at least: the dimension to which the information belongs and the information position; based on the dimension to which the information belongs and the information position, the initial information code is stored in the corresponding long-term storage space in the vector database.

[0129] It is worth noting that after receiving the short-term storage information from the short-term memory module, it must first be encoded. This process aims to convert these information of different forms and contents into specific code forms that the computer can better process and recognize, that is, the initial information encoding. For example, for short-term storage information such as text, the text is mapped into a vector-shaped encoding through word embedding and other technologies; the image information extracts key features and then encodes them into corresponding vectors, and the audio information is similarly converted into a code representing its features, so that the initial information encoding is obtained, laying the foundation for subsequent further processing.

[0130] The initial information encoding may be too high in dimension or contain redundant information, which is where knowledge distillation comes into play. Based on the existing knowledge system and rules, it extracts and compresses the initial information encoding, removes dimensions that are not critical to expressing the core meaning of the information, and retains the most representative and important parts, thereby optimizing the information dimension. It is like extracting the key points from a lengthy article, which not only retains the core content but also makes the data more streamlined, which helps with subsequent efficient storage and processing, while reducing the burden of computing and storage.

[0131] Based on the optimized information dimension after compression, the internal connection between each initial information encoding can be analyzed. For example, the similarity measurement method is used to determine the similarity of the information represented by different encodings in terms of semantics, features, etc., and then establish the association. If two information encodings are semantically similar after analysis, or target different aspects of the same topic, then the corresponding association is established so that the information is no longer isolated, preparing for the construction of a complete long-term memory map.

[0132] Based on the previously established relevance and optimized information dimensions, further clarify the map attributes of each initial information encoded in the long-term memory map, such as the dimension to which the information belongs (user interest dimension, behavior dimension, etc.) and the location of the information (specific node location in the map structure, etc.). This is equivalent to accurately "positioning" the information in the map so that it can be integrated into the logical structure of the map.

[0133] Finally, according to the determined information dimension and information location, the initial information encoding is accurately stored in the corresponding long-term storage space of the vector database, making it long-term storage information, which is convenient for subsequent efficient retrieval and utilization based on graph structure and vector features, and improves the information storage and management mechanism of the long-term memory module.

[0134] Further optionally, in the above steps, after the information to be stored is stored in the short-term memory module according to the importance score and the timeliness score, or after the key information stored in the short-term memory module is transferred to the long-term memory module according to the dynamic transfer strategy, it can also be implemented as the following steps:

[0135] 301. Assign corresponding time weights to the short-term storage information stored in the short-term memory module and the long-term storage information stored in the long-term memory module respectively;

[0136] 302. Dynamically adjust the time weight through an exponential decay function; wherein the exponential decay function is constructed based on a dynamic forgetting algorithm of time decay, and the dynamic forgetting algorithm is obtained according to the setting of the Ebbinghaus forgetting curve;

[0137] 303. Monitor the access frequency, recent usage time and correlation between short-term storage information and long-term storage information and other information;

[0138] 304. Based on the monitoring results, a forgetting assessment model is used to rank the importance of short-term storage information and long-term storage information;

[0139] 305. Performing context relevance analysis on the short-term storage information and the long-term storage information to obtain the interactive topic relevance of the short-term storage information and the long-term storage information;

[0140] 306. Determine the short-term storage information and / or long-term storage information to be deleted according to the importance ranking result, the interaction topic relevance, and the time weight.

[0141] It is understandable that, first, the above steps adopt a dynamic forgetting algorithm based on time decay (Time-BasedDecay Algorithm). The algorithm draws on the principle of the Ebbinghaus forgetting curve, assigns a time weight to each piece of memory information, and dynamically adjusts the weight value through an exponential decay function. Over time, the system automatically reduces the weight of older information, but retains key knowledge points. Drawing on the principle of the Ebbinghaus forgetting curve, the curve reveals that people forget knowledge in a pattern of first fast and then slow over time. In the system, corresponding time weights are configured for short-term storage information stored in the short-term memory module and long-term storage information stored in the long-term memory module, which means that the newly stored information will be given a relatively high weight because it is closer to the user interaction context at the moment and has a stronger timeliness. For example, the product details that the user has just asked about have a larger weight when initially stored.

[0142] These time weights are dynamically adjusted through an exponential decay function. As time goes by, just as people gradually forget what they learned in the past, the system automatically reduces the weight of older information. However, for information that is a key knowledge point, even as time passes, the reduction in its weight will be relatively small or it will still maintain a certain importance, ensuring that it will not be easily forgotten. For example, information related to core interests and hobbies that users have been paying attention to for a long time, even if it has not been accessed for a long time, due to its critical nature, the weight can still be maintained at a level that is not deleted. Therefore, in the overall memory management, it not only reflects the impact of time on memory, but also ensures the retention of key content.

[0143] Second, the module integrates a frequency-based importance assessment mechanism (Frequency-Based Importance Assessment). The system builds a multi-dimensional importance scoring model by tracking the access frequency, recent usage time, and correlation with other information of each piece of information. A multi-dimensional importance scoring model is built by tracking the access frequency, recent usage time, and correlation with other information of each piece of information. Information with high access frequency means that it is often called during the user interaction process, and is often closely related to the user's current needs, so its importance is naturally higher; the closer the recent usage time is, the more valuable it is in the current interaction scenario; and the correlation with other information reflects the status of the information in the entire knowledge system. The more correlations there are, the greater the role it may play in understanding the overall user interaction intention. For example, in a knowledge question-and-answer system, a knowledge point that is frequently queried and related to multiple related topics will perform well in these dimensions, and its importance score will be higher.

[0144] The improved TD-IDF algorithm is used in combination with the attention mechanism to sort the importance of information and provide a basis for forgetting decisions. The TD-IDF algorithm is good at measuring the importance of words in a document. After improvement, it can be better used to evaluate the importance of memorized information. The attention mechanism can focus on the key parts of the information and the importance of the correlation between different information. Through the combination of the two, the order of importance of each piece of information can be accurately determined, providing a reliable basis for subsequent forgetting decisions, so that the system can more scientifically distinguish which information is more worth keeping and which can be considered forgotten.

[0145] Third, a contextual relevance analysis engine was introduced. The engine uses an improved Transformer architecture to analyze the semantic associations between information through a self-attention mechanism. The system retains information that is highly relevant to the user's recent interaction topics, while gradually fading or cleaning up low-relevance content. The system uses a reinforcement learning framework to optimize the forgetting strategy (Reinforcement Learning for Forgetting Strategy). By designing a suitable reward function, the system can continuously learn and adjust the forgetting strategy from interactive feedback, striking a balance between maintaining the efficiency of the memory system and retaining important information. The Deep Q-Network (DQN) model is used for forgetting decisions, which make decisions based on the temporal characteristics, importance scores, and contextual relevance of the information.

[0146] During the interaction between the user and the system, different interaction topics will be formed. Information that is highly relevant to the user's recent interaction topics is critical to understanding the current user's intentions and providing accurate responses, and the system will retain this part of the content. For example, in a conversation about travel planning, recently mentioned information such as attractions and transportation will be retained because of their strong relevance to the interaction topic, while content that is irrelevant to the topic or has a very low relevance will be gradually faded or cleaned up to ensure that the memory system focuses on currently useful information. The system uses a reinforcement learning framework to optimize the forgetting strategy. By designing a suitable reward function, it continuously learns and adjusts the forgetting strategy based on the actual interaction feedback after the information is forgotten or retained (such as whether it improves retrieval efficiency, whether it is more conducive to accurately responding to users, etc.), and finds a balance between maintaining the efficiency of the memory system and retaining important information. The forgetting decision adopts the Deep Q-Network (DQN) model, which comprehensively considers the temporal characteristics of the information (i.e., the temporal weight), the importance score (from the previous importance assessment mechanism), and the contextual relevance, and makes decisions on which short-term stored information and / or long-term stored information should be deleted after comprehensive consideration, making the entire forgetting process more intelligent and reasonable, in line with the system's operating requirements and the user's interaction characteristics.

[0147] To sum up, these three aspects work together to build a relatively complete memory information management and forgetting decision-making mechanism, so that the information in the short-term memory module and the long-term memory module can be dynamically adjusted according to its actual value, time factors, contextual relevance and other factors, to ensure the efficiency and effectiveness of the memory system.

[0148] As an optional embodiment, in 306, the short-term storage information and / or long-term storage information to be deleted is determined according to the importance ranking result, the interaction topic relevance, and the time weight, including at least one of the following:

[0149] Setting the short-term storage information and / or long-term storage information whose relevance to the interactive topic within the set time period is lower than the set relevance threshold as the short-term storage information and / or long-term storage information to be deleted;

[0150] Setting the short-term storage information and / or long-term storage information whose importance ranking is lower than the set ranking threshold as the short-term storage information and / or long-term storage information to be deleted;

[0151] The short-term storage information and / or long-term storage information whose time weight is lower than the set time weight threshold is set as the short-term storage information and / or long-term storage information to be deleted.

[0152] In 102, based on the real-time interaction intention, the short-term memory module and the long-term memory module are memory retrieved to obtain the target retrieval information. Specifically, in the information classification and indexing module, a multi-level technical architecture is adopted to realize an efficient information retrieval system. First, an improved vector database technology is adopted at the bottom layer, and a high-performance vector indexing framework such as FAISS or Milvus is used to convert all information content into a high-dimensional vector representation. These vectors are generated by pre-trained Transformer encoders (such as BERT or RoBERTa) to ensure the accurate capture of semantic information. At the classification level, the system organically combines the hierarchical topic model (HTM) and the dynamic clustering algorithm. HTM can automatically mine the hierarchical structure contained in the text topic, sort out the clear context from the big topic to the subdivided small topic, and make the information category hierarchy clear. At the same time, the dynamic clustering algorithm can adjust the category system in real time according to the newly added information, so that the classification system can always adapt to the changing information content. In specific implementation, an adaptive multi-level classification system is constructed by combining the improved LDA (latent Dirichlet allocation) model with the HDBSCAN clustering algorithm to more accurately classify various types of information and facilitate rapid locating of the corresponding category to find the target retrieval information during retrieval.

[0153] For example, suppose there is a knowledge base of an online learning platform, which stores a huge amount of learning materials, including various types of information such as various subject knowledge, learning methods, and examination information. At the classification level, the hierarchical topic model (HTM) is used. For example, for the subject knowledge section, it can first sort out major themes such as "natural sciences" and "humanities and social sciences", and then subdivide "physics", "chemistry", "biology" and other small themes under "natural sciences", clearly presenting a hierarchical structure, so that the knowledge categories are clear at a glance. At the same time, the dynamic clustering algorithm comes into play. When new learning materials are uploaded, such as content related to the emerging interdisciplinary subject "biochemistry", it will adjust the category system in real time based on the new information, and reasonably classify "biochemistry" into relevant levels, so that it can be integrated into the existing classification framework. Specifically, by combining the improved LDA (latent Dirichlet allocation) model with the HDBSCAN clustering algorithm, taking learning method materials as an example, the LDA model can analyze the method themes that different materials focus on, such as "memory methods" and "problem-solving skills", and the HDBSCAN clustering algorithm then accurately clusters the materials based on these themes to build an adaptive multi-level classification system. In this way, when users search for materials related to "problem-solving skills in physics", they can quickly locate the corresponding category and efficiently find the target search information.

[0154] In order to improve the retrieval efficiency, a hybrid index structure is designed, which makes comprehensive use of inverted index and vector index. The inverted index is responsible for fast keyword matching, while the vector index handles semantic similarity search. At the same time, an attention mechanism-enhanced relevance sorting algorithm is introduced, which can dynamically adjust the retrieval weight according to the query context. The system also uses local sensitive hashing (LSH) technology to improve the retrieval speed of large-scale data through dimensionality reduction processing.

[0155] In order to improve the retrieval efficiency, a hybrid index structure is specially designed, which integrates the inverted index and the vector index to work together. The inverted index is responsible for achieving fast keyword matching. When there are clear keywords in the real-time interaction intention, it can quickly filter out information containing these keywords; the vector index focuses on processing semantic similarity search, and searches for information with similar semantics to the query intention based on the high-dimensional vector representation generated previously. The two complement each other and fully cover different retrieval needs. On the one hand, an attention mechanism-enhanced relevance sorting algorithm is introduced, which can dynamically change the retrieval weight with reference to the query context. For example, in the retrieval, if the user's real-time interaction intention focuses on a certain aspect of content, it will increase the weight of the relevant information accordingly, making it more advanced in the retrieval results, so that the target retrieval information is more in line with the user's current needs. On the other hand, the local sensitive hashing (LSH) technology is used to reduce the dimensionality of high-dimensional vectors, effectively speeding up the retrieval speed under large-scale data, preventing low retrieval efficiency due to the large amount of data, and ensuring that the target retrieval information can be quickly screened out from a large amount of information.

[0156] For example, in a news information retrieval system, the user's real-time interactive intention is "the latest application results of artificial intelligence in the field of science and technology". The hybrid index structure is in operation. The inverted index quickly selects the content containing the corresponding vocabulary from the massive news information based on the keywords "science and technology", "artificial intelligence" and "application results". For example, the relevant reports with these words in the title are first screened out. The vector index relies on the high-dimensional vector representation generated by the news text to find information with similar semantics, such as those that introduce the new application of artificial intelligence in different scenarios. Even if they do not use exactly the same expression, they can be retrieved. The two work together to meet multiple retrieval needs. Then, the relevance sorting algorithm enhanced by the attention mechanism takes effect. Because the user focuses on the "latest application results", the system increases the weight of related content such as recent innovative application cases, so that it is presented more advanced in the search results. Finally, in the face of a large amount of news data (large-scale data), the local sensitive hashing (LSH) technology reduces the dimension of the news vector, speeds up the retrieval speed, and helps to quickly select the news information that best meets the user's needs as the target retrieval information, effectively improving the overall retrieval efficiency.

[0157] In terms of index updates, we use an incremental update strategy, combined with Bloom Filter for fast duplicate checking, and use the Skip-gram model to continuously optimize the word vector space. In this way, we can only update new or changed information to avoid large-scale reconstruction of the entire index, saving resources and time. In addition, we use Bloom Filter for fast duplicate checking to quickly determine whether the information is already in the index, improving update efficiency. At the same time, we use the Skip-gram model to continuously optimize the word vector space, so that the word vector can better reflect the semantic relationship between words, further improve the retrieval accuracy, ensure that the index is in a good state to serve memory retrieval, and make it more accurate and efficient to obtain target retrieval information.

[0158] For example, suppose there is an e-commerce product information index system. As new products are constantly put on the shelves and product information is updated, the index needs to be maintained. An incremental update strategy is adopted. For example, when a new smart watch is launched, the system will not rebuild the entire product index, but only add the brand, model, function, price and other information of the new watch to the corresponding index position, saving a lot of time and resources. In this process, BloomFilter is combined for fast duplication checking. For example, when a merchant may mistakenly upload the same smart watch information twice, BloomFilter can quickly determine that the information already exists in the index, avoid repeated insertion, and improve the update efficiency. At the same time, the Skip-gram model is used to continuously optimize the word vector space. For words in product descriptions, such as "high-definition screen" and "long battery life", the Skip-gram model can learn the semantic relationship between these words and other related words (such as "clear display" and "durable battery") by analyzing a large amount of product text data, so that the word vector can express the meaning of the word more accurately. In this way, when a user searches for "watch with clear screen", even if there is no exact matching word in the index, the smart watch can be found through the semantic association of the word vector, further improving the retrieval accuracy and ensuring that the target retrieval information can be obtained more accurately and efficiently, allowing users to quickly find their favorite products.

[0159] In order to process multimodal information, the system integrates a cross-modal attention network to achieve unified indexing and retrieval of different types of data such as text and images. With it, unified indexing and retrieval of different types of data can be achieved. This means that no matter what kind of modal information the user's real-time interaction intention involves, the system can retrieve it under a unified framework, eliminate modal differences, and more comprehensively find the target retrieval information related to the user's intention, thereby enhancing the completeness and accuracy of the retrieval.

[0160] For example, in a travel guide sharing platform, the user's real-time interactive intention is to find relevant information about the "Spring Beauty of West Lake in Hangzhou". The platform contains both text travel notes shared by tourists describing the spring scenery of West Lake, such as "West Lake is weeping willows, peach blossoms are in full bloom, and the lake is sparkling", and pictures of the spring beauty of West Lake taken by everyone, showing the image information of flowers, willows, lakes and mountains. The cross-modal attention network integrated by the system plays a role. When the user initiates a search, it can associate the key elements mentioned in the text with the corresponding features presented in the image, and index and retrieve them under a unified framework. Even if the user only enters the text "West Lake Peach Blossom", not only can the travel notes containing these words be retrieved, but also the pictures of West Lake that focus on the scene of peach blossoms in full bloom can be retrieved together. Similarly, if a user uploads a new photo of West Lake in spring, the system can analyze the photo content through a cross-modal network, combine it with the existing text description, accurately determine the subject of the photo, and index it reasonably. When a user searches for relevant information in the future, it can fully present various modal information related to it, eliminating the modal differences between text and images, enhancing the completeness and accuracy of retrieval, and better meeting user needs.

[0161] In order to fully optimize the retrieval performance, the system implements an adaptive caching mechanism and uses the LRU-K algorithm to manage hot data. It effectively caches frequently accessed hot data, and the next retrieval can be directly obtained from the cache, which speeds up the retrieval. The distributed storage architecture is used to improve the scalability of the system, so that it can cope with the growing amount of data and easily handle large-scale memory information storage and retrieval tasks. In addition, a reinforcement learning-based index optimizer is introduced, which can automatically adjust the index strategy according to the changing query pattern, continuously optimize the index structure to better adapt to the actual retrieval needs, further improve the retrieval efficiency, and help more efficiently obtain the target retrieval information that meets the real-time interaction intention.

[0162] As an optional embodiment, in 102, performing memory retrieval on the short-term memory module and the long-term memory module based on the real-time interaction intention to obtain target retrieval information can be implemented as follows:

[0163] 401. Performing deep semantic analysis on the real-time interaction intention to obtain a real-time search target corresponding to the target user;

[0164] 402. Retrieve first candidate information associated with the real-time retrieval target from the short-term memory module;

[0165] 403. Retrieve second candidate information associated with the real-time retrieval target and / or the first candidate information from the long-term memory module;

[0166] 404. Map the real-time retrieval target, the first candidate context information, and the second candidate information into the same vector space to obtain respective corresponding projection vectors;

[0167] 405. Calculate a first similarity between a first projection vector corresponding to the first candidate information and a reference projection vector corresponding to the real-time retrieval target;

[0168] 406. Calculate a second similarity between a second projection vector corresponding to the second candidate information and the reference projection vector;

[0169] 407. Generate the target retrieval information based on the first similarity and the second similarity; the target retrieval information is obtained based on inference of a multi-hop Bayesian inference network.

[0170] In an embodiment of the present application, the corresponding proportions of the first candidate information and the second candidate information in the target retrieval information are dynamically configured using an attention mechanism; the attention mechanism is associated with a retrieval strategy pre-configured in the retrieval scenario.

[0171] Suppose an intelligent learning assistant system, the user inputs the real-time interaction intention of "understanding the application cases of the buoyancy principle in physics in life". First, the system conducts a deep semantic analysis of the intention and clarifies that the real-time retrieval target is the application of the physical buoyancy principle in life. Then, the buoyancy-related notes mentioned by the user when learning physics recently are retrieved from the short-term memory module as the first candidate information, such as the understanding record of the buoyancy calculation formula. Then, the knowledge points of the buoyancy principle application in life accumulated in the past, such as the floating of boats and the use of swimming rings, are found from the long-term memory module as the second candidate information. After that, the real-time retrieval target, the first candidate information, and the second candidate information are mapped to the same vector space to obtain their respective projection vectors, and then the first and second similarities are calculated respectively. For example, if you have just learned buoyancy calculation recently and are exploring its principle application, then the first similarity corresponding to the first candidate information is high, and the second similarity of the common application cases accumulated in the past is also good. Finally, based on the similarity between the two, the target retrieval information is obtained through multi-hop Bayesian inference network reasoning. The attention mechanism dynamically configures the ratio of the two. For example, when discussing the principle application scenario at this moment, the system uses the attention mechanism to increase the proportion of the second candidate information (life application cases) in the target search information based on the pre-configured retrieval strategy, and gives priority to displaying specific application content such as boats and swimming rings, so that the presented retrieval results are more in line with the current user's needs to have a deeper understanding of the application cases, helping to efficiently obtain useful information.

[0172] In the specific implementation of the above step 102, in terms of retrieval technology, a hybrid retrieval strategy combining vector retrieval (Vector Retrieval) and semantic retrieval (Semantic Search) is adopted. The query content and stored memory information are mapped to the same vector space through the dense passage retrieval (DPR) model to achieve fast similarity matching. At the same time, a BERT-based semantic understanding model is introduced to perform deep semantic analysis on the retrieval content to improve retrieval accuracy. The system also integrates an attention mechanism to dynamically adjust the weights of different retrieval strategies.

[0173] In order to improve retrieval efficiency, the system implements a hierarchical index structure and uses locality sensitive hashing (LSH) technology to build a fast retrieval index. When processing large-scale memory data, a distributed retrieval architecture is used to improve retrieval speed through asynchronous parallel processing. In addition, the system also integrates an adaptive cache mechanism to dynamically adjust the storage level of hot data according to the access frequency.

[0174] For example, in an enterprise's internal knowledge management system, an employee wants to find content related to "recommendations on the selection of online marketing channels in new product promotion plans." Using a hybrid search strategy that combines vectorized search with semantic search, the system first maps the employee's query content and the memory information such as the numerous product promotion knowledge documents stored in the system to the same vector space through the dense paragraph search (DPR) model, quickly screening out documents that mention keywords such as "new product promotion" and "online marketing channels," and completing preliminary similarity matching.

[0175] At the same time, the BERT-based semantic understanding model deeply analyzes the semantics of the query content, accurately grasps the key semantic requirement of "selecting suggestions", and further filters out documents that truly meet the deep meaning of employees' desire to obtain suggestions, thereby improving retrieval accuracy.

[0176] The attention mechanism dynamically adjusts the weights. If the system determines that semantic retrieval is more in line with the needs at this moment, it will increase its weight and focus on presenting results with high semantic matching.

[0177] The hierarchical index structure is combined with the fast search index built with local sensitive hashing (LSH) technology, which can quickly locate relevant documents in large-scale knowledge data. The distributed search architecture performs asynchronous parallel processing to speed up the overall search speed.

[0178] The adaptive caching mechanism comes into play. If the knowledge related to "online marketing channels" has been frequently accessed recently, it will be upgraded to a higher storage level as hot data, so that it can respond faster the next time there is a similar search. Employees will be able to efficiently obtain accurate and useful content related to online marketing channel selection recommendations in new product promotion plans.

[0179] For example, taking the intelligent customer service system as an example, when a user has multiple rounds of conversations with customer service to inquire about product usage issues, the multi-head attention mechanism based on the Transformer architecture comes into play. For example, the user first describes some phenomena of product failure, and then mentions the previous operation steps after a few sentences. The system can adaptively pay attention to the historical information of the previous related operation steps by encoding them into query, key and value matrices, and can associate them even if they are far apart, understand the conversation context, and grasp the overall situation. In the hierarchical semantic understanding framework, the underlying BERT-type pre-trained model performs basic semantic encoding on each sentence of the user, the middle-level graph neural network (GNN) conversation topic tracking mechanism can lock in the topic of discussing product failure, and the top-level attention pooling completes global semantic integration. For example, when a user talks about the performance of a failure and asks questions about the operation, it can capture semantics at different levels at the same time and accurately understand the intention.

[0180] In long-term conversations, if there are many conversation turns, the dynamic context management mechanism based on sliding windows can adjust the window size and adaptively retain key product failure-related content and remove irrelevant chat information according to the importance score calculated by the improved REINFORCE algorithm. The conversation state tracking mechanism based on the hierarchical recurrent neural network (HRED) maintains coherence. For example, when the user switches from talking about the fault phenomenon to asking about the repair method, it can accurately capture the topic transition and reset the context when necessary. The integrated conditional random field (CRF) model better models the state transition sequence. Finally, the integrated learning strategy weights the outputs of sub-models such as intent recognition and sentiment analysis. If the intent recognition sub-model is more critical to judging user needs at this moment, the adaptive online learning algorithm will increase its weight, allowing the customer service system to understand the context more reliably and respond to customers accurately.

[0181] Optionally, a multi-head attention mechanism based on the Transformer architecture is used to capture long-distance dependencies in the conversation. By encoding the conversation history into three matrices: query, key, and value, the system can adaptively focus on relevant historical information and achieve dynamic understanding of the context. At the same time, the introduction of positional encoding ensures the retention of conversation order information. The module integrates a hierarchical semantic understanding framework. The BERT-like pre-trained model is used at the bottom layer for basic semantic encoding, the middle layer adopts a conversation topic tracking mechanism based on a graph neural network (GNN), and the top layer implements global semantic integration based on attention pooling. This hierarchical design can capture semantic information at the word level, sentence level, and conversation level at the same time. In order to handle long-term conversation scenarios, the module also implements a dynamic context management mechanism based on sliding windows. By setting an adjustable context window size and combining an importance scoring algorithm, the system can adaptively retain key context information while avoiding the accumulation of irrelevant information. The importance scoring adopts an improved REINFORCE algorithm to optimize the context selection strategy through reinforcement learning. When processing multi-round conversations, the module introduces a conversation state tracking mechanism based on a hierarchical recurrent neural network (HRED). This mechanism can maintain the coherence of the conversation, accurately capture the topic transition points, and trigger context reset when necessary. At the same time, by integrating the conditional random field (CRF) model, the system can better model the state transition sequence in the conversation.

[0182] Optionally, in order to improve the robustness of the system, an integrated learning strategy is adopted to weight the outputs of multiple specialized sub-models (such as intent recognition, sentiment analysis, reference resolution, etc.) to obtain more reliable context understanding results. The weight update adopts an adaptive online learning algorithm, which can dynamically adjust the importance of each sub-model according to the actual dialogue effect.

[0183] As an optional embodiment, in 103, the target search information is personalized in combination with the personalized knowledge base of the target user to obtain response information that matches the target user's usage habits, including:

[0184] Perform multi-head attention calculation on the target retrieval information; based on the calculation results, evaluate the consistency between the target retrieval information and the historical interaction information stored in the short-term memory module and the long-term memory module; if the consistency evaluation is passed, generate initial response information based on the target retrieval information; use control codes to perform conditional text generation processing on the initial response information to optimize the personalized expression of the initial response information; wherein the personalized expression includes at least: tone, style, and idioms; generate dynamic image adjustment parameters based on the image information in the historical interaction information, and optimize the image information in the initial response information through the dynamic image adjustment parameters to obtain response information that matches the target user's usage habits.

[0185] Specifically, in this step, a multi-head attention mechanism is used to process the target retrieval information. The multi-head attention mechanism is like multiple "perspectives", which can focus on the key content of the target retrieval information and the relationship between its internal elements from different aspects. For example, for target retrieval information in various forms such as text and images, it can focus on the connection between different semantic blocks of the text part, as well as the correspondence between image elements and text descriptions. Through such calculations, richer and more detailed information features can be excavated, providing a basis for subsequent operations such as consistency evaluation.

[0186] The results calculated by multi-head attention are used to measure the consistency between the target retrieval information and the historical interaction information stored in the short-term memory module and the long-term memory module. The historical interaction information records every bit of the previous interaction with the target user, reflecting the user's past concerns, preferences, etc. Through comparative analysis, for example, check whether the topics and key content mentioned in the target retrieval information are consistent with the content frequently discussed by the user before, and whether the expression style of the text is close to the previous communication style. If there is a high degree of consistency in many aspects, it means that the target retrieval information is consistent with the user's past interaction habits, and the consistency assessment can be passed, which means that the information is in line with the user's consistent interaction context and can proceed to the next step of response information generation.

[0187] When the consistency assessment is passed, the initial response information is generated based on the target search information. This process will comprehensively consider various elements in the target search information and convert them into a prototype of content that can be replied to the user. For example, if the target search information revolves around a product function that the user inquires about, the initial response information will integrate relevant function introductions, usage suggestions and other content, and present them in a relatively complete and clear manner to form a basic text content that can respond to user needs. Of course, it may also be accompanied by corresponding elements such as images to make its content richer.

[0188] Next, the control code is used to generate conditional text for the initial response information, aiming to optimize its personalized expression. The control code plays a guiding role here. Different target users have different expected tones, styles, and idioms. For example, for young and lively users, the control code may be used to guide the generation of relaxed, humorous, and colloquial response information; while for more professional and rigorous users, it is prompted to generate formal and professional expressions. In this way, the initial response information is more in line with the user's preferences in terms of expression, enhancing the user's intimacy and recognition of the response content.

[0189] Finally, considering that the response information may contain image elements, image information is extracted from the historical interaction information, and the user's past preferences for images, such as favorite tones and composition methods, are analyzed to generate dynamic image adjustment parameters. These parameters are used to optimize the image information in the initial response information, such as adjusting the brightness and contrast of the image to make its visual effect more in line with user habits, or changing the composition to make its presentation more satisfactory to users. After such a series of processing, the response information that matches the target user's usage habits is finally obtained, which fully meets the user's personalized needs and enhances the user's experience of interacting with the system.

[0190] As an optional embodiment, the personalized adaptation module builds a comprehensive and advanced user personalized adaptation system, covering a variety of cutting-edge technologies and innovative strategies, aiming to accurately grasp user needs and provide highly tailored personalized services. For example, the underlying vector embedding technology is used to convert various types of data such as user interaction behaviors and preference characteristics into high-dimensional vector forms, so that they can be effectively mathematically represented in high-dimensional vector space for subsequent calculations and analysis. In this process, combined with the attention mechanism, dynamic weights are flexibly assigned to different user features based on the user's real-time interaction and interest change trends. For example, when a user frequently browses a certain type of specific product information recently, the feature weights related to the product will be increased accordingly, thereby accurately capturing the dynamic fluctuations of user interests and ensuring that the user portrait is always close to the user's current interest status.

[0191] For example, we can further build a user knowledge graph with the help of graph neural network (GNN), integrate various scattered information of users, and clearly present the internal correlation between them, so as to fully display the overall picture of users in a structured way. For example, we can connect information nodes such as users' purchase records, browsing history, search keywords, and behavior performance in different scenarios in the form of graphs to form an organic knowledge network, and deeply explore the potential connections and rules behind user behaviors.

[0192] For example, through incremental learning algorithms, the system can acquire and integrate new user interaction data in real time, update and optimize existing user models in a timely manner, and seamlessly adapt to the continuous evolution of user interests. Every new interaction behavior is regarded as a learning opportunity, so that the model can keep pace with the times, accurately reflect the latest changes in users' interests and needs, and ensure the timeliness and accuracy of personalized services.

[0193] At the same time, the Meta-Learning framework is introduced, especially the Model-Agnostic Meta-Learning (MAML) algorithm, which enables the system to quickly learn and adapt when facing new users. The algorithm can quickly adjust the model parameters with a small number of samples, find personalized configurations suitable for new users, effectively solve the cold start problem of new users, provide new users with efficient and accurate personalized experience, and accelerate the adaptation process between the system and new users.

[0194] In terms of adaptive strategies, the module uses the policy gradient-based REINFORCE algorithm in reinforcement learning to continuously collect feedback signals through continuous interaction with users and optimize personalized strategies based on these signals. For example, when the content recommended by the system to users receives positive feedback from users (such as clicks, favorites, good reviews, etc.), the system will strengthen the recommendation strategy; otherwise, the strategy will be adjusted and improved to gradually improve the quality and effect of personalized recommendations, making the interaction between the system and users smoother and more satisfactory.

[0195] In order to find the best balance between exploring new personalized strategies and using existing effective strategies, the system uses the Multi-armed Bandit algorithm. When faced with multiple possible personalized strategy options, the algorithm can try new strategies to discover potential better solutions (exploration) while not excessively abandoning proven effective strategies (utilization), thereby ensuring the stability of the overall service quality and improving efficiency in the process of continuously optimizing personalized services.

[0196] To enhance the robustness of the system, the module integrates contrastive learning technology. Through self-supervised learning of user behavior data, it can automatically discover the intrinsic structure and similarity in the data, thereby generating a more stable and reliable user representation. For example, when processing user browsing data, contrastive learning can identify similar patterns and difference features between different browsing behaviors. Even in the presence of certain noise or incomplete data, it can accurately extract key information about users, providing a solid foundation for subsequent personalized services.

[0197] At the same time, the adversarial training mechanism is introduced to allow the model to continuously learn and optimize in the process of confronting potential noise data, thereby improving its resistance to noise data. For example, when faced with abnormal user behavior data (which may be caused by misoperation, system failure, or malicious attack, etc.), the model can effectively filter out these noise interferences through the feature recognition capabilities learned through adversarial training, maintain accurate judgment of the user's true intentions and interests, and ensure the stability and reliability of personalized services.

[0198] In order to make the effects of personalized services explainable and allow users and developers to clearly understand the basis and process of personalized decisions, the module integrates technologies such as attention visualization and decision trees. Through attention visualization technology, it is possible to intuitively show the degree of attention the system pays to different features when processing user information. For example, when generating personalized recommendations, it is clear at a glance which user features have a key impact on the recommendation results.

[0199] At the same time, with the help of SHAP (SHapley Additive exPlanations) value analysis, the system can accurately quantify the contribution of different features to personalized decisions, and deeply analyze the role of each feature in the final decision. This not only helps users understand why they get specific personalized services, but also provides developers with a strong basis for optimizing personalized models. By adjusting and optimizing the weights and processing methods of key features, the quality and effect of personalized services can be further improved, making the entire personalized adaptation process more transparent, controllable and efficient.

[0200] In terms of retrieval technology, the memory retrieval and reasoning module uses a comprehensive set of strategies at the retrieval technology level, aiming to achieve efficient and accurate information retrieval. A combination of vector retrieval and semantic search is adopted. With the help of the Dense Passage Retrieval (DPR) model, the query content entered by the user and the memory information stored in the system can be mapped to the same vector space. Based on this space, similarity matching operations can be completed quickly, and information related to the query can be quickly screened out, thereby improving the speed of retrieval. At the same time, a BERT-based semantic understanding model is introduced. This model can conduct in-depth semantic analysis of the search content, explore its inherent meaning, and further accurately grasp the query intent, thereby effectively improving the accuracy of the search, avoiding the search errors that may be caused by relying solely on surface keyword matching, and ensuring that the search results are more in line with actual needs.

[0201] In addition, the system also integrates the Attention Mechanism, which plays an important role in the entire retrieval process. It can dynamically adjust the weights of two different retrieval strategies, vectorized retrieval and semantic retrieval, according to different retrieval scenarios and actual needs, so that the two can better cooperate and optimize the retrieval effect.

[0202] In terms of reasoning mechanism, a multi-hop reasoning network is used. This network has the ability to perform multi-step logical deduction and can gradually reason and draw conclusions along a complex logical chain. In the specific implementation process, a graph neural network (GNN) is used to build a knowledge graph. With the help of the message passing mechanism in the knowledge graph, each node transmits information to each other and performs collaborative calculations to complete various complex reasoning tasks, providing a strong structural support and information interaction channel for the reasoning process.

[0203] At the same time, considering the uncertainty factors in actual reasoning, the module introduces an uncertainty reasoning mechanism and uses Bayesian networks to deal with probabilistic reasoning problems. Bayesian networks can scientifically calculate posterior probabilities based on prior probabilities and newly acquired evidence information, and then make reasonable inferences about uncertain situations, effectively improving the reliability of reasoning results and making them more adaptable to complex and changeable actual application scenarios.

[0204] In order to improve the retrieval efficiency in all aspects, especially to cope with the retrieval pressure of large-scale memory data, the system deploys a number of targeted technical means. This can realize a hierarchical index structure and use local sensitive hashing (LSH) technology to build a fast retrieval index. The hierarchical index structure makes the organization of information more orderly and facilitates fast location and search; while the local sensitive hashing technology uses clever algorithm design to realize fast approximate nearest neighbor search in high-dimensional vector space, further speeding up the retrieval speed, allowing the system to respond quickly when faced with massive data.

[0205] When processing large-scale memory data, a distributed retrieval architecture is used. By assigning retrieval tasks to multiple nodes for asynchronous parallel processing, each node works simultaneously, making full use of computing resources, greatly improving the overall retrieval speed and ensuring that the system can efficiently handle large-scale data retrieval needs.

[0206] In addition, the system also integrates an adaptive cache mechanism, which dynamically adjusts the storage level of hot data according to the access frequency of the data. For those frequently accessed hot data, it is stored in a more accessible level, and can be directly and quickly retrieved the next time it is retrieved, avoiding repeated searches, and further improving the efficiency of retrieval.

[0207] In terms of ensuring the interpretability of the reasoning results, the module uses corresponding mechanisms to make the reasoning process clear and the results reliable. It integrates an attention-based explanation mechanism, through which the reasoning path in the reasoning process can be tracked, clearly showing how each step of the reasoning is carried out, and can generate visual instructions to present the abstract reasoning process in an intuitive form, making it easier for users and relevant personnel to understand the logic and basis of the reasoning. At the same time, it is equipped with a confidence assessment mechanism, which will score the reliability of the reasoning results, measure the credibility of the reasoning results from a quantitative perspective, and provide an important reference for subsequent decision-making based on the reasoning results, helping decision makers to better judge the effectiveness and risk level of the results and make more scientific and reasonable decisions.

[0208] Through the above-mentioned comprehensive design and application in retrieval technology, reasoning mechanism, retrieval efficiency improvement and explainability of reasoning results, the memory retrieval and reasoning module can provide strong support for the operation of the entire system more efficiently, accurately and reliably to meet diverse application needs.

[0209] The output generation module occupies a vital position in the personalized memory system. As the last key link, it undertakes the core task of organically integrating the retrieved memory information with the current context, and then generating a coherent and personalized response. The output generation module uses a decoder network based on the Transformer architecture as the main framework, and integrates the attention mechanism and context-aware encoder on this basis. In the specific processing flow, facing the information obtained from the short-term memory and long-term memory modules, the system will use multi-head attention calculation to process it. Through the multi-head attention mechanism, it is like examining this information from multiple angles. The system can accurately capture the intrinsic relationship between information and their connection with historical interaction content, thereby ensuring that the final output content is highly consistent with the past historical interaction situation, so that the generated response can fit the user's previous interaction context, giving people a coherent and natural feeling.

[0210] The module also integrates conditional text generation technology, which uses control codes to flexibly adjust the tone, style, and personalized features of the output content. The system will fully rely on user portrait information, because user portraits comprehensively depict key elements such as user expression habits and preferences, and then dynamically adjust the corresponding generation parameters. For example, for users who prefer formal and rigorous styles, the system will guide the generation of content that conforms to this style through control codes; and for users who are accustomed to relaxed and lively expressions, the corresponding output content will be generated to ensure that the output content can meet the user's personalized needs to the greatest extent, and enhance the user's recognition and acceptance of the output content.

[0211] In order to further improve the quality of output, the module introduces an output optimization mechanism based on reinforcement learning. By carefully designing a specific reward function, the system can learn from it and explore a better output strategy. This reward function is multi-dimensional, covering multiple aspects such as information accuracy, language fluency, context relevance, and personalized matching. For example, when the output content is accurate in facts, the language expression is fluent and natural, closely related to the context, and highly matches the user's personalized characteristics, the system can obtain higher reward feedback, and then continuously adjust and optimize the output strategy based on such feedback, while ensuring that the output content is accurate and fluent, it can also fully demonstrate personalized colors, achieve balanced development in all aspects, and provide users with high-quality response content.

[0212] In the actual operation stage, the module adopts a hierarchical decoding strategy. First, the overall framework of the output content is planned, just like building the main structure of a building first, clarifying the overall ideas and framework layout; then gradually refining the specific content details, so that the generated content has clear structural integrity and rigorous logical coherence from macro to micro, avoiding content confusion and illogical situations. At the same time, by integrating a memory-based sampling mechanism, the system can fully explore those high-quality expressions in historical interactions and apply them reasonably to the current output content, so that the generated responses can not only reflect the actual needs of the moment, but also integrate successful and popular expression elements from the past, thereby improving the overall expression quality.

[0213] The module is specially equipped with an output quality control unit, which ensures that the generated content meets the predetermined quality standards by setting multiple constraints and post-processing rules. This includes factual checks on the output content to avoid erroneous information that is inconsistent with the facts; grammatical normative verification to ensure that the language expression complies with grammatical rules and is smooth and correct; and matching evaluation with user personalized characteristics to reconfirm that the output content does meet the user's preferences in terms of style and tone. Through such strict quality control links, the output content presented to users can maintain a high level of quality, effectively improving the user experience.

[0214] In summary, the output generation module is committed to generating high-quality, personalized and coherent response content for the personalized memory system through this series of complete and mutually coordinated technical solutions to meet the user's interaction needs in different scenarios.

[0215] As an optional embodiment, in 103, before the target search information is personalized in combination with the personalized knowledge base of the target user and the response information matching the target user's usage habits is obtained, it can also be implemented as follows:

[0216] Obtain personal preference data input by a target user; the personal preference data at least includes: the target user's business field, personal habits, and usage preferences; extract the personal preference data through an information extraction module, and construct the extracted user usage habit data as structured data corresponding to the target user to obtain the personalized knowledge base; based on the target user's historical access data, dynamically set the update strategy of the personalized knowledge base.

[0217] For example, suppose that in an online reading platform, when the target user starts using the platform, he enters personal preference data. For example, in terms of business field, he indicates that he is engaged in the education industry, and his personal habit is to read before going to bed at night. His usage preference is to prefer reading humanities and social science books, and he likes simple typesetting style, moderate font size page presentation, etc. The information extraction module of the platform will extract these personal preference data, extract key user usage habit data such as "education industry", "reading before going to bed at night", "humanities and social science books", "simple typesetting, moderate font size", etc., and construct the structured data corresponding to the target user to form a personalized knowledge base. Based on the historical access data of the target user, if the user frequently views history and philosophy related books on the platform later, and the reading time is basically fixed at night, the platform will dynamically set the update strategy of the personalized knowledge base. For example, based on new reading preferences, the weight of more specific history and philosophy categories in the humanities and social sciences will be increased. If the user stops and looks at a new typesetting style multiple times during the reading process, the corresponding page presentation preference will also be updated. This allows the personalized knowledge base to always keep up with the user's changing usage habits. When the target retrieval information (such as recommended books and other related content) is subsequently processed, matching response information can be generated, such as recommending new books that suit their preferences.

[0218] It can be understood that, based on the above-mentioned principles and steps, in a core workflow in an embodiment of the present application, the system uses multimodal fusion technology in the information input stage, uses multiple specialized encoders to cope with different types of input information, and builds the foundation for multimodal information processing.

[0219] For text information, we select pre-trained models such as BERT or RoBERTa for encoding, converting text content into a vector representation suitable for subsequent processing; for image information, we rely on visual models such as VisionTransformer or ResNet to complete feature extraction and accurately capture the key features of the image; and when processing voice information, we use speech recognition models such as Wav2Vec or Whisper to perform corresponding operations. On this basis, we use the cross-modal attention mechanism to align and fuse the features extracted from different modalities, integrate them into a unified feature representation, eliminate modal differences, and provide coherent and integrated input data for the subsequent deep understanding stage.

[0220] Entering the deep understanding stage, the system builds a hierarchical semantic understanding architecture to dig deep into the connotation of information. First, a large-scale language model based on Transformer (such as the GPT series) is used to perform preliminary semantic analysis on the input information, extract key entities, events and the relationship between them, and sort out the basic context of the information. Subsequently, with the help of the reasoning module enhanced by the knowledge graph, the extracted information is analyzed in association with the existing knowledge of the system. This process relies on the graph neural network (GNN) technology, and through its message passing mechanism, the relationship between entities is carefully modeled to achieve a deeper level of semantic understanding and to explore the logical connections hidden behind the information. In addition, the attention mechanism is also used in this process to highlight important information, avoid key content being ignored in complex information processing, ensure that the core content can be accurately grasped, and lay a solid foundation for the further use of information.

[0221] In the short-term memory storage stage, the system introduces an improved neural graph storage network to manage information storage. A storage architecture based on a graph structure is adopted, in which each node corresponds to an information unit, and the edges between nodes are used to reflect the relationship between information, thus building an information storage network with a logical structure. During storage operations, the idea of ​​differential neural computers (DNCs) is incorporated, and differentiable read and write operations are used to flexibly manage memory content to achieve orderly storage and call of information. At the same time, an attention-based indexing mechanism is introduced to achieve fast information retrieval through vector similarity calculation, thereby improving information search efficiency. In order to avoid information redundancy and forgetting problems, the system integrates a memory update strategy based on importance scoring, and uses a reinforcement learning algorithm to dynamically adjust the storage strategy according to actual conditions to ensure that the information stored in short-term memory is always valuable and in line with demand.

[0222] These three stages are closely connected through an end-to-end deep learning framework to form a complete information processing pipeline. The stages collaborate with each other and progress sequentially to complete the information processing process together.

[0223] In terms of overall training, the system uses optimization algorithms such as gradient descent for comprehensive training, and adopts a multi-task learning approach to optimize multiple objectives at the same time. Information understanding accuracy, storage efficiency, and retrieval speed are all within the scope of optimization considerations, improving system performance in all aspects. In addition, by introducing contrastive learning and self-supervised learning techniques, the generalization ability and robustness of the model are further improved, enabling it to better cope with information processing tasks in different scenarios and ensure the stable and efficient operation of the entire information processing workflow. These three stages are closely connected through an end-to-end deep learning framework to form a complete information processing pipeline. The system uses optimization algorithms such as gradient descent for overall training, and optimizes multiple objectives simultaneously through multi-task learning, including information understanding accuracy, storage efficiency, and retrieval speed. At the same time, by introducing contrastive learning and self-supervised learning techniques, the generalization ability and robustness of the model are improved.

[0224] Based on the above principles and steps, in another core workflow in the embodiment of the present application, this module is at the starting link of the entire workflow, mainly relying on the attention mechanism based on the Transformer architecture, and combined with the BERT-type pre-training model to carry out context encoding. First, for the input dialogue history, word segmentation and vectorization processing will be performed to convert the textual dialogue content into a vector form that the computer can better process and analyze. Then, the multi-head self-attention mechanism is used to capture the key information and semantic dependencies in the context from multiple angles, just like examining the dialogue content from different perspectives to dig out the key points and internal logical connections. At the same time, in order to accurately grasp the timing characteristics of the dialogue, a timing perception mechanism is also introduced to effectively maintain the dialogue sequence information through position encoding to avoid understanding deviations caused by disordered order, so as to ensure that the timing context of the current dialogue can be accurately understood, and provide clear and accurate basic information for the operation of subsequent modules.

[0225] After completing the context analysis, it enters the long-term memory activation module. This module uses a structure similar to the neural graph network (GNN) to present the stored long-term memory in the form of a knowledge graph, building a knowledge network with rich associations.

[0226] With the help of a specially designed similarity calculation algorithm, the system can quickly locate and activate relevant memory nodes in a large-scale long-term memory bank based on the current context information output by the context analysis module. In this process, an improved attention mechanism and sparse activation strategy are used. The improved attention mechanism can focus more accurately on key memory nodes, while the sparse activation strategy ensures that when faced with massive memory information, only those truly relevant parts are selectively activated, effectively avoiding the interference of irrelevant information, thereby achieving efficient and accurate long-term memory activation, and preparing the necessary and appropriate knowledge foundation for subsequent information retrieval and reasoning.

[0227] The information retrieval reasoning module integrates a hybrid architecture of vector retrieval and symbolic reasoning to complete complex and accurate information processing tasks.

[0228] At the vector space level, an improved nearest neighbor search algorithm (such as HNSW) is used to carry out efficient similarity matching, quickly find content that is similar to the demand in vector representation among a large amount of information, and ensure the efficiency of retrieval; at the symbol level, a rule-based reasoning engine is relied upon to process structured knowledge, and rigorous reasoning operations are performed according to established rules and logic to ensure the accuracy of reasoning. These two levels are not independent of each other, but cooperate and work together through a carefully designed fusion mechanism, so that the entire retrieval and reasoning process can not only quickly obtain relevant information, but also make reasonable deductions based on rules to obtain reliable results. In addition, in order to further enhance the credibility of the results, a confidence assessment mechanism is introduced to score the reliability of the results obtained by retrieval and reasoning, screen out more trustworthy output results, and provide strong support for the final response of the system.

[0229] The three modules work closely together through a specially designed coordination mechanism to form an organic whole. The output of the context analysis module directly determines the scope and intensity of long-term memory activation, that is, it points out the direction and limits the scope of long-term memory activation; and the activated long-term memory is like a "knowledge treasure house", providing essential knowledge support for subsequent information retrieval and reasoning links, helping it to complete retrieval and reasoning tasks more accurately.

[0230] The whole process thus forms a closed-loop information processing chain, with each link interconnected and linked to each other, ensuring the consistency and accuracy of the system response, making the output information consistent with the dialogue logic and trustworthy. At the same time, it is worth mentioning that each module is designed with the concept of explainability. Whether it is the capture of key information in context analysis, the selection of nodes when long-term memory is activated, or the operation and result judgment in the information retrieval and reasoning process, it is easy to track and deeply understand, which is convenient for subsequent optimization of the system's decision-making process, so that the entire system can continuously improve performance and better serve information processing needs.

[0231] Based on the above principles and steps, in another core workflow in the embodiment of the present application, the memory integration and update module plays a key basic role in the entire information processing process. It uses a hybrid architecture of a hierarchical attention network (HierarchicalAttention Network) and a dynamic memory network (Dynamic Memory Network) to carry out its work.

[0232] First, the module uses a multi-head self-attention mechanism to calculate the weights of new and old information. It is like looking at this information from multiple angles, assigning corresponding weights to different information fragments according to their respective characteristics and importance, and through this importance scoring method, clarifying the retention priority of different information fragments in memory. On this basis, the graph neural network (GNN) is further used to construct an association network between information. Each piece of information is like a node in the network. Through the message passing mechanism, the nodes communicate and transmit information with each other, thereby updating the representation of the nodes, allowing the association between information to be dynamically reflected, and finally realizing the dynamic integration of memory, so that the memory content can be continuously optimized and adjusted according to the importance and mutual relationship of the information, forming a more logical and valuable knowledge system.

[0233] After completing the memory integration and update, the forgetting mechanism optimization module begins to play a role. Its core is to adopt an improved version of the LSTM (Long Short-Term Memory) network and introduce a forgetting gate control mechanism based on time decay.

[0234] The system will consider the characteristics of multiple dimensions such as frequency of use, time interval, and information importance to build an adaptive forgetting strategy. Through the reinforcement learning algorithm, the forgetting threshold is dynamically adjusted according to the actual information usage and various characteristics to determine which information should be forgotten and which is worth keeping. At the same time, the comparative learning method is used to differentiate between information of different values. For high-value information, the emphasis is on maintaining its representation quality so that it can exist stably and accurately in memory; for low-value information, appropriate compression or direct deletion is performed to avoid useless information occupying too much memory resources, thereby optimizing the entire memory system and ensuring that the memory content always maintains a high degree of effectiveness and value.

[0235] After the memory is processed by the previous two modules, the personalized adaptation module uses the MAML (Model-Agnostic Meta-Learning) algorithm based on the meta-learning framework to achieve rapid adaptation to meet the personalized needs of different users.

[0236] The personalized adaptation module first collects a large amount of user interaction data, extracts key elements that reflect user characteristics and preferences, and then establishes a personalized user portrait vector. This vector is like a "personality label" for the user in the system, which comprehensively and accurately depicts the user's uniqueness. Subsequently, the Transformer encoder is used to model user preferences more deeply, converting abstract preferences into a form that computers can better process and analyze. In addition, the system also integrates transfer learning technology. With the help of feature transfer and model fine-tuning, the common features and knowledge learned from existing data are flexibly applied to personalized services for different users, and adaptation adjustments are made according to the specific circumstances of each user, so as to truly provide different users with services that meet their personalized needs and improve user experience.

[0237] There is a close and orderly collaborative relationship between these three modules, which together form a complete and coherent workflow. The memory integration and update module lays a good memory foundation for subsequent operations. The memory content processed by it is more organized and valuable, providing a clear target for the forgetting mechanism optimization module; the forgetting mechanism optimization module screens and optimizes the integrated memory according to the established strategy to ensure the efficiency of the memory system, and at the same time prepares high-quality, redundant memory information for the personalized adaptation module; and the personalized adaptation module, based on the optimized memory and combined with the personalized needs of users, realizes accurate personalized services, further improving the applicability and effectiveness of the entire system in practical applications.

[0238] In summary, this workflow, through the unique technical application of each module and the coordination between each other, has formed a complete information processing system from memory integration, forgetting optimization to personalized adaptation, which effectively improves the system's capabilities and level in memory management and personalized services.

[0239] Further optionally, the system uses the Retrieval-Augmented Generation (RAG) architecture, combined with the Attention Mechanism and the context-aware sequence generation model to achieve the task of response generation.

[0240] In the memory retrieval stage, the work is mainly carried out based on the dual encoder architecture. Among them, the query encoder is responsible for processing the information input by the current user and the related contextual content, and converting it into a vector form that is convenient for subsequent calculation and comparison; while the document encoder encodes the stored memory content. After that, the degree of association between the two is measured by vector similarity calculation (such as the commonly used cosine similarity), so as to quickly locate the memory fragments that are most relevant to the current needs. In this process, a sparse-dense hybrid retrieval strategy is adopted. The advantage of this strategy is that it not only guarantees the efficiency of retrieval, allowing the system to quickly screen out potentially relevant parts from a large number of memory contents, but also fully considers semantic relevance to ensure that the retrieved memory fragments are indeed consistent with the user input and context at the semantic level, laying the foundation for the subsequent generation of high-quality responses.

[0241] After completing the memory retrieval, the stage of deeply integrating the retrieved memory information with the current context begins. This link uses the cross-attention mechanism to achieve this goal, and adopts a multi-head attention structure. The multi-head attention structure is like multiple "observation perspectives", which allows the model to focus on information features at different levels at the same time, and explore the intrinsic connections between information from multiple angles, thereby achieving more fine-grained information integration, so that the memory information and the current context can be more closely and comprehensively integrated. At the same time, in order to better dynamically control the importance of memory information and context according to actual conditions, a context-aware gating mechanism is also introduced. This mechanism will flexibly adjust the weights of the two according to the specific interaction scenarios and information characteristics, highlight key information, make the fusion process more scientific and reasonable, and further improve the fusion effect.

[0242] Finally, we come to the response generation stage, which uses an improved Transformer decoder architecture and integrates a memory-enhanced self-attention layer. The system generates response content word by word through auto-regressive decoding, generates each word in a certain order, and gradually builds a complete response sentence. In the generation process, a decoding strategy based on nucleus sampling is used. This strategy can cleverly find a balance between the diversity and coherence of the response, avoiding the generated content from being too single and stereotyped, while ensuring that the sentences are smooth and logically coherent, in line with normal language expression habits. In addition, in order to make the generated content more in line with the user's personalized needs, a post-processing mechanism is introduced to make the generated response content consistent with the user's historical interaction style in style, enhance the user's familiarity and recognition of the response content, and comprehensively improve the quality of the response and user experience.

[0243] In summary, through the orderly collaboration of these key links, the system achieves the generation of high-quality response content based on retrieval of memory and integration of current context, providing users with responses that are accurate, coherent and consistent with their interaction style.

[0244] Combining the above steps, an overall example of the embodiment of the present application is as follows: When the user inputs a meeting record information by voice, the system starts to operate. In the information input and multimodal processing stage, the multimodal information perception module captures the voice information, the voice processing unit first converts the audio signal into a digital signal, and after noise reduction and audio enhancement processing, the voice recognition engine converts it into text data, and the multimodal fusion device integrates the intonation and tone of the text and the original audio to form a unified information representation. Entering the information understanding and extraction stage, the information understanding module deeply analyzes the fused meeting records, extracts key elements such as meeting time, location, participants, main topics, and resolutions, and the sentiment analysis module identifies the speaker's tone and attitude to grasp the focus and urgency. The information extractor organizes structured information according to the preset template to facilitate subsequent operations. The following is the memory storage processing stage, where the information to be processed immediately is stored in the short-term memory module, and the long-term value information such as detailed records and resolutions is stored in the long-term memory module. Timestamps, importance tags and association tags are added during storage to build an information association network. Then, in the context analysis and memory retrieval stage, the system analyzes the context of the meeting records, activates relevant historical memories, retrieves similar or related historical meeting records, and actively retrieves to-do items and schedules to avoid conflicts between new meeting arrangements and existing plans. In the memory update and maintenance stage, the memory integration module associates new meeting information with historical records and updates the information network. The forgetting mechanism adjusts the storage strategy based on timeliness, importance, and frequency of use, and handles completed or expired information accordingly. The information classification module updates the index structure to ensure retrieval efficiency. In the control and personalized adaptation stage, the central coordination module monitors each process to ensure coordinated operation. The personalized adaptation module learns the user's meeting management habits and updates the user portrait, covering personalized features such as time management and task priority setting, and optimizes subsequent information processing. Finally, in the output generation stage, the system generates personalized responses, such as meeting minutes organization, to-do reminders, related suggestions, etc., and selects the display form according to user preferences. It also provides an interactive confirmation mechanism for easy modification and supplementation.

[0245] In addition, in actual applications, the system can cope with a variety of special situations. It actively prompts for supplementation when input information is incomplete; activates the priority processing mechanism in case of emergencies; provides solution suggestions when information conflicts; and establishes specific pattern recognition and processing procedures for periodic meetings. The entire system adopts a "hierarchical memory management architecture" to dynamically adjust storage locations and priorities in short-term and long-term memory levels based on information importance, timeliness and relevance, and uses a "context-aware memory fusion mechanism" to analyze scenarios in real time, activate related historical memories and intelligently integrate them, improve context understanding and interactive intelligence, simulate the human memory association process, and more efficiently serve user meeting management needs.

[0246] In the technical solution of this application, the memory capacity of the large language model in the interaction process is improved, the information forgetting rate is reduced, the information retrieval efficiency is improved, the effective classification and organization of personalized information is achieved, and the user experience is improved. At the same time, the embodiment of this application can maintain a high degree of coherence and personalized characteristics in long-term interactions through improved context understanding capabilities and personalized adaptation mechanisms, thereby greatly improving user experience and interaction quality.

[0247] In another embodiment of the present application, a memory retrieval device based on a large language model is also provided. Figure 2 The device comprises the following units:

[0248] An interaction unit is configured to respond to an interaction instruction of a target user, perform context analysis on the interaction instruction, and obtain a real-time interaction intention of the target user;

[0249] A retrieval unit is configured to perform memory retrieval on a short-term memory module and a long-term memory module based on the real-time interaction intention to obtain target retrieval information; wherein the short-term memory module and the long-term memory module support bidirectional transmission of stored information; the short-term memory module is used to temporarily store the real-time interaction information and real-time retrieval information with the target user within a preset period of time; the long-term memory module is used to store key information in the real-time interaction information and the real-time retrieval information that complies with the preset dynamic transfer strategy;

[0250] The generating unit is configured to perform personalized processing on the target search information in combination with the personalized knowledge base of the target user, and obtain response information matching the usage habits of the target user to respond to the interactive instruction.

[0251] Further optionally, the storage unit, before the retrieval unit performs memory retrieval on the short-term memory module and the long-term memory module based on the real-time interaction intention to obtain the target retrieval information, is configured to: collect multimodal information of the target user; the multimodal information at least includes: image information, audio information, and text information; extract information to be stored from the multimodal information; the information to be stored at least includes: key content information associated with the target user in the multimodal information; the information to be stored is extracted based on the information extraction model using a multi-task learning mode; the information to be stored is stored in the short-term memory module according to the importance score and the timeliness score;

[0252] The key information stored in the short-term memory module is transferred to the long-term memory module according to the dynamic transfer strategy; the dynamic transfer strategy is dynamically adjusted using a reinforcement learning algorithm; wherein the storage mechanism in the short-term memory module is temporary storage that depends on the scoring priority sorting method; and the storage mechanism in the long-term memory module is classified compressed storage based on a knowledge graph structure.

[0253] Further optionally, the storage unit stores the information to be stored in the short-term memory module according to the importance score and the timeliness score, and is configured to: use a short-term memory model to determine the information importance and timeliness of the information to be stored to obtain a real-time importance score of the information to be stored in the current interactive session; prioritize the information to be stored based on the real-time importance score to obtain a real-time sorting result; adaptively configure the dynamic storage space corresponding to the information to be stored in the short-term memory module based on the real-time sorting result; and store the information to be stored in the corresponding dynamic storage space in the short-term memory module as short-term storage information.

[0254] Further optionally, the corresponding dynamic storage space in the short-term memory module belongs to a dynamic buffer zone; and the short-term memory module is provided with an interactive window mechanism for maintaining and managing the dynamic buffer zone.

[0255] Further optionally, the storage unit transfers the key information stored in the short-term memory module to the long-term memory module according to the dynamic transfer strategy, and is configured to: establish a dynamic priority queue based on the real-time sorting result through a memory integration model; obtain short-term storage information in the dynamic priority queue whose priority meets the transfer conditions indicated in the dynamic transfer strategy according to the memory update cycle or update event indicated in the dynamic transfer strategy; and transfer the short-term storage information that meets the transfer conditions from the short-term memory module to the long-term memory module.

[0256] Further optionally, the storage unit transfers the short-term storage information that meets the transfer condition from the short-term memory module to the long-term memory module, and is configured to: receive the short-term storage information that meets the transfer condition from the short-term memory module; embed the received short-term storage information into the long-term memory map related to the target user through the long-term memory model, and based on the embedding result, store the received short-term storage information in the corresponding long-term storage space in the vector database as long-term storage information.

[0257] Further optionally, the storage unit embeds the received short-term storage information into the long-term memory graph related to the target user through the long-term memory model, and stores the received short-term storage information in the corresponding long-term storage space in the vector database based on the embedding result as the long-term storage information, and is configured to: encode the received short-term storage information to obtain the initial information code; compress the information dimension of the initial information code through knowledge distillation processing to obtain the optimized information dimension corresponding to the initial information code; establish the correlation between the initial information codes according to the optimized information dimension; determine the graph attribute corresponding to the initial information code in the long-term memory graph according to the correlation and the optimized information dimension; wherein the graph attribute includes at least: the dimension to which the information belongs and the information position; based on the dimension to which the information belongs and the information position, store the initial information code in the corresponding long-term storage space in the vector database.

[0258] Further optionally, the storage unit, after storing the information to be stored in the short-term memory module according to the importance score and the timeliness score, or after transferring the key information stored in the short-term memory module to the long-term memory module according to the dynamic transfer strategy, is also configured to: respectively configure corresponding time weights for the short-term storage information stored in the short-term memory module and the long-term storage information stored in the long-term memory module; dynamically adjust the time weight through an exponential decay function; wherein the exponential decay function is constructed based on a dynamic forgetting algorithm of time decay, and the dynamic forgetting algorithm is obtained according to the setting of the Ebbinghaus forgetting curve; monitor the access frequency, recent usage time and correlation with other information of the short-term storage information and the long-term storage information; based on the monitoring results, use a forgetting assessment model to sort the importance of the short-term storage information and the long-term storage information; perform contextual correlation analysis on the short-term storage information and the long-term storage information to obtain the interactive topic correlation of the short-term storage information and the long-term storage information; determine the short-term storage information and / or long-term storage information to be deleted according to the importance sorting result, the interactive topic correlation and the time weight.

[0259] Further optionally, the storage unit determines the short-term storage information and / or long-term storage information to be deleted based on the importance sorting result, the interaction topic relevance and the time weight, and is also configured to: set the short-term storage information and / or long-term storage information whose relevance to the interaction topic within a set time period is lower than a set relevance threshold as the short-term storage information and / or long-term storage information to be deleted; or, set the short-term storage information and / or long-term storage information whose importance ranking is lower than the set ranking threshold as the short-term storage information and / or long-term storage information to be deleted; or, set the short-term storage information and / or long-term storage information whose time weight is lower than the set time weight threshold as the short-term storage information and / or long-term storage information to be deleted.

[0260] Further optionally, the retrieval unit performs memory retrieval on the short-term memory module and the long-term memory module based on the real-time interaction intention to obtain target retrieval information, and is configured to: perform deep semantic analysis on the real-time interaction intention to obtain the real-time retrieval target corresponding to the target user; retrieve the first candidate information associated with the real-time retrieval target from the short-term memory module; retrieve the second candidate information associated with the real-time retrieval target and / or the first candidate information from the long-term memory module; map the real-time retrieval target, the first candidate context information, and the second candidate information into the same vector space to obtain their respective corresponding projection vectors; calculate the first similarity between the first projection vector corresponding to the first candidate information and the reference projection vector corresponding to the real-time retrieval target; calculate the second similarity between the second projection vector corresponding to the second candidate information and the reference projection vector; generate the target retrieval information based on the first similarity and the second similarity; the target retrieval information is obtained based on inference of a multi-hop Bayesian inference network; wherein the corresponding proportions of the first candidate information and the second candidate information in the target retrieval information are dynamically configured using an attention mechanism; the attention mechanism is associated with a retrieval strategy pre-configured in the retrieval scenario.

[0261] Further optionally, the personalization unit, which combines the personalized knowledge base of the target user to personalize the target retrieval information to obtain response information that matches the target user's usage habits, is configured to: perform multi-head attention calculation on the target retrieval information; based on the calculation results, evaluate the consistency between the target retrieval information and the historical interaction information stored in the short-term memory module and the long-term memory module; if the consistency evaluation passes, generate initial response information based on the target retrieval information; use a control code to perform conditional text generation processing on the initial response information to optimize the personalized expression of the initial response information; wherein the personalized expression includes at least: tone, style, and idioms; generate dynamic image adjustment parameters based on the image information in the historical interaction information, and optimize the image information in the initial response information through the dynamic image adjustment parameters to obtain response information that matches the target user's usage habits.

[0262] Further optionally, the personalization unit, in combination with the personalized knowledge base of the target user, personalizes the target retrieval information to obtain response information that matches the target user's usage habits, and is also configured to: obtain personal preference data input by the target user; the personal preference data at least includes: the target user's business field, personal habits, and usage preferences; extract the personal preference data through the information extraction module, and construct the extracted user usage habit data as structured data corresponding to the target user to obtain the personalized knowledge base; based on the target user's historical access data, dynamically set the update strategy of the personalized knowledge base.

[0263] Further optionally, the storage information of the short-term memory module and the long-term memory module carries a memory code; the memory code is used to identify the information content type of the corresponding storage information.

[0264] The system can implement various steps in the above method embodiment, which will not be expanded here.

[0265] In the embodiment of the present application, a memory retrieval device based on a large language model is adopted.

[0266] See also Figure 3 , Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present application. Figure 3 As shown, an embodiment of the present application provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520, and when the processor 520 executes the computer program 511, a memory retrieval method based on a large language model is implemented.

[0267] See also Figure 4 , Figure 4 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present application. Figure 4 As shown, this embodiment provides a computer-readable storage medium 600 on which a computer program 611 is stored. When the computer program 611 is executed by a processor, a memory retrieval method based on a large language model is implemented.

[0268] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0269] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0270] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0271] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A memory retrieval method based on a large language model, characterized in that: The method comprises: In response to an interaction instruction issued by a target user, context parsing is performed on the interaction instruction to obtain a real-time interaction intention of the target user; Based on the real-time interaction intention, the short-term memory module and the long-term memory module are memory-searched to obtain target retrieval information; wherein the short-term memory module and the long-term memory module support bidirectional transmission of stored information; the short-term memory module is used to temporarily store the real-time interaction information and real-time retrieval information between the target user within a preset period of time; the long-term memory module is used to store the key information in the real-time interaction information and the real-time retrieval information that conforms to the preset dynamic transfer strategy; The target search information is personalized in combination with the personalized knowledge base of the target user to obtain response information that matches the target user's usage habits to respond to the interactive instruction.

2. The memory retrieval method based on a large language model according to claim 1, characterized in that: Before performing memory retrieval on the short-term memory module and the long-term memory module based on the real-time interaction intention to obtain target retrieval information, the method further includes: Collecting multimodal information of the target user; the multimodal information includes at least: image information, audio information, and text information; Extracting information to be stored from the multimodal information; the information to be stored includes at least: key content information associated with the target user in the multimodal information; the information to be stored is extracted based on the information extraction model using a multi-task learning mode; Storing the information to be stored in the short-term memory module according to the importance score and the timeliness score; The key information stored in the short-term memory module is transferred to the long-term memory module according to a dynamic transfer strategy; the dynamic transfer strategy is dynamically adjusted using a reinforcement learning algorithm; Among them, the storage mechanism in the short-term memory module is temporary storage that depends on the scoring priority sorting method; the storage mechanism in the long-term memory module is classified compressed storage based on the knowledge graph structure.

3. The memory retrieval method based on a large language model according to claim 2, characterized in that: The storing of the information to be stored in the short-term memory module according to the importance score and the timeliness score includes: Using a short-term memory model to determine the information importance and timeliness of the information to be stored, so as to obtain a real-time importance score of the information to be stored in the current interactive session; Prioritizing the information to be stored based on the real-time importance score to obtain a real-time ranking result; Adaptively configuring the dynamic storage space corresponding to the information to be stored in the short-term memory module based on the real-time sorting result; Storing the information to be stored in the corresponding dynamic storage space in the short-term memory module as short-term storage information; Wherein, the corresponding dynamic storage space in the short-term memory module belongs to the dynamic buffer; The short-term memory module is provided with an interactive window mechanism for maintaining and managing the dynamic buffer.

4. The memory retrieval method based on a large language model according to claim 3, characterized in that: The transferring of the key information stored in the short-term memory module to the long-term memory module according to the dynamic transfer strategy includes: Establishing a dynamic priority queue based on the real-time sorting result through a memory integration model; According to the memory update cycle or update event indicated in the dynamic transfer strategy, obtain the short-term storage information of the dynamic priority queue whose priority meets the transfer condition indicated in the dynamic transfer strategy; The short-term storage information satisfying the transfer condition is transferred from the short-term memory module to the long-term memory module.

5. The memory retrieval method based on a large language model according to claim 4, characterized in that: The transferring of the short-term storage information satisfying the transfer condition from the short-term memory module to the long-term memory module comprises: receiving short-term storage information satisfying the transfer condition from the short-term memory module; Through the long-term memory model, the received short-term storage information is embedded into the long-term memory map related to the target user, and based on the embedding result, the received short-term storage information is stored in the corresponding long-term storage space in the vector database as long-term storage information.

6. The memory retrieval method based on a large language model according to claim 5, characterized in that: The method embeds the received short-term storage information into the long-term memory map related to the target user through the long-term memory model, and stores the received short-term storage information into the corresponding long-term storage space in the vector database based on the embedding result as the long-term storage information, including: Encoding the received short-term storage information to obtain an initial information code; By means of knowledge distillation, the information dimension of the initial information encoding is compressed to obtain an optimized information dimension corresponding to the initial information encoding; According to the optimized information dimension, establishing the correlation between the initial information codes; According to the relevance and the optimized information dimension, determine the corresponding graph attribute of the initial information encoding in the long-term memory graph; wherein the graph attribute includes at least: the dimension to which the information belongs and the position of the information; Based on the dimension to which the information belongs and the information position, the initial information encoding is stored in the corresponding long-term storage space in the vector database.

7. The memory retrieval method based on a large language model according to claim 2, characterized in that: After storing the information to be stored in the short-term memory module according to the importance score and the timeliness score, or transferring the key information stored in the short-term memory module to the long-term memory module according to the dynamic transfer strategy, the method further includes: Configure corresponding time weights for the short-term storage information stored in the short-term memory module and the long-term storage information stored in the long-term memory module; Dynamically adjusting the time weight through an exponential decay function; wherein the exponential decay function is constructed based on a dynamic forgetting algorithm of time decay, and the dynamic forgetting algorithm is obtained according to the setting of the Ebbinghaus forgetting curve; Monitor the access frequency, recent usage time and relevance of short-term and long-term storage information to other information; Based on the monitoring results, the forgetting assessment model is used to rank the importance of short-term storage information and long-term storage information; Performing context relevance analysis on the short-term storage information and the long-term storage information to obtain the interactive topic relevance of the short-term storage information and the long-term storage information; Determine the short-term storage information and / or long-term storage information to be deleted according to the importance ranking result, the interaction topic relevance and the time weight; The step of determining the short-term storage information and / or long-term storage information to be deleted according to the importance ranking result, the interaction topic relevance, and the time weight includes: Setting the short-term storage information and / or long-term storage information whose relevance to the interactive topic within the set time period is lower than the set relevance threshold as the short-term storage information and / or long-term storage information to be deleted; or Setting the short-term storage information and / or long-term storage information whose importance ranking is lower than the set ranking threshold as the short-term storage information and / or long-term storage information to be deleted; or The short-term storage information and / or long-term storage information whose time weight is lower than the set time weight threshold is set as the short-term storage information and / or long-term storage information to be deleted.

8. The memory retrieval method based on a large language model according to claim 1, characterized in that: The memory retrieval of the short-term memory module and the long-term memory module based on the real-time interaction intention to obtain target retrieval information includes: Performing deep semantic analysis on the real-time interaction intention to obtain the real-time retrieval target corresponding to the target user; Retrieving first candidate information associated with the real-time retrieval target from the short-term memory module; Retrieving second candidate information associated with the real-time retrieval target and / or the first candidate information from the long-term memory module; Mapping the real-time retrieval target, the first candidate context information, and the second candidate information into the same vector space to obtain respective corresponding projection vectors; Calculating a first similarity between a first projection vector corresponding to the first candidate information and a reference projection vector corresponding to the real-time retrieval target; Calculating a second similarity between a second projection vector corresponding to the second candidate information and the reference projection vector; The target retrieval information is generated based on the first similarity and the second similarity; the target retrieval information is obtained based on multi-hop Bayesian inference network reasoning; Among them, the corresponding proportions of the first candidate information and the second candidate information in the target retrieval information are dynamically configured using an attention mechanism; and the attention mechanism is associated with a retrieval strategy pre-configured in a retrieval scenario.

9. The memory retrieval method based on a large language model according to claim 1, characterized in that: The target search information is personalized in combination with the personalized knowledge base of the target user to obtain response information that matches the target user's usage habits, including: Performing multi-head attention calculation on the target retrieval information; Based on the calculation results, evaluating the consistency between the target retrieval information and the historical interaction information stored in the short-term memory module and the long-term memory module; If the consistency assessment passes, generating initial response information based on the target retrieval information; The control code is used to perform conditional text generation processing on the initial response information to optimize the personalized expression of the initial response information; wherein the personalized expression includes at least: tone, style, and idiomatic expressions; Dynamic image adjustment parameters are generated based on the image information in the historical interaction information, and the image information in the initial response information is optimized by the dynamic image adjustment parameters to obtain response information that matches the target user's usage habits.

10. A memory retrieval device based on a large language model, characterized in that: The device comprises the following units, wherein: An interaction unit is configured to respond to an interaction instruction of a target user, perform context analysis on the interaction instruction, and obtain a real-time interaction intention of the target user; A retrieval unit is configured to perform memory retrieval on a short-term memory module and a long-term memory module based on the real-time interaction intention to obtain target retrieval information; wherein the short-term memory module and the long-term memory module support bidirectional transmission of stored information; the short-term memory module is used to temporarily store the real-time interaction information and real-time retrieval information with the target user within a preset period of time; the long-term memory module is used to store key information in the real-time interaction information and the real-time retrieval information that complies with the preset dynamic transfer strategy; The generating unit is configured to perform personalized processing on the target search information in combination with the personalized knowledge base of the target user, and obtain response information matching the usage habits of the target user to respond to the interactive instruction.

Images