Long and short-term memory collaborative adaptation fine-tuning RAG fusion large language model system

Abstract

The invention discloses a long-short-term memory collaborative adaptation fine-tuning RAG fusion large language model system, which relates to the technical field of large language models and comprises a short-term memory module and a long-term memory fine-tuning processing module. Wherein the short-term memory module comprises a text data partitioning unit, an embedded vector unit, a short-term memory buffer area and a vector index unit; and the long-term memory fine tuning processing module comprises a self-adaptive routing LoRA activation classifier, a plurality of LoRA modules, a preference loss calculation unit, a feature extractor and a multi-stage processing architecture. According to the method, the defects of an existing large language model in dimensions of memory architecture layering, memory updating causal traceability, online and offline learning collaboration and the like are relieved.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, and in particular to a long short-term memory collaborative adaptive fine-tuning RAG fusion large language model system. Background Technology

[0002] With the breakthrough development of large language model technology, it has demonstrated outstanding capabilities in natural language understanding, dialogue generation and multi-task reasoning, and has been widely applied in intelligent scenarios such as customer service, education and healthcare.

[0003] However, existing large language models still face two major challenges in practical applications: First, traditional memory management mechanisms use fixed-length context windows, lacking the ability to dynamically store long-term user preferences and historical interaction information, resulting in a "one-size-fits-all" memory storage model that fails to prioritize important information. Second, existing fine-tuning methods mostly rely on offline batch training, making it difficult to capture user behavior characteristics and scene changes in real time, leading to personalized service delays and memory redundancy issues. How to enable models to maintain long-term memory and effectively learn new knowledge during continuous interaction has become a key challenge. Large Language Models (LLMs) have shortcomings in dimensions such as hierarchical memory architecture (e.g., short-term emotion caching and long-term personality profile hierarchical storage in emotional companionship), causal traceability of memory updates (e.g., logical backtracking of attack chains in military decision-making), and online-offline learning collaboration (e.g., instant fine-tuning for sudden road conditions and periodic iteration of driving habits in autonomous driving).

[0004] Specifically, the performance of LLM-equipped agents in complex environments largely depends on the effectiveness of their information acquisition and memory management mechanisms. Existing research mainly revolves around three core information sources: inside-trial information, cross-trial information, and external knowledge. Although these mechanisms enhance the capabilities of agents in their respective fields, they all face significant limitations and challenges in practical applications.

[0005] First, there's internally generated information, primarily referring to short-term memories or task-related contextual information generated by the agent during task execution. For example, MemoChat generates memories by analyzing historical chat logs, helping the agent better understand the context in subsequent conversations. Similarly, TiM (Task-internal Memory) generates multiple thoughts after completing a task, forming a short-term memory mechanism for quick retrieval in similar tasks. These methods typically rely on task-internal contextual information and have strong task relevance. However, internal information is highly task-specific and almost impossible to transfer to other types of tasks. Once the task scenario changes, these short-term memories become invalid. Both MemoChat and TiM have memory capacity limited by the context window size of the model (such as LLM). As task complexity increases and conversation rounds lengthen, the earliest key information may disappear.

[0006] Second, there is cross-trial information, which refers to information shared and utilized by the agent across different tasks or trials. Reflexion proposed a language-based reinforcement learning framework that allows the agent to verbally record past experiences and apply these experiences to subsequent tasks, thereby improving task execution efficiency. Retroformer further improves the Reflexion model by fine-tuning it to enable the agent to extract cross-task information from past trials more effectively. These methods significantly improve the agent's generalization ability and task execution efficiency through the sharing and utilization of cross-task information.

[0007] Thirdly, external knowledge refers to information acquired by an agent from external resources (such as knowledge bases, web crawlers, etc.). This type of method enhances the breadth and depth of the agent's knowledge by combining external knowledge with the task context. For example, some agents access external knowledge bases or use web crawling techniques to extract relevant information from web pages to assist in task execution. The introduction of external knowledge enables agents to handle more complex tasks and make more accurate decisions in the absence of internal information. However, calling APIs, crawling web pages, or searching large knowledge bases are time-consuming operations. This significantly increases the agent's decision-making latency, making it difficult for them to handle tasks requiring rapid, real-time responses.

[0008] In summary, all single information acquisition mechanisms have significant performance bottlenecks. Summary of the Invention

[0009] To address the aforementioned technical problems in existing technologies, this invention provides a Long Short-Term Memory (LSTM) collaborative adaptive fine-tuning RAG fusion large language model system. The technical solution is as follows: On one hand, a system for adaptive fine-tuning RAG fusion large language model with long short-term memory collaboration is provided, including: a short-term memory module and a long-term memory fine-tuning processing module; wherein, the short-term memory module includes a text data segmentation unit, an embedding vector unit, a short-term memory buffer, and a vector indexing unit; the long-term memory fine-tuning processing module includes an adaptive routing LoRA activation classifier, multiple LoRA modules, a preference loss calculation unit, a feature extractor, and a multi-level processing architecture; the text segmentation unit is used to segment the input text data into multiple text data blocks; the embedding vector unit is used to vectorize the multiple text data blocks to obtain multiple embedding vectors; the short-term memory buffer is used to store the multiple embedding vectors; the vector indexing unit is used to index the short-term memory based on a preset search method. The system retrieves embedded vectors within a buffer and deletes those with a retrieval frequency below a first preset threshold, sending those with a retrieval frequency above a second preset threshold to the long-term memory fine-tuning processing module. Each LoRA module corresponds to a downstream task. The feature extractor extracts hidden features from all layers. The adaptive routing LoRA activation classifier inputs the hidden features into a dynamic multilayer perceptron during training to obtain LoRA weights and activates the corresponding LoRA module based on these weights during inference. The preference loss calculation unit learns the mapping from the data distribution to the activation states of the multiple LoRA modules during training based on a preset preference loss function. The multi-level processing architecture employs a multi-level decision-making mechanism to obtain the target prediction result during inference.

[0010] Optionally, the short-term memory module further includes a vector extraction module for extracting high-order features and extracting vectors from the plurality of embedded vectors.

[0011] Optionally, the adaptive routing LoRA activation classifier is a dynamic multilayer sensing architecture.

[0012] Optionally, the preset preference loss function in the preference loss calculation unit includes the mean squared error loss function.

[0013] Optionally, the long-term memory fine-tuning processing module uses a sparse initialization strategy during the training phase to reserve the expansion capacity of the LoRA module.

[0014] Optionally, the multi-level processing architecture includes a three-level processing architecture of hierarchical feature aggregation, cross-layer hidden layer feature enhancement, and task-aware projection.

[0015] This invention provides an adaptive fine-tuning RAG fusion large language model system with long short-term memory collaboration. Through an adaptive LoRA activation selector and task preference loss function, the model learns the mapping of data distribution to the LoRA module, effectively preventing conflicts and mutual interference between new and old tasks in parameter updates and ensuring the stability of old knowledge. A frequency filtering mechanism effectively solves the problem of context window limitations. During inference, a multi-level decision-making mechanism improves the accuracy of the final output, alleviating the shortcomings of existing large language models in areas such as hierarchical memory architecture, causal traceability of memory updates, and collaborative online and offline learning. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the structure of a Long Short-Term Memory Collaborative Adaptive Fine-Tuning RAG Fusion Large Language Model System provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the principle of a short-term memory module provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the principle of a long-term memory fine-tuning processing module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the overall working principle of a long short-term memory collaborative adaptive fine-tuning RAG fusion large language model system provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0019] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0020] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0021] Figure 1This is a schematic diagram of the structure of a Long Short-Term Memory (LSTM) collaborative adaptive fine-tuning RAG fusion large language model system provided by an embodiment of the present invention. Figure 1 As shown, it includes a short-term memory module 10 and a long-term memory fine-tuning processing module 20. Specifically, the short-term memory module 10 includes a text data segmentation unit 11, an embedding vector unit 12, a short-term memory buffer 13, and a vector indexing unit 14; the long-term memory fine-tuning processing module 20 includes an adaptive routing LoRA activation classifier 21, multiple LoRA modules 22, a preference loss calculation unit 23, a feature extractor 24, and a multi-level processing architecture 25.

[0022] Specifically, the text segmentation unit 11 is used to segment the input text data into multiple text data blocks; Embedded vector unit 12 is used to vectorize multiple text data blocks to obtain multiple embedded vectors; Short-term memory buffer 13 is used to store multiple embedding vectors; Vector indexing unit 14 is used to retrieve embedded vectors in the short-term memory buffer based on a preset search method, and to delete embedded vectors whose retrieval frequency is lower than a first preset threshold and send embedded vectors whose retrieval frequency is higher than a second preset threshold to the long-term memory fine-tuning processing module.

[0023] In this embodiment of the invention, each LoRA module 22 corresponds to a downstream task.

[0024] Specifically, feature extractor 24 is used to extract hidden layer features from all layers; An adaptive routing LoRA activation classifier 21 is used to input hidden layer features into a dynamic multilayer perceptron during the training phase to obtain LoRA weights, and to activate the corresponding LoRA module based on the LoRA weights during the inference phase. The preference loss calculation unit 23 is used to learn the mapping process from the data distribution to the activation states of multiple LoRA modules based on a preset preference loss function during the training phase; preferably, the preset preference loss function in the preference loss calculation unit includes the mean squared error loss function.

[0025] A multi-level processing architecture 25 is used to obtain target prediction results by employing a multi-level decision-making mechanism during the inference phase.

[0026] Specifically, the multi-level processing architecture 25 includes a three-level processing architecture of hierarchical feature aggregation, cross-layer hidden layer feature enhancement, and task-aware projection.

[0027] Specifically, such as Figure 1 As shown, the short-term memory module 10 also includes a vector extraction module 15, which is used to extract high-order features and vectors from multiple embedded vectors.

[0028] Figure 2 This is a schematic diagram illustrating the principle of a short-term memory module according to an embodiment of the present invention. Figure 2 As shown, this invention is inspired by human neuroscience. The short-term memory module 10 uses a RAG (Retrieval-Augmented Generation) vector model for embedding encoding and temporary storage in a buffer. Vector indexes are created for existing data content (using vector indexes to represent the original text) to facilitate subsequent retrieval and feature extraction. Unlike previous direct text storage, this invention mimics RAG for vector encoding, aiming to extract features at a higher dimension. The specific steps are as follows: (1) First, the original text data needs to be segmented into blocks, dividing the document into appropriately sized paragraphs while preserving its original meaning. For example, the text can be divided into sentences or paragraphs, rather than cutting individual sentences into multiple parts. The reason for this step is that the length of the input to the subsequent embedding model is fixed. For example, the standard Transformer encoder model based on BERT can process a maximum of 512 tokens, while OpenAI ada-002 can process longer sequences of 8191 tokens. The size of the blocks is important here. Large blocks may contain irrelevant information, resulting in a decrease in the relevance of the retrieved content to the query, and processing larger blocks will increase the consumption of computing resources. Small blocks may not contain enough information to answer the query, resulting in incomplete or context-lacking content.

[0029] (2) Next, the text that has been divided into blocks will be embedded by vector transformation through an embedding vector unit.

[0030] (3) The vectorized content is then stored as a search index. The most basic search index implementation is a flat index, which calculates the query from step 2 using a brute-force method. Efficient search indexing methods are optimized for large-scale retrieval of more than 10,000 elements, using vector indexes such as faiss, nmslib, or annoy. These indexes employ an approximate nearest neighbor method. An additional approach is to have the LLM generate a hypothetical question for each block and embed these questions as vectors. At runtime, a query search is performed on the index against this question vector (replacing the block vector in the index of this invention with the question vector). After retrieval, the original text block is sent to the LLM as context to obtain the answer. This method improves search quality due to the higher semantic similarity between the query and the hypothetical question.

[0031] (4) Regularly delete vector content with low retrieval rate based on retrieval frequency.

[0032] If the retrieval frequency is high and exceeds a certain set threshold, the data is processed and enters the long-term memory fine-tuning processing module 20. Data that has not been retrieved again for a long time is discarded.

[0033] Figure 3 This is a schematic diagram illustrating the principle of a long-term memory fine-tuning processing module according to an embodiment of the present invention. Figure 3 As shown, during the training phase, a learnable adaptive routing LoRA activation classifier is introduced at the model level, with one lightweight LoRA module trained for each downstream task. The LoRA (Low-Rank Adaptation) module is used for efficient fine-tuning of large language models, avoiding the computational cost of full-parameter training. Specifically, traditional full-parameter fine-tuning requires updating all model parameters (e.g., GPT-3 has 175 billion parameters), while the LoRA module only trains a small number of low-rank adapter parameters (typically 0.1%-1% of the original model parameters). For example, for a 7 billion parameter model, LoRA only needs to train about 100,000 to 1 million parameters, reducing memory usage by more than 90%. The LoRA module decomposes the weight matrix of a large language model into a low-rank matrix (e.g., rank r = 8-64), fixing the original model parameters during fine-tuning and only updating the low-rank matrix. This approach allows the model to learn new task features while retaining general knowledge from the pre-training phase, avoiding more severe "catastrophic forgetting." Because fewer parameters need updating, the LoRA module trains and iterates faster, reducing training time by 5-10 times under the same hardware conditions. The finely tuned LoRA adapter can be saved separately (file size typically a few MB to several hundred MB), without requiring modification to the original model structure, facilitating integration into online services and reducing deployment resource consumption. The LoRA module performs exceptionally well in online incremental learning scenarios, primarily due to its parameter efficiency, model compatibility, and flexible update mechanism.

[0034] However, LoRA modules alone are still insufficient to significantly reduce catastrophic forgetting because the distribution of newly trained data influences the distribution of previously learned data. This invention addresses this by designing a task-specific preference loss function, which drives the classifier to preferentially activate the corresponding LoRA module under a specific data distribution. This part uses mean squared error loss. The reason for designing such a preference loss function is to allow the model to better learn the distribution from the input data distribution to the corresponding LoRA module distribution, without affecting other LoRA modules. This is analogous to the activation of fixed neurons in the human brain, and the mechanism also simulates the plasticity of neural connections in the human brain. The specific implementation is as follows: (1) Dynamic routing mechanism: The adaptive routing LoRA activation classifier is a dynamic multi-layer perceptual architecture. First, for a new task, a new LoRA module is added. The model input, after being encoded using a vector similar to RAG, is fed into a hybrid structure of the underlying model and LoRA modules. A feature extractor then extracts the hidden features from all layers. These features are then input into a dynamic multilayer perceptron to obtain the LoRA weights. The purpose of this invention is to enable the dynamic multilayer perceptron to learn as many input distributions as possible and activate the corresponding LoRA modules. Therefore, this invention generates a target_mask based on the current number of LoRA modules, representing the target LoRA allocation. Assuming there are twelve downstream tasks, this translates to one LoRA module in the range [0,0,0,0,0,0,0,0,0,0,0,0,0,0]. The distribution vector is 12, where 1 represents activation and 0 represents inactivation. This invention aims to map the mathematical distribution of input features to the distribution vector of the LoRA module using the dynamic multilayer perceptron. Therefore, a preference loss function is introduced to reduce the gap between these two distributions. The preference loss function uses mean squared error loss. Due to the presence of the squared term, MSE is more sensitive to large errors between the predicted and true values, which helps the model correct large biases faster. Moreover, it is smooth and differentiable throughout the entire domain, making it very suitable for optimization using gradient descent. Gradient calculation is very direct, which is conducive to efficient updating of model parameters. In continuous learning scenarios such as regression tasks, feature reconstruction, and weight allocation, MSE loss can effectively measure the distance between the predicted and target values.

[0035] (2) Extensibility Design

[0036] The long-term memory fine-tuning module employs a sparse initialization strategy during the training phase to reserve expansion capacity for the LoRA module. Sparse initialization means that when initializing the LoRA module parameters, most weights are set to 0, with only a small number assigned non-zero values. This means that the LoRA module layer initially has very little impact on the main model, almost equivalent to an "inactive" state. Sparse initialization implies that most parameters are not yet "used," and subsequent new tasks can gradually activate and utilize this unused parameter space. Each time a new LoRA module is added, sparse initialization provides the new task with an independent parameter space, reducing catastrophic forgetting.

[0037] Specifically, such as Figure 3As shown, a multi-level decision-making mechanism is adopted in the inference stage. First, features are extracted using a feature extractor, which supports receiving all hidden layer outputs of the pre-trained model, constructing a three-level processing architecture of "hierarchical feature aggregation - cross-layer hidden layer feature enhancement - task-aware projection". This design can fully utilize semantic information at different levels: the bottom layer captures fine-grained features (such as lexical structure), the middle layer extracts syntactic relations, and the top layer abstracts high-level semantics, forming multi-scale feature representations. The features are fed into a dynamic multilayer perceptron, which learns the distribution mapping from previous training to obtain the weight allocation vector of the LoRA module. The weights are sorted, and the LoRA module corresponding to the maximum value is selected. Only the dominant LoRA module is activated for forward inference, and the final prediction result is output.

[0038] The multi-level decision-making mechanism based on fully hidden layer features provided by this invention significantly enhances the accuracy and adaptability of LoRA module selection by deeply mining the multi-scale semantic information contained in each layer of the pre-trained model, providing a better solution for continuous learning.

[0039] Figure 4 This is a schematic diagram illustrating the overall working principle of a long short-term memory collaborative adaptive fine-tuning RAG fusion large language model system according to an embodiment of the present invention. Figure 4 As shown, this invention proposes a continuous learning method for agents that rely on a hybrid long short-term memory. Neuroscience indicates that the brain performs additional encoding operations on text and its various multimodal information. Therefore, for short-term memory, this invention uses RAG embedding vectors for high-dimensional embedding and stores them in a cache. Content with low retrieval frequency in the cache is periodically deleted to ensure that the prompts embedded in the model are not too long. For some frequently retrieved high-frequency content, this invention sends it to the long-term memory module to adjust the model's parameters and perform continuous learning.

[0040] Furthermore, the human brain has a fixed number of memory-sensitive neurons responsible for corresponding tasks, and there are only 103 of them. Therefore, for the model's long-term memory, this invention also establishes multiple simulated neuron components. However, the complexity of a single neuron in a neural network is too low. This invention treats a LoRA as a memory neuron component and additionally trains an adaptive routing LoRA activation selector. By incorporating task preference loss, the system learns the mapping process from data distribution to LoRA activation states, unlike previous works which activated all LoRAs, thus avoiding the interference between different tasks. During each interaction, text information can be buffered and stored, and the long-term memory module can be trained during idle time.

[0041] As can be seen from the above description, the adaptive fine-tuning RAG fusion large language model system with long short-term memory collaboration provided by the embodiments of the present invention has the following technical effects compared with the prior art: 1. This invention, through an innovative "adaptive routing LoRA activation selector" and "task preference loss function" (mean squared error loss), enables the model to learn a precise mapping from "data distribution" to "specific LoRA modules." During training and inference, the system activates only the LoRA modules most relevant to the current task (and even only one dominant LoRA module during inference), rather than activating all LoRA modules. This simulates the brain's mechanism of activating specific clusters of neurons, effectively preventing conflicts and interference between new and old tasks in parameter updates, and ensuring the stability of old knowledge.

[0042] 2. The main body of the solution uses LoRA (Low-Rank Adaptation) modules as "memory neuron components." Compared to full-parameter fine-tuning, the number of training parameters (only 0.1%-1%) and memory usage (reduced by more than 90%) are significantly reduced, while training speed is increased by 5-10 times. During the inference phase, the "multi-level decision-making mechanism" activates only a single dominant LoRA module, greatly reducing computational overhead. In addition, the "sparse initialization" strategy ensures the system's efficient scalability when adding new LoRA modules (i.e., learning new tasks), enabling low-cost online incremental learning.

[0043] 3. This invention designs a "short-term-long-term" hybrid memory mechanism: Short-term memory (STM) adopts a RAG vector buffer and effectively solves the problem of context window limitation (excessively long prompts) through a frequency filtering mechanism (periodically deleting low-frequency and consolidating high-frequency), and simulates an efficient memory filtering process. During inference, the "multi-level decision mechanism" utilizes multi-scale semantic information from all hidden layers (all hidden layer features) of the model, enabling the routing selector (MLP) to more accurately determine the attribution of the input task, thereby activating the most suitable LoRA module and improving the accuracy of the final output.

[0044] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A long short-term memory collaborative adaptive fine-tuning RAG fusion large language model system, characterized in that, include: The system includes a short-term memory module and a long-term memory fine-tuning module. The short-term memory module comprises a text data segmentation unit, an embedding vector unit, a short-term memory buffer, and a vector indexing unit. The long-term memory fine-tuning module comprises an adaptive routing LoRA activation classifier, multiple LoRA modules, a preference loss calculation unit, a feature extractor, and a multi-level processing architecture. The text segmentation unit is used to segment the input text data into multiple text data blocks. The embedding vector unit is used to vectorize the plurality of text data blocks to obtain a plurality of embedding vectors; The short-term memory buffer is used to store the plurality of embedding vectors; The vector indexing unit is used to retrieve the embedded vectors in the short-term memory buffer based on a preset search method, and to delete embedded vectors whose retrieval frequency is lower than a first preset threshold and send embedded vectors whose retrieval frequency is higher than a second preset threshold to the long-term memory fine-tuning processing module. Each LoRA module corresponds to one downstream task; The feature extractor is used to extract hidden layer features from all layers; The adaptive routing LoRA activation classifier is used to input the hidden layer features into the dynamic multilayer perceptron during the training phase to obtain LoRA weights, and to activate the corresponding LoRA module based on the LoRA weights during the inference phase. The preference loss calculation unit is used to learn the mapping process from the data distribution to the activation state of the multiple LoRA modules based on a preset preference loss function during the training phase. The multi-level processing architecture is used to obtain target prediction results by employing a multi-level decision-making mechanism during the inference stage.

2. The system according to claim 1, characterized in that, The short-term memory module further includes a vector extraction module for extracting high-order features and vectors from the multiple embedded vectors.

3. The system according to claim 1, characterized in that, The adaptive routing LoRA activation classifier is a dynamic multilayer sensing architecture.

4. The system according to claim 1, characterized in that, The preset preference loss function in the preference loss calculation unit includes the mean squared error loss function.

5. The system according to claim 1, characterized in that, The long-term memory fine-tuning processing module uses a sparse initialization strategy during the training phase to reserve expansion capacity for the LoRA module.

6. The system according to claim 1, characterized in that, The multi-level processing architecture includes a three-level processing architecture: hierarchical feature aggregation, cross-layer hidden layer feature enhancement, and task-aware projection.

Images