Memory optimization system for large language model

Abstract

The invention discloses a memory optimization system for a large-scale language model, and the system comprises a dynamic semantic memory modeling module which is used for generating a context-aware memory vector through a bidirectional Transform, and supporting the increment to be updated to one hundred million entity records; the self-adaptive memory resource scheduling module dynamically allocates memory pool partitions based on inference load, high-frequency memory is stored in a working area, and quantitative compression is started in an archiving area; the memory attenuation and enhancement module adopts a two-factor attenuation mechanism to execute enhanced storage and regular pruning on the high-value memory; the cross-scene memory migration module is used for realizing cross-domain adaptation through a domain adapter and comparative learning; and the memory conflict resolution module identifies conflicts based on the rule base and adopts a multi-factor voting mechanism to resolve the conflicts. According to the method, the memory retention integrity and the retrieval efficiency are improved, the resource utilization is optimized, the cross-scene adaptation capability is enhanced, the memory conflict is eliminated, and the model reliability is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model memory optimization, and particularly relates to a memory optimization system for a large language model. BACKGROUND

[0002] The breakthrough progress of large language models (LLMs) in natural language processing, knowledge question answering, and multi-turn dialogue has made it a core carrier for the landing of artificial intelligence technology. However, as the complexity of the tasks processed by the model increases (such as long text understanding, cross-domain reasoning, and professional knowledge question answering), the limitations of its memory mechanism become increasingly prominent, becoming a key bottleneck restricting the further improvement of model performance.

[0003] Currently, the memory management of LLMs generally relies on a fixed-length context window, which is difficult to adapt to the dynamic changes in task requirements. In multi-turn dialogue scenarios, early key information (such as user historical preferences and core constraint conditions) is often truncated due to the limitations of the window length, leading to broken reasoning chains. For example, in a medical consultation scenario, if the patient's initial symptom description is pushed out of the context window after the 5th turn of dialogue, the model may give incorrect diagnosis recommendations due to the loss of key information; in a legal consultation task, the context association of the cited law may generate legal opinions that are inconsistent with the premises after being truncated. The essence of this information truncation problem is that the model lacks a long-term storage and dynamic calling mechanism for key memories, and cannot flexibly adjust the memory priority according to the progress of the task.

[0004] In the prior art, there are significant defects in the coordination mechanism of memory and reasoning. On the one hand, the memory resource scheduling is disconnected from the real-time reasoning load, resulting in a memory overflow rate as high as 45% in high-concurrency scenarios. For example, when the model is processing multiple long dialogue tasks simultaneously, the statically allocated memory pool cannot be dynamically expanded, frequently triggering memory overflow, forcing the task to be interrupted or downgraded. On the other hand, the memory value evaluation lacks association with the reasoning target, and a large amount of redundant information (such as repeated small talk content and low-relevance background knowledge) occupies more than 83% of the working memory, leading to increased retrieval delay of key memories (such as task parameters and constraint conditions), and even incorrect pruning. In addition, when migrating cross-domain memories, existing solutions often destroy the original relationship structure of entities, such as directly mapping the "evidence-conclusion" relationship in the legal domain to the "symptom-diagnosis" relationship in the medical domain, which, due to the lack of association adaptation mechanism, leads to logical contradictions in the migrated memories, with an accuracy drop of more than 30%.

[0005] The weakness of the memory conflict resolution mechanism further exacerbates the unreliability of the model output. When dealing with multi-source knowledge, there are often semantic contradictions between different sources of information (such as different versions of drug contraindication descriptions and ambiguous law interpretation), and traditional text-level conflict detection can only identify literal contradictions and cannot handle implicit conflicts in the vector space. For example, when the model simultaneously remembers "drug A is suitable for children" and "drug A is contraindicated for adolescents", due to the lack of multi-dimensional decision-making mechanism based on semantic similarity, time decay and source credibility, the model may randomly select one piece of information to output, resulting in an increase in the risk of safety recommendations.

[0006] In addition, the memory optimization modules of existing systems are mostly independently designed, lacking cross-module coordination mechanisms. For example, the memory compression algorithm does not consider the real-time needs of the reasoning task, and excessive compression leads to loss of key information; the preference vector generated by the user intent enhancement module cannot update the memory value score in real time, resulting in an insufficient personalized memory recall accuracy of less than 50%. This "every man for himself" design greatly reduces the effectiveness of memory optimization and makes it difficult to meet the comprehensive needs of complex tasks for memory integrity, retrieval efficiency and cross-scene adaptability.

[0007] In summary, the memory mechanism of existing large language models has significant shortcomings in structural flexibility, resource coordination, conflict handling capacity, etc., and there is an urgent need to build a dynamic and collaborative memory optimization system to achieve precise storage, efficient scheduling, cross-domain reuse and conflict resolution of key memories, providing core support for the reliable application of LLMs in complex scenarios. SUMMARY

[0008] The present application proposes a memory optimization system for large language models to solve the problems mentioned in the prior art.

[0009] To achieve the above purpose, the present application adopts the following technical solution: a memory optimization system for large language models, comprising:

[0010] A dynamic semantic memory modeling module: a topic-entity-relation three-layer memory graph is constructed, a context-aware memory vector is generated through a bidirectional Transformer encoder, an incremental training mechanism is adopted to support dynamic expansion of memory capacity to 100 million entity records; BERT model is used for entity recognition, TPLinker algorithm is used for relation extraction, and semantic information in the context window is fused to generate a memory vector, and the TransE algorithm is used to enhance the representation of entity association through knowledge graph embedding to provide structured memory support for model reasoning;

[0011] Adaptive memory resource scheduling module: based on model inference load to build resource prediction model, using reinforcement learning PPO algorithm to dynamically allocate memory pool partition, real-time response to inference load changes, when GPU memory occupancy rate > 85% for 30 seconds, automatically trigger buffer compression, token generation rate < 5 tokens / s, temporarily expand the working area capacity by 50%, ensure that memory resources and computing needs are dynamically matched during model inference process;

[0012] Memory value optimization module: using double-factor decay mechanism to evaluate memory value, keeping complete inference link for high-value memory, memories with scores < 0.3 are marked for pruning, improving the utilization efficiency of key memories by the model;

[0013] Cross-scene memory migration module: through the adversarial domain adapter, the memory is migrated across domains, the target domain corpus is collected to fine-tune the adapter, and the domain difference is eliminated through adversarial training, so that the model can reuse effective memories in different scenes;

[0014] Memory conflict resolution module: based on multi-factor voting mechanism to solve memory conflicts; each module is represented by a unified memory vector to achieve cooperation:

[0015] The context-aware memory vector generated by the dynamic semantic memory modeling module constitutes the core source of the inference vector, and the inference vector simultaneously drives memory resource scheduling, value evaluation, and conflict resolution based on unified memory vector representation.

[0016] The alignment vector output by the cross-scene migration module updates the memory value score in real time;

[0017] The conflict resolution result is fed back to the memory structure to realize dynamic reconstruction;

[0018] Cooperatively operating inference enhancement module: when the cross-scene memory migration module is activated, the decay factor of the memory value optimization module is automatically reset; the memory conflict resolution module calls the working area data of the adaptive memory resource scheduling module for real-time verification; when the memory value score exceeds the threshold, it is forcibly locked in the working area of the adaptive memory resource scheduling module.

[0019] Further, it further comprises:

[0020] Memory compression and index optimization module: using three-level compression of quantization-distillation-hashing, first preserving core memories through knowledge distillation, then performing INT4 quantization, and finally generating 128-bit local sensitive hash (LSH) index.

[0021] User intent memory enhancement module: through user behavior sequence analysis to build user preference vector, execute weighted enhancement on memories that meet the preferences, and realize memory recall relevance improvement in dialogue scenarios.

[0022] Furthermore, the dynamic semantic memory modeling module employs an improved BERT model for entity recognition, uses the TPLinker algorithm for relation extraction, integrates context window semantic information during memory vector generation, and enhances entity association representation through knowledge graph embedding of the TransE algorithm.

[0023] Furthermore, the load adjustment process of the adaptive memory resource scheduling module is as follows: when the GPU memory usage rate is >85% for 30 seconds, cache compression is automatically triggered; when the token generation rate is <5 tokens / s, the working area capacity is temporarily expanded by 50%; when the archive access frequency is >1 time / minute, the corresponding memory is automatically migrated to the cache area, and the entire scheduling process does not interrupt model inference.

[0024] Furthermore, the scoring method for the memory value optimization module is as follows: Memory comprehensive score S = (α) t ·0.4)+(γ f ·0.3)+(i·0.3), where t is the time since the last visit, f is the number of times not visited, and i is the importance score. Memory marks with a score <0.3 are to be pruned. Core relationship information of related entities is preserved during pruning.

[0025] Furthermore, the domain adaptation steps of the cross-scene memory transfer module are as follows: collecting target domain corpus to fine-tune the domain adapter, eliminating domain differences through adversarial training, and adopting the transfer learning loss function L = L 任务 +0.1L 对齐 Optimize model parameters.

[0026] Furthermore, the conflict handling mechanism of the memory conflict resolution module is as follows: when a conflict is detected, low-quality memories are first filtered out by source credibility score, then timestamps are compared, and finally the final content to be retained is determined by knowledge graph consistency verification.

[0027] Furthermore, the retrieval acceleration method of the memory compression and index optimization module is as follows: construct a two-level index structure, use the Approximate Nearest Neighbor Search (ANNS) algorithm to generate a candidate set, and then filter the optimal result through similarity calculation.

[0028] Furthermore, the user intent memory enhancement module is based on the fundamental function of constructing user preference vectors through user behavior sequence analysis. Its preference learning process is as follows: collect user interaction data within 30 days, extract preference features through the Transformer-XL model, generate user interest vectors, calculate preference matching degree through vector dot product during memory recall, and prioritize returning the memory content with the top 30% matching degree to improve user satisfaction.

[0029] Compared with existing technologies, the beneficial effects of this invention are:

[0030] 1. The dynamic semantic memory modeling module constructs a three-layer memory graph of "topic-entity-relationship" and, combined with context-aware memory vector generation, achieves accurate capture and associated storage of key information. This effectively solves the information truncation problem caused by traditional fixed windows, enabling the model to fully retain and call up key memories in multi-turn dialogues and complex reasoning, thereby improving the accuracy and coherence of the output.

[0031] 2. The adaptive memory resource scheduling mechanism significantly optimizes system performance. By dynamically adjusting memory partitions in real time by monitoring model load, and combining flexible compression and migration strategies, it balances memory usage and processing efficiency while ensuring uninterrupted model inference. This avoids memory overflow and wasted computing power, improves system response speed and throughput, and enables the model to stably handle high-concurrency requests and complex task processing requirements.

[0032] 3. The memory decay and enhancement mechanism prioritizes high-value information by distinguishing memory importance, reducing interference from redundant information and improving the accuracy of memory retrieval. The cross-scenario memory transfer module, through domain adapters and contrastive learning, enables the effective reuse of knowledge memory in different domains, reducing the adaptation cost of cross-domain applications and expanding the model's application scope.

[0033] 4. The memory conflict resolution module effectively identifies and resolves contradictory information through logical verification and a multi-factor voting mechanism, ensuring the consistency and reliability of the memory bank. Overall, this application's system enhances the long-term memory capacity, resource utilization efficiency, and cross-scenario adaptability of large-scale language models, improving the accuracy and reliability of the output. Attached Figure Description

[0034] Figure 1 This is a schematic block diagram of a memory optimization system for large language models proposed in this invention.

[0035] Figure 2 This is a schematic diagram comparing the memory retention rates of a memory optimization system for large-scale language models proposed in this invention.

[0036] Figure 3 This diagram illustrates a comparison of memory retrieval efficiency in a memory optimization system for large-scale language models proposed in this invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0039] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly; for example, they may refer to a fixed connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. The invention will now be described in further detail with reference to the accompanying drawings.

[0040] Reference Figures 1-3 A specific implementation of a memory optimization system for large language models:

[0041] A memory optimization system for large language models includes:

[0042] Dynamic semantic memory modeling module: Constructs a three-layer memory graph of topic-entity-relationship, generates context-aware memory vectors through a bidirectional Transformer encoder, adopts an incremental training mechanism, and supports dynamic expansion of memory capacity to 100 million entity records; entity recognition adopts the BERT model, relation extraction uses the TPLinker algorithm, and memory vectors are generated by fusing context window semantic information. The TransE algorithm is embedded in the knowledge graph to enhance the entity association representation and provide structured memory support for model reasoning.

[0043] Adaptive memory resource scheduling module: Based on the model inference load, a resource prediction model is built. The reinforcement learning PPO algorithm is used to dynamically allocate memory pool partitions and respond to changes in inference load in real time. When the GPU memory usage rate is >85% for 30 seconds, cache compression is automatically triggered. When the token generation rate is <5 tokens / s, the working area capacity is temporarily expanded by 50%, ensuring that memory resources and computing needs are dynamically matched during the inference of large language models.

[0044] Memory value optimization module: A two-factor decay mechanism is used to evaluate memory value. High-value memories retain the complete reasoning chain, and memories with a score <0.3 are marked as needing pruning, thereby improving the utilization efficiency of key memories in large language models.

[0045] Cross-scenario memory transfer module: It realizes cross-domain memory transfer through adversarial domain adapter, collects target domain corpus to fine-tune the adapter, eliminates domain differences through adversarial training, and enables large language models to reuse effective memories in different scenarios;

[0046] Memory conflict resolution module: Resolves memory conflicts based on a multi-factor voting mechanism; the various modules collaborate through a unified memory vector representation.

[0047] The context-aware memory vector generated by the dynamic semantic memory modeling module constitutes the core source of the inference vector. Based on a unified memory vector representation, it achieves collaborative driving. The inference vector simultaneously drives memory resource scheduling, value assessment, and conflict resolution.

[0048] The alignment vector output by the cross-scene migration module is updated in real time to update the memory value score;

[0049] The conflict resolution results are fed back to the memory structure to enable dynamic reconstruction;

[0050] The collaborative reasoning enhancement module automatically triggers the reset of the decay factor of the memory value optimization module when the cross-scene memory migration module is activated; the memory conflict resolution module calls the work area data of the adaptive memory resource scheduling module to achieve real-time verification; when the memory value score exceeds the threshold, it is forcibly locked in the work area of ​​the adaptive memory resource scheduling module.

[0051] This invention also includes:

[0052] Memory compression and index optimization module: It adopts a three-level compression of quantization-distillation-hashing. First, the core memory is preserved through knowledge distillation, then INT4 quantization is performed, and finally a 128-bit Locality Sensitive Hash (LSH) index is generated.

[0053] User intent memory enhancement module: Constructs user preference vectors through user behavior sequence analysis, performs weighted enhancement on memories that match preferences, and improves memory recall relevance in dialogue scenarios.

[0054] In this invention, the dynamic semantic memory modeling module uses an improved BERT model for entity recognition, the TPLinker algorithm for relation extraction, and integrates context window semantic information when generating memory vectors. It also enhances entity association representation by embedding the TransE algorithm into a knowledge graph.

[0055] In this invention, the load adjustment process of the adaptive memory resource scheduling module is as follows: when the GPU memory usage rate is >85% for 30 seconds, cache compression is automatically triggered; when the token generation rate is <5 tokens / s, the working area capacity is temporarily expanded by 50%; when the archive access frequency is >1 time / minute, the corresponding memory is automatically migrated to the cache area, and the entire scheduling process does not interrupt model inference.

[0056] In this invention, the scoring method for the memory value optimization module is as follows: Memory comprehensive score S = (α) t ·0.4)+(γ f ·0.3)+(i·0.3), where t is the time since the last visit, f is the number of times not visited, and i is the importance score. Memory marks with a score <0.3 are to be pruned. Core relationship information of related entities is preserved during pruning.

[0057] In this invention, the domain adaptation step of the cross-scene memory transfer module is as follows: collecting target domain corpus to fine-tune the domain adapter, eliminating domain differences through adversarial training, and using the transfer learning loss function L = L 任务 +0.1L 对齐 Optimize model parameters.

[0058] In this invention, the conflict handling mechanism of the memory conflict resolution module is as follows: when a conflict is detected, low-quality memories are first filtered out by source credibility score, then timestamps are compared, and finally the final content to be retained is determined by knowledge graph consistency verification.

[0059] In this invention, the retrieval acceleration method of the memory compression and index optimization module is as follows: construct a two-level index structure, use the Approximate Nearest Neighbor Search (ANNS) algorithm to generate a candidate set, and then filter the optimal result through similarity calculation.

[0060] In this invention, the user intent memory enhancement module is based on the basic function of constructing user preference vectors through user behavior sequence analysis. Its preference learning process is as follows: collect user interaction data within 30 days, extract preference features through the Transformer-XL model, generate user interest vectors, calculate preference matching degree through vector dot product during memory recall, and prioritize returning the memory content with the top 30% matching degree to improve user satisfaction.

[0061] Example 1

[0062] This application system addresses the pain points of memory management for large language models by achieving efficient memory optimization through multi-module collaboration, making it suitable for scenarios such as long dialogues and cross-domain reasoning. Taking a medical consultation dialogue scenario as an example, the system operation flow is as follows:

[0063] I. Core Module Operation Mechanism

[0064] Dynamic semantic memory modeling module

[0065] When constructing a three-layer memory graph of "topic-entity-relationship", the topic layer is bound to the current inference stage of LLM (e.g., the "symptom description" topic is automatically activated in the "initial consultation" stage, and the "drug-contraindication" topic is activated in the "treatment suggestion" stage); the entity layer identifies entities such as diseases and drugs by improving the BERT model and integrates the referential resolution results in the LLM dialogue (e.g., associating "the patient said 'that antipyretic'" with the entity "acetaminophen"); when the relation layer extracts relationships such as "cause-symptom" using the TPLinker algorithm, it simultaneously verifies the consistency with the LLM inference logic (e.g., excluding associations such as "cold-chemotherapy" that do not conform to medical inference).

[0066] When generating memory vectors, the dynamic semantic memory modeling module uses a bidirectional Transformer (1024-dimensional hidden layer) to not only integrate the context of the first 5 sentences plus the last 3 sentences, but also introduces LLM attention weights (weighting the top 20% of LLM attention context). The entity vectors (512-dimensional) constructed using the TransE algorithm additionally encode the association frequency of entities in historical LLM reasoning (e.g., the "pneumonia-fever" association vector dynamically adjusts its distance based on reasoning frequency). Simultaneously, the module incorporates an attention-memory collaboration mechanism. When a key entity focused by LLM attention is detected not in the current workspace, a lightweight retrieval priority boost is automatically triggered. This temporarily unlocks sub-partitions of the workspace (≤10%) to enable rapid retrieval of key memories, avoiding access blockage caused by workspace locking. This mechanism calculates the conflict coefficient by comparing the LLM attention weight distribution with the workspace memory index. Among them, Ω attend For focusing attention on the entity set, Ω work For the work area entity set, when C>0.3, sub-partition unlocking is initiated to ensure that the key memory that the attention is directed to can be accessed first, balancing the resource protection of work area locking with the need for key memory retrieval.

[0067] Incremental updates are no longer triggered by a fixed 100,000 tokens. Instead, they are automatically initiated when the adaptive memory resource scheduling module reports that "the proportion of newly added entities in the high-frequency memory of the work area exceeds 30%". After the update, the list of high-frequency entities is synchronized to the dynamic semantic memory modeling module in real time.

[0068] Adaptive memory resource scheduling module

[0069] The system is divided into a working area (1GB), a cache area (4GB), and an archive area (10GB). However, the partition capacity is linked to the dynamic semantic memory modeling module: the working area capacity is dynamically adjusted according to the number of high-frequency entities output by the dynamic semantic memory modeling module (expanding by 10% for every 500 new high-frequency entities); when the cache area adopts the LRU-K (K=3) strategy, the replacement priority is based on the memory decay and the initial score of the enhancement module (low-scoring memories are replaced first); the quantization compression intensity of the archive area is bound to the memory decay and the enhancement module (memories with a score <0.2 are quantized using INT4, and those with a score of 0.2-0.3 are quantized using INT8).

[0070] Memory decay and enhancement module

[0071] The two-factor decay mechanism adds an "LLM inference dependency" factor (weight 0.2), and the overall score formula is adjusted to: S = (α t ·0.4)+(γ f ·0.3)+(i·0.3), where t is the time since the last visit, f is the number of unvisited visits, and i is the importance score. During pruning every 24 hours, in addition to retaining memories with S≥0.3, the "core entity association memory" marked by the dynamic semantic modeling module is also prioritized for retention; high-value memories (i≥4 stars) in enhanced storage will be simultaneously pushed to the cross-scenario memory migration module with the "domain core memory" tag;

[0072] The dynamic semantic memory modeling module is the core module of the large-scale language model memory optimization system. Its main function is to build structured memory and generate context-aware memory vectors to provide accurate memory support for model reasoning.

[0073] This module constructs a three-layer memory graph of "topic-entity-relationship". The topic layer is bound to the current inference stage of the model, such as activating the "symptom description" topic in the "preliminary consultation" stage; the entity layer identifies entities such as diseases and drugs by improving the BERT model and integrates the referential resolution results in the dialogue; the relationship layer uses the TPLinker algorithm to extract relationships such as "cause-symptom" and at the same time verify the consistency with the model's inference logic.

[0074] When generating memory vectors, a bidirectional Transformer (1024-dimensional hidden layer) integrates the context of the first 5 sentences and the last 3 sentences, and also introduces the model's attention weights to weight the top 20% of the contexts that receive the most attention. The 512-dimensional entity vectors constructed by the TransE algorithm additionally encode the frequency of entity associations in the model's historical inferences. For example, the distance of the "pneumonia-fever" association vector is dynamically adjusted according to the inference frequency.

[0075] In addition, the module adopts an incremental training mechanism, which supports dynamic expansion of memory capacity to 100 million entity records. Incremental updates are no longer triggered by tokens, but are automatically started when the adaptive memory resource scheduling module reports that "the proportion of newly added entities in the high-frequency memory of the work area exceeds 30%". After the update, the high-frequency entity list is synchronized to the adaptive memory resource scheduling module, realizing dynamic optimization and efficient utilization of memory.

[0076] Cross-scene memory transfer module

[0077] When migrating from the medical field to the legal field, the "topic-entity-relationship" structure of the dynamic semantic modeling module is used as the migration skeleton (e.g., "symptom-diagnosis" in medical terms corresponds to "evidence-conclusion" in law). When fine-tuning the domain adapter, priority is given to adapting entity types that are frequently used in legal scenarios by LLM (e.g., "legal provisions" and "cases"). When constructing the alignment space through comparative learning, the "domain core memory" pushed by the memory decay and enhancement modules is integrated to improve the recall rate of the core memory after migration.

[0078] Memory Conflict Resolution Module

[0079] When a conflict is detected in "Drug A Indication", in addition to source credibility and timestamp, entity relationship data from the dynamic semantic modeling module (such as the correlation between "Drug A - Indication" and "Patient Symptoms") and the score from the memory decay and enhancement module (high score memory weight is increased) are also called. The conflict is resolved through multi-factor voting (semantic correlation 0.4, source credibility 0.3, time freshness 0.3), and the result is fed back to the dynamic semantic memory modeling module to update the relationship layer.

[0080] The three-level compression method of "quantization-distillation-hashing" is adopted: first, knowledge distillation (compression ratio of 4:1 for teacher model to student model) is used to retain the core memory, then INT4 quantization is performed (precision loss ≤2%), and finally a 128-bit Local Sensitive Hash (LSH) index is generated, reducing the storage cost of a single memory to 1 / 16 of the original.

[0081] The rule base contains over 1000 rules, including drug contraindications and legal provisions. When a conflict is detected (such as a contradiction in the indications for drug A), the following calculation is made: C = 0.5s + 0.3c + 0.2t ， s represents semantic similarity (0-1), c represents source credibility (0-1), and t represents time freshness (0-1). The memory with the highest C is retained, and the response time is ≤50ms.

[0082] Other module details

[0083] A two-level index structure is constructed (the first level is classified by topic, and the second level is partitioned by hash bucket). The Approximate Nearest Neighbor Search (ANNS) algorithm is used to generate a candidate set (returning the top 200 results), and then the optimal results are selected by precise similarity calculation (cosine similarity).

[0084] The memory compression and index optimization module performs "distillation (4:1) → INT4 quantization → 128-bit LSH", reducing storage cost to 1 / 16; the user intent memory enhancement module uses Transformer-XL to extract preference vectors and prioritizes recalling the top 30% of memories with the highest matching degree.

[0085] Each module forms a closed loop through "real-time data synchronization and dynamic parameter adaptation": the dynamic semantic memory modeling module provides high-frequency entity data for the adaptive memory resource scheduling module; the adaptive memory resource scheduling module provides storage constraints for the memory value optimization module; the memory value optimization module marks core memories for the cross-scenario memory migration module; and the migration results of the cross-scenario memory migration module feed back into the dynamic semantic memory modeling module to supplement cross-domain associations. In the medical-to-legal scenario, the accuracy of cross-domain memory migration is improved to 92%, and the memory retrieval latency remains ≤5ms, a 15% improvement compared to independent operation.

[0086] II. Performance Data Representation

[0087] Indicator Conventional system System of the present application Multi-turn dialogue memory retention rate 45％ 92％ Memory retrieval delay 80 ms 8 ms Cross-domain migration adaptation time 24 hours 1.5 hours Memory conflict resolution accuracy rate 60％ 95％

[0088] The data shows that the proposed system has significant advantages: improved memory retention in multi-turn dialogues, resolving information truncation issues; significantly reduced retrieval latency, improving response speed; shorter cross-domain adaptation time, reducing migration costs; and improved conflict resolution accuracy, ensuring output consistency. These improvements make the model more reliable in long dialogues and cross-domain tasks, with more efficient resource utilization, making it suitable for professional scenarios such as medicine and law.

[0089] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A memory optimization system for large language models, characterized in that, include: Dynamic semantic memory modeling module: Constructs a three-layer memory graph of topic-entity-relationship, generates context-aware memory vectors through a bidirectional Transformer encoder, adopts an incremental training mechanism, and supports dynamic expansion of memory capacity to 100 million entity records; entity recognition adopts the BERT model, relation extraction uses the TPLinker algorithm, memory vectors are generated by fusing context window semantic information, and entity association representation is enhanced by embedding the TransE algorithm into the knowledge graph. Adaptive memory resource scheduling module: Based on the model inference load, a resource prediction model is built, and the reinforcement learning PPO algorithm is used to dynamically allocate memory pool partitions and respond to changes in inference load in real time; when the GPU memory usage rate is >85% for 30 seconds, cache compression is automatically triggered, and when the token generation rate is <5 tokens / s, the working area capacity is temporarily expanded by 50%. Memory value optimization module: A two-factor decay mechanism is used to evaluate memory value. High-value memories retain the complete reasoning chain, and memories with a score <0.3 are marked as needing pruning. Cross-scene memory transfer module: Enables cross-domain memory transfer through adversarial domain adapter; Memory conflict resolution module: Resolves memory conflicts based on a multi-factor voting mechanism; The modules collaborate through a unified memory vector representation; The context-aware memory vector generated by the dynamic semantic memory modeling module constitutes the core source of the inference vector. Based on a unified memory vector representation, it achieves collaborative driving. The inference vector simultaneously drives memory resource scheduling, value assessment, and conflict resolution. The alignment vector output by the cross-scene migration module is updated in real time to update the memory value score; The conflict resolution results are fed back to the memory structure to enable dynamic reconstruction; The collaborative reasoning enhancement module automatically triggers the reset of the decay factor of the memory value optimization module when the cross-scene memory migration module is activated; the memory conflict resolution module calls the work area data of the adaptive memory resource scheduling module to achieve real-time verification; when the memory value score exceeds the threshold, it is forcibly locked in the work area of ​​the adaptive memory resource scheduling module.

2. The memory optimization system for large language models according to claim 1, characterized in that, Also includes: Memory compression and index optimization module: It adopts a three-level compression of quantization-distillation-hashing. First, the core memory is preserved through knowledge distillation, then INT4 quantization is performed, and finally a 128-bit Locality Sensitive Hash (LSH) index is generated.

3. The memory optimization system for large language models according to claim 1, characterized in that, Also includes: User intent memory enhancement module: Constructs user preference vectors through user behavior sequence analysis, performs weighted enhancement on memories that match preferences, and improves memory recall relevance in dialogue scenarios.

4. A memory optimization system for large language models according to claim 1, characterized in that, The dynamic semantic memory modeling module uses an improved BERT model for entity recognition, the TPLinker algorithm for relation extraction, and integrates context window semantic information when generating memory vectors. It also enhances entity association representation by embedding the TransE algorithm into a knowledge graph.

5. A memory optimization system for large-scale language models according to claim 1, characterized in that, The load adjustment process of the adaptive memory resource scheduling module is as follows: when the GPU memory usage rate is >85% for 30 seconds, cache compression is automatically triggered; when the token generation rate is <5 tokens / s, the working area capacity is temporarily expanded by 50%; when the archive access frequency is >1 time / minute, the corresponding memory is automatically migrated to the cache area. The entire scheduling process does not interrupt model inference.

6. A memory optimization system for large language models according to claim 1, characterized in that, The scoring method for the memory value optimization module is as follows: Memory comprehensive score S = (α) t ·0.4+(γ f ·0.3)+(i·0.3), where t is the time since the last visit, f is the number of times not visited, and i is the importance score. Memory marks with a score <0.3 are to be pruned. Core relationship information of related entities is preserved during pruning.

7. A memory optimization system for large language models according to claim 1, characterized in that, The domain adaptation steps of the cross-scene memory transfer module are as follows: collecting target domain corpus to fine-tune the domain adapter, eliminating domain differences through adversarial training, and adopting the transfer learning loss function L = L 任务 +0.1L 对齐 Optimize model parameters.

8. A memory optimization system for large language models according to claim 1, characterized in that, The conflict resolution module's conflict handling mechanism is as follows: when a conflict is detected, low-quality memories are first filtered out by source credibility score, then timestamps are compared, and finally, the final content to be retained is determined by knowledge graph consistency verification.

9. A memory optimization system for large-scale language models according to claim 2, characterized in that, The retrieval acceleration method of the memory compression and index optimization module is as follows: construct a two-level index structure, use the near nearest neighbor search ANNS algorithm to generate a candidate set, and then filter the optimal results through similarity calculation.

10. A memory optimization system for large language models according to claim 2, characterized in that, The user intent memory enhancement module is based on the fundamental function of constructing user preference vectors through user behavior sequence analysis. Its preference learning process is as follows: collect user interaction data within 30 days, extract preference features through the Transformer-XL model, generate user interest vectors, calculate preference matching degree through vector dot product during memory recall, and prioritize returning the memory content with the top 30% matching degree to improve user satisfaction.

Images