A memory optimization system for large language models

Abstract

The application discloses a memory optimization system for a large language model, comprising: a dynamic semantic memory modeling module, which generates context-aware memory vectors through a bidirectional Transformer and supports incremental updating to 100 million entity records; an adaptive memory resource scheduling module, which dynamically allocates memory pool partitions based on inference load, stores high-frequency memories in the working area, and enables quantization compression in the archive area; a memory decay and enhancement module, which adopts a double-factor decay mechanism, performs enhanced storage on high-value memories, and prunes them regularly; a cross-scenario memory migration module, which realizes cross-domain adaptation through a domain adapter and contrastive learning; and a memory conflict resolution module, which identifies conflicts based on a rule base and resolves contradictions using a multi-factor voting mechanism. The application improves memory retention integrity and retrieval efficiency, optimizes resource utilization, enhances cross-scenario adaptation capability, resolves memory conflicts, and improves model reliability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model memory optimization, and particularly relates to a memory optimization system for a large language model. BACKGROUND

[0002] The breakthrough progress of large language models (LLMs) in natural language processing, knowledge question answering, and multi-turn dialogue has made it a core carrier for the landing of artificial intelligence technology. However, as the complexity of the tasks processed by the model increases (such as long text understanding, cross-domain reasoning, and professional knowledge question answering), the limitations of its memory mechanism become increasingly prominent, becoming a key bottleneck restricting the further improvement of model performance.

[0003] Currently, the memory management of LLMs generally relies on a fixed-length context window, which is difficult to adapt to the dynamic changes in task requirements. In multi-turn dialogue scenarios, early key information (such as user historical preferences and core constraint conditions) is often truncated due to the limitations of the window length, leading to broken reasoning chains. For example, in a medical consultation scenario, if the patient's initial symptom description is pushed out of the context window after the 5th turn of dialogue, the model may give incorrect diagnosis recommendations due to the loss of key information; in a legal consultation task, the context association of the cited law may generate legal opinions that are inconsistent with the premises after being truncated. The essence of this information truncation problem is that the model lacks a long-term storage and dynamic calling mechanism for key memories, and cannot flexibly adjust the memory priority according to the progress of the task.

[0004] In the prior art, there are significant defects in the coordination mechanism of memory and reasoning. On the one hand, the memory resource scheduling is disconnected from the real-time reasoning load, resulting in a memory overflow rate as high as 45% in high-concurrency scenarios. For example, when the model is processing multiple long dialogue tasks simultaneously, the statically allocated memory pool cannot be dynamically expanded, frequently triggering memory overflow, forcing the task to be interrupted or downgraded. On the other hand, the memory value evaluation lacks association with the reasoning target, and a large amount of redundant information (such as repeated small talk content and low-relevance background knowledge) occupies more than 83% of the working memory, leading to increased retrieval delay of key memories (such as task parameters and constraint conditions), and even incorrect pruning. In addition, when migrating cross-domain memories, existing solutions often destroy the original relationship structure of entities, such as directly mapping the "evidence-conclusion" relationship in the legal domain to the "symptom-diagnosis" relationship in the medical domain, which, due to the lack of association adaptation mechanism, leads to logical contradictions in the migrated memories, with an accuracy drop of more than 30%.

[0005] The weakness of the memory conflict resolution mechanism further exacerbates the unreliability of the model output. When dealing with multi-source knowledge, there are often semantic contradictions between different sources of information (such as different versions of drug contraindication descriptions and ambiguous law interpretation), and traditional text-level conflict detection can only identify literal contradictions and cannot handle implicit conflicts in the vector space. For example, when the model simultaneously remembers "drug A is suitable for children" and "drug A is contraindicated for adolescents", due to the lack of multi-dimensional decision-making mechanism based on semantic similarity, time decay and source credibility, the model may randomly select one piece of information to output, resulting in an increase in the risk of safety recommendations.

[0006] In addition, the memory optimization modules of existing systems are mostly independently designed, lacking cross-module coordination mechanisms. For example, the memory compression algorithm does not consider the real-time needs of the reasoning task, and excessive compression leads to loss of key information; the preference vector generated by the user intent enhancement module cannot update the memory value score in real time, resulting in an insufficient personalized memory recall accuracy of less than 50%. This "every man for himself" design greatly reduces the effectiveness of memory optimization and makes it difficult to meet the comprehensive needs of complex tasks for memory integrity, retrieval efficiency and cross-scene adaptability.

[0007] In summary, the memory mechanism of existing large language models has significant shortcomings in structural flexibility, resource collaboration and conflict handling capability, and there is an urgent need to build a dynamic collaborative memory optimization system to achieve precise storage, efficient scheduling, cross-domain reuse and conflict resolution of key memories, providing core support for the reliable application of LLMs in complex scenarios. SUMMARY

[0008] The present application proposes a memory optimization system for large language models to solve the problems mentioned in the prior art.

[0009] To achieve the above purpose, the present application adopts the following technical solution: a memory optimization system for large language models, comprising:

[0010] A dynamic semantic memory modeling module: a topic-entity-relation three-layer memory graph is constructed, a context-aware memory vector is generated through a bidirectional Transformer encoder, an incremental training mechanism is adopted to support dynamic expansion of memory capacity to 100 million entity records; BERT model is used for entity recognition, TPLinker algorithm is used for relation extraction, and semantic information in the context window is fused to generate memory vectors, and TransE algorithm is used to enhance the representation of entity association through knowledge graph embedding to provide structured memory support for model reasoning;

[0011] Adaptive memory resource scheduling module: based on model inference load to build resource prediction model, using reinforcement learning PPO algorithm to dynamically allocate memory pool partition, real-time response to inference load changes, when GPU memory occupancy rate > 85% for 30 seconds, automatically trigger buffer compression, token generation rate < 5 tokens / s, temporarily expand the working area capacity by 50%, ensure that memory resources and computing needs are dynamically matched during model inference process;

[0012] Memory value optimization module: using double-factor decay mechanism to evaluate memory value, keeping complete inference link for high-value memory, memories with scores < 0.3 are marked for pruning, improving the utilization efficiency of key memories by the model;

[0013] Cross-scene memory migration module: through the adversarial domain adapter, the memory is migrated across domains, the target domain corpus is collected to fine-tune the adapter, and the domain difference is eliminated through adversarial training, so that the model can reuse effective memories in different scenes;

[0014] Memory conflict resolution module: based on multi-factor voting mechanism to solve memory conflicts; each module is represented by a unified memory vector to achieve cooperation:

[0015] The context-aware memory vector generated by the dynamic semantic memory modeling module constitutes the core source of the inference vector, and the inference vector simultaneously drives memory resource scheduling, value evaluation, and conflict resolution based on unified memory vector representation.

[0016] The alignment vector output by the cross-scene migration module updates the memory value score in real time;

[0017] The conflict resolution result is fed back to the memory structure to realize dynamic reconstruction;

[0018] Cooperatively operating inference enhancement module: when the cross-scene memory migration module is activated, the decay factor of the memory value optimization module is automatically reset; the memory conflict resolution module calls the working area data of the adaptive memory resource scheduling module for real-time verification; when the memory value score exceeds the threshold, it is forcibly locked in the working area of the adaptive memory resource scheduling module.

[0019] Further, it further comprises:

[0020] Memory compression and index optimization module: using three-level compression of quantization-distillation-hashing, first preserving core memories through knowledge distillation, then performing INT4 quantization, and finally generating 128-bit local sensitive hash (LSH) index.

[0021] User intent memory enhancement module: through user behavior sequence analysis to build user preference vector, execute weighted enhancement on memories that meet the preferences, and realize memory recall relevance improvement in dialogue scenarios.

[0022] Furthermore, the dynamic semantic memory modeling module employs an improved BERT model for entity recognition, uses the TPLinker algorithm for relation extraction, integrates context window semantic information during memory vector generation, and enhances entity association representation through knowledge graph embedding of the TransE algorithm.

[0023] Furthermore, the load adjustment process of the adaptive memory resource scheduling module is as follows: when the GPU memory usage rate is >85% for 30 seconds, cache compression is automatically triggered; when the token generation rate is <5 tokens / s, the working area capacity is temporarily expanded by 50%; when the archive access frequency is >1 time / minute, the corresponding memory is automatically migrated to the cache area, and the entire scheduling process does not interrupt model inference.

[0024] Furthermore, the scoring method for the memory value optimization module is as follows: Memory comprehensive score S = (α) t ·0.4)+(γ f ·0.3)+(i·0.3), where t is the time since the last visit, f is the number of times not visited, and i is the importance score. Memory marks with a score <0.3 are to be pruned. Core relationship information of related entities is preserved during pruning.

[0025] Furthermore, the domain adaptation steps of the cross-scene memory transfer module are as follows: collecting target domain corpus to fine-tune the domain adapter, eliminating domain differences through adversarial training, and adopting the transfer learning loss function L = L 任务 +0.1L 对齐 Optimize model parameters.

[0026] Furthermore, the conflict handling mechanism of the memory conflict resolution module is as follows: when a conflict is detected, low-quality memories are first filtered out by source credibility score, then timestamps are compared, and finally the final content to be retained is determined by knowledge graph consistency verification.

[0027] Furthermore, the retrieval acceleration method of the memory compression and index optimization module is as follows: construct a two-level index structure, use the Approximate Nearest Neighbor Search (ANNS) algorithm to generate a candidate set, and then filter the optimal result through similarity calculation.

[0028] Furthermore, the user intent memory enhancement module is based on the fundamental function of constructing user preference vectors through user behavior sequence analysis. Its preference learning process is as follows: collect user interaction data within 30 days, extract preference features through the Transformer-XL model, generate user interest vectors, calculate preference matching degree through vector dot product during memory recall, and prioritize returning the memory content with the top 30% matching degree to improve user satisfaction.

[0029] Compared with existing technologies, the beneficial effects of this invention are:

[0030] 1. The "topic-entity-relation" three-layer memory graph constructed by the dynamic semantic memory modeling module, combined with the context-aware memory vector generation, realizes the accurate capture and associated storage of key information, effectively solves the information truncation problem caused by traditional fixed windows, and enables the model to retain and call key memories in multiple rounds of dialogue and complex reasoning, improving the accuracy and coherence of the output.

[0031] 2. The adaptive memory resource scheduling mechanism significantly optimizes system performance, dynamically adjusts memory partitions by real-time monitoring of model load, balances memory occupancy and processing efficiency under the premise of uninterrupted model reasoning, avoids memory overflow and waste of computing power, improves system response speed and throughput, and enables the model to stably handle high-concurrency requests and complex task processing requirements.

[0032] 3. The memory decay and enhancement mechanism distinguishes the importance of memories, prioritizes the retention of high-value information, and reduces redundant information interference, improving the accuracy of memory retrieval. The cross-scene memory migration module, with the help of domain adapters and contrastive learning, enables effective reuse of knowledge memories in different domains, reduces the adaptation cost of cross-domain applications, and expands the application range of the model.

[0033] 4. The memory conflict resolution module effectively identifies and resolves conflicting information through logical verification and multi-factor voting mechanisms, ensuring the consistency and reliability of the memory bank. Overall, the system enhances the long-term memory capacity, resource utilization efficiency, and cross-scene adaptability of large language models, improving the accuracy and reliability of the output. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A schematic block diagram of a memory optimization system for a large language model is proposed for the present application;

[0035] Figure 2 A memory retention rate comparison diagram of a memory optimization system for a large language model is proposed for the present application;

[0036] Figure 3 A memory retrieval efficiency comparison diagram of a memory optimization system for a large language model is proposed for the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0038] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0039] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features referred to. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited. In addition, the terms "mounting", "connecting", "connecting" should be broadly understood, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances, and the present application will be further described in detail below with reference to the drawings.

[0040] Reference Figures 1-3 : A specific embodiment of a memory optimization system for a large language model:

[0041] A memory optimization system for a large language model, comprising:

[0042] Dynamic semantic memory modeling module: build a three-layer memory graph of topic-entity-relation, generate context-aware memory vector through bidirectional Transformer encoder, adopt incremental training mechanism, support dynamic expansion of memory capacity to 100 million entity records; entity recognition uses BERT model, relationship extraction uses TPLinker algorithm, fusion of context window semantic information to generate memory vector, enhanced entity association representation through knowledge graph embedding TransE algorithm, provide structured memory support for model reasoning;

[0043] Adaptive memory resource scheduling module: based on model inference load, a resource prediction model is constructed, a reinforcement learning PPO algorithm is used to dynamically allocate memory pool partition, real-time response to inference load change, when GPU memory occupancy rate> 85% for 30 seconds, automatic trigger buffer compression, token generation rate <5 tokens / s, temporarily expand the working area capacity by 50%, ensure that the memory resources and computing demand are dynamically matched during the inference process of large language model;

[0044] Memory value optimization module: using double-factor decay mechanism to evaluate memory value, keeping complete inference link for high-value memory, marking memory with score <0.3 as pruning, improving the utilization efficiency of large language model for key memory;

[0045] Cross-scene memory migration module: through the adversarial domain adapter, the memory is migrated across domains, the target domain corpus is collected to adapt the adapter, and the domain difference is eliminated through adversarial training, so that the large language model can reuse effective memory in different scenes;

[0046] Memory conflict resolution module: based on a multi-factor voting mechanism to solve memory conflicts; wherein each module is cooperated through unified memory vector representation;

[0047] The context-aware memory vector generated by the dynamic semantic memory modeling module constitutes the core source of the inference vector, and the inference vector simultaneously drives memory resource scheduling, value evaluation and conflict resolution based on unified memory vector representation;

[0048] The alignment vector output by the cross-scene migration module updates the memory value score in real time;

[0049] The conflict resolution result is fed back to the memory structure to realize dynamic reconstruction;

[0050] Cooperatively operating inference enhancement module: when the cross-scene memory migration module is activated, the decay factor reset of the memory value optimization module is automatically triggered; the memory conflict resolution module calls the working area data of the adaptive memory resource scheduling module to realize real-time verification; when the memory value score exceeds the threshold, it is forcibly locked in the working area of the adaptive memory resource scheduling module.

[0051] In the present application, it also includes:

[0052] Memory compression and index optimization module: adopting three-level compression of quantization-distillation-hash, first, the core memory is reserved through knowledge distillation, then INT4 quantization is performed, and finally, 128-bit local sensitive hash (LSH) index is generated.

[0053] User intention memory enhancement module: through user behavior sequence analysis, a user preference vector is constructed, and memories that meet the preferences are weighted and enhanced, so as to improve the relevance of memory recall in the dialogue scene.

[0054] In this invention, the dynamic semantic memory modeling module uses an improved BERT model for entity recognition, the TPLinker algorithm for relation extraction, and integrates context window semantic information when generating memory vectors. It also enhances entity association representation by embedding the TransE algorithm into a knowledge graph.

[0055] In this invention, the load adjustment process of the adaptive memory resource scheduling module is as follows: when the GPU memory usage rate is >85% for 30 seconds, cache compression is automatically triggered; when the token generation rate is <5 tokens / s, the working area capacity is temporarily expanded by 50%; when the archive access frequency is >1 time / minute, the corresponding memory is automatically migrated to the cache area, and the entire scheduling process does not interrupt model inference.

[0056] In this invention, the scoring method for the memory value optimization module is as follows: Memory comprehensive score S = (α) t ·0.4)+(γ f ·0.3)+(i·0.3), where t is the time since the last visit, f is the number of times not visited, and i is the importance score. Memory marks with a score <0.3 are to be pruned. Core relationship information of related entities is preserved during pruning.

[0057] In this invention, the domain adaptation step of the cross-scene memory transfer module is as follows: collecting target domain corpus to fine-tune the domain adapter, eliminating domain differences through adversarial training, and using the transfer learning loss function L = L 任务 +0.1L 对齐 Optimize model parameters.

[0058] In this invention, the conflict handling mechanism of the memory conflict resolution module is as follows: when a conflict is detected, low-quality memories are first filtered out by source credibility score, then timestamps are compared, and finally the final content to be retained is determined by knowledge graph consistency verification.

[0059] In this invention, the retrieval acceleration method of the memory compression and index optimization module is as follows: construct a two-level index structure, use the Approximate Nearest Neighbor Search (ANNS) algorithm to generate a candidate set, and then filter the optimal result through similarity calculation.

[0060] In this invention, the user intent memory enhancement module is based on the basic function of constructing user preference vectors through user behavior sequence analysis. Its preference learning process is as follows: collect user interaction data within 30 days, extract preference features through the Transformer-XL model, generate user interest vectors, calculate preference matching degree through vector dot product during memory recall, and prioritize returning the memory content with the top 30% matching degree to improve user satisfaction.

[0061] Example 1

[0062] The system of the present application is aimed at the memory management pain points of large language models. Through multi-module cooperation, efficient memory optimization is achieved, which is suitable for long dialogue, cross-domain reasoning and other scenarios. Taking the medical consultation dialogue scenario as an example, the system running process is as follows:

[0063] I. Core module running mechanism

[0064] Dynamic semantic memory modeling module

[0065] When constructing the "topic-entity-relation" three-layer memory graph, the topic layer is bound to the current reasoning stage of LLM (such as automatically activating the "symptom description" topic in the "initial consultation" stage, and activating the "drug-contraindication" topic in the "treatment suggestion" stage); The entity layer identifies diseases, drugs and other entities through improved BERT model, and integrates the results of LLM dialogue anaphora resolution (such as associating "the patient said 'that fever-reducing drug'" to the entity "acetaminophen"); When the relationship layer extracts "cause-symptom" and other relationships using TPLinker algorithm, the consistency with LLM reasoning logic is verified synchronously (such as excluding "cold-chemotherapy" which does not conform to medical reasoning).

[0066] When the dynamic semantic memory modeling module generates memory vectors, the bidirectional Transformer (hidden layer 1024 dimensions) not only integrates the context of the previous 5 sentences and the next 3 sentences, but also introduces the attention weight of LLM (weighting the context with the top 20% of LLM attention); The entity vector (dimension 512) constructed by the TransE algorithm additionally encodes the association frequency of entities in LLM historical reasoning (such as "pneumonia- fever" association vector dynamically adjusting the distance according to the reasoning frequency). At the same time, the module has a built-in attention-memory cooperation mechanism. When it is detected that the key entity focused by LLM attention is not in the current working area, the priority of lightweight retrieval is automatically triggered to improve, and the key memory is quickly retrieved through temporary unlocking of the working area sub-partition (occupying ≤10%) to avoid access blockage caused by working area locking. This mechanism compares the LLM attention weight distribution with the working area memory index to calculate the conflict coefficient Where, Ω attend is the set of attention focused entities, Ω work is the set of working area entities, and when C>0.3, the sub-partition unlocking is started to ensure that the key memory pointed by the attention can be accessed preferentially, balancing the resource protection of the working area locking and the key memory retrieval demand.

[0067] Instead of triggering fixedly every 100,000 tokens, incremental update is automatically started when the adaptive memory resource scheduling module feedbacks that "the proportion of newly added entities in the high-frequency memory of the working area is more than 30%", and after the update, the high-frequency entity list is synchronized to the dynamic semantic memory modeling module in real time.

[0068] Adaptive memory resource scheduling module

[0069] Partition working area (1 GB), cache area (4 GB), archive area (10 GB): but partition capacity is linked with dynamic semantic memory modeling module: working area capacity is dynamically adjusted according to the number of high-frequency entities output by the dynamic semantic memory modeling module (expand 10% for every 500 new high-frequency entities); when the cache area adopts the LRU-K (K=3) strategy, the replacement priority refers to the preliminary score of the memory attenuation and enhancement module (low-score memory is preferentially replaced); the archive area quantization compression strength is bound with the memory attenuation and enhancement module (memories with a score <0.2 are quantized by INT4, and memories with a score of 0.2-0.3 are quantized by INT8).

[0070] Memory attenuation and enhancement module

[0071] The double-factor attenuation mechanism adds an "LLM inference dependence" factor (weight 0.2), and the comprehensive score formula is adjusted to: S=(a t ·0.4)+(g f ·0.3)+(i·0.3), where t is the time since the last access, f is the number of times of non-access, and i is the importance score. During the pruning every 24 hours, in addition to retaining memories with S≥0.3, "core entity associated memories" marked by the dynamic semantic modeling module are also preferentially retained; high-value memories (i≥4 stars) stored in the enhancement are simultaneously pushed to the cross-scene memory migration module with the "domain core memory" label;

[0072] The dynamic semantic memory modeling module is the core module of the large language model memory optimization system, and its main function is to build a structured memory and generate a context-aware memory vector to provide accurate memory support for model reasoning.

[0073] This module builds a "topic-entity-relation" three-layer memory graph, the topic layer is bound with the current reasoning stage of the model, such as activating the "symptom description" topic in the "preliminary diagnosis" stage; the entity layer identifies diseases, drugs and other entities through the improved BERT model, and integrates the results of anaphora resolution in the dialogue; the relation layer uses the TPLinker algorithm to extract "cause-symptom" relations, and simultaneously checks the consistency with the reasoning logic of the model.

[0074] When generating the memory vector, a bidirectional Transformer (hidden layer 1024 dimensions) fuses the context of the previous 5 sentences + the next 3 sentences, and also introduces the attention weight of the model, and weights the top 20% of the context; the 512-dimensional entity vector constructed by the TransE algorithm additionally encodes the association frequency of the entity in the history reasoning of the model, such as the "pneumonia- fever" association vector dynamically adjusting the distance according to the reasoning frequency.

[0075] In addition, the module adopts an incremental training mechanism, supports dynamic expansion of memory capacity to 100 million entity records, and incremental update is no longer triggered fixedly by tokens, but automatically started when the adaptive memory resource scheduling module feedbacks that "the proportion of newly added entities in the high-frequency memory of the work area is over 30%", and the high-frequency entity list is synchronized to the adaptive memory resource scheduling module after the update, realizing dynamic optimization and efficient utilization of memory.

[0076] Cross-scene memory migration module

[0077] When migrating from the medical field to the legal field, the "topic-entity-relation" structure of the dynamic semantic modeling module is called as the migration skeleton (such as medical "symptom-diagnosis" corresponding to legal "evidence-conclusion"); when fine-tuning the domain adapter, the entity types frequently called by the LLM in the legal field are preferentially adapted (such as "statute" and "case"); when constructing the alignment space through contrastive learning, the "domain core memory" pushed by the memory decay and enhancement module is fused to improve the recall rate of the core memory after migration.

[0078] Memory conflict resolution module

[0079] When detecting the "drug A indication" conflict, in addition to the source credibility and timestamp, the entity relationship data of the dynamic semantic modeling module (such as the correlation degree of "drug A-indication" and "patient symptoms") and the score of the memory decay and enhancement module (high-score memory weight increase) are called, and the contradiction is resolved through multi-factor voting (semantic correlation degree 0.4, source credibility 0.3, time freshness 0.3), and the result is fed back to the dynamic semantic memory modeling module to update the relationship layer.

[0080] "Quantization-distillation-hash" three-level compression is adopted: first, the core memory is retained through knowledge distillation (teacher model→student model parameter compression ratio 4:1), then INT4 quantization (precision loss≤2%) is performed, and finally 128-bit local sensitive hash (LSH) index is generated, and the storage cost of a single memory is reduced to 1 / 16 of the original;

[0081] The rule base contains 1000+ rules such as drug contraindications and legal provisions, and when a conflict (such as "drug A indication" contradiction) is detected, C=0.5s+0.3c+0.2t is calculated ， s is the semantic similarity (0-1), c is the source credibility (0-1), t is the time freshness (0-1), and the memory with the highest C is retained, and the response is≤50ms.

[0082] Other module details

[0083] A two-level index structure is constructed (one level is classified by topic and the second level is partitioned by hash bucket), an approximate nearest neighbor search (ANNS) algorithm is used to generate a candidate set (returning the first 200), and then the optimal result is selected through precise similarity calculation (cosine similarity).

[0084] Memory compression and index optimization module performs "distillation (4:1) -> INT4 quantization -> 128-bit LSH", and the storage cost is reduced to 1 / 16; the user intention memory enhancement module extracts the preference vector using Transformer-XL, and preferentially recalls the top 30% of the matching degree of memory.

[0085] Each module forms a closed loop through "real-time data synchronization - dynamic parameter adaptation": the dynamic semantic memory modeling module provides high-frequency entity basis for the adaptive memory resource scheduling module, the adaptive memory resource scheduling module provides storage constraints for the memory value optimization module, the memory value optimization module marks the core memory for the cross-scene memory migration module, and the migration result of the cross-scene memory migration module feeds back to the dynamic semantic memory modeling module to supplement the cross-domain association. In the medical to legal scene, the cross-domain memory migration accuracy is improved to 92%, and the memory retrieval delay is maintained at ≤5ms, which is improved by 15% compared with independent running.

[0086] II. Effect data representation

[0087] Indicator Conventional system System of the present application Multi-turn dialogue memory retention rate 45％ 92％ Memory retrieval delay 80 ms 8 ms Cross-domain migration adaptation time 24 hours 1.5 hours Memory conflict resolution accuracy rate 60％ 95％

[0088] The data shows that the system of the present application has significant advantages: multi-round dialogue memory retention rate is improved, solving the information truncation problem; retrieval delay is greatly reduced, improving response speed; cross-domain adaptation time is shortened, reducing migration cost; conflict resolution accuracy is improved, ensuring output consistency. These improvements make the model more reliable in long dialogue and cross-domain tasks, more efficient in resource utilization, and suitable for professional scenarios such as medical care and law.

[0089] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can make equivalent replacement or change according to the technical solution and inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A memory optimization system for large language models, comprising: Comprise: Dynamic semantic memory modeling module: build a three-layer memory graph of topic-entity-relation, generate context-aware memory vectors through a bidirectional Transformer encoder, adopt an incremental training mechanism to support dynamic expansion of memory capacity to 100 million entity records; entity recognition uses a BERT model, and relationship extraction uses a TPLinker algorithm to generate memory vectors by fusing context window semantic information and enhance entity association representation through knowledge graph embedding TransE algorithm; Adaptive memory resource scheduling module: build a resource prediction model based on model inference load, dynamically allocate memory pool partitions using reinforcement learning PPO algorithm, and respond to inference load changes in real time; automatically trigger cache compression when GPU memory occupancy rate > 85% for 30 seconds, and temporarily expand working area capacity by 50% when token generation rate < 5 tokens / s; Memory value optimization module: uses a double-factor decay mechanism to evaluate memory value, retains complete inference links for high-value memories, and marks memories with a score < 0.3 as to-be-pruned; Cross-scene memory migration module: realizes memory cross-domain migration through an adversarial domain adapter; Memory conflict resolution module: resolves memory conflicts based on a multi-factor voting mechanism; Wherein each module is realized through unified memory vector representation to achieve cooperation; The context-aware memory vectors generated by the dynamic semantic memory modeling module constitute the core source of the inference vectors, which are cooperatively driven based on unified memory vector representation, and the inference vectors simultaneously drive memory resource scheduling, value evaluation, and conflict resolution; The alignment vectors output by the cross-scene migration module update the memory value score in real time; The conflict resolution results are fed back to the memory structure to realize dynamic reconstruction; Cooperatively operating inference enhancement module: when the cross-scene memory migration module is activated, the decay factor reset of the memory value optimization module is automatically triggered; the memory conflict resolution module calls the working area data of the adaptive memory resource scheduling module for real-time verification; when the memory value score exceeds the threshold, it is forcibly locked in the working area of the adaptive memory resource scheduling module.

2. The memory optimization system for large language models of claim 1, wherein, Also include: Memory compression and index optimization module: adopts a three-level compression of quantization-distillation-hashing, first preserves core memories through knowledge distillation, then performs INT4 quantization, and finally generates a 128-bit local sensitive hash (LSH) index.

3. The memory optimization system for large language models of claim 1, wherein, Also include: User intent memory enhancement module: constructs a user preference vector through user behavior sequence analysis, performs weighted enhancement on memories that meet the preferences, and improves the relevance of memory recall in dialogue scenarios.

4. The memory optimization system for large language models of claim 1, wherein, The dynamic semantic memory modeling module uses an improved BERT model for entity recognition, a TPLinker algorithm for relationship extraction, and fuses context window semantic information when generating memory vectors to enhance entity association representation through knowledge graph embedding TransE algorithm.

5. The memory optimization system for large language models of claim 1, wherein, The load adjustment process of the adaptive memory resource scheduling module is as follows: when the GPU memory usage rate is >85% for 30 seconds, cache compression is automatically triggered; when the token generation rate is <5 tokens / s, the working area capacity is temporarily expanded by 50%; when the archive access frequency is >1 time / minute, the corresponding memory is automatically migrated to the cache area. The entire scheduling process does not interrupt model inference.

6. The memory optimization system for large language models of claim 1, wherein, The score calculation method of the memory value optimization module is: memory comprehensive score S=(a t ·0.4+(g f ·0.3)+(i·0.3), wherein t is the time from the last access, f is the number of times of not accessing, i is the importance score, the memory with a score less than 0.3 is marked as to be pruned, and the core relationship information of the associated entity is retained when pruning.

7. The memory optimization system for large language models of claim 1, wherein, The domain adaptation step of the cross-scene memory transfer module is: collecting a target domain corpus fine-tuning a domain adapter, eliminating domain differences through adversarial training, and using a transfer learning loss function L = L 任务 + 0.1L 对齐 Optimize the model parameters.

8. The memory optimization system for large language models of claim 1, wherein, The conflict resolution module's conflict handling mechanism is as follows: when a conflict is detected, low-quality memories are first filtered out by source credibility score, then timestamps are compared, and finally, the final content to be retained is determined by knowledge graph consistency verification.

9. The memory optimization system for large language models of claim 2, wherein, The retrieval acceleration method of the memory compression and index optimization module is as follows: construct a two-level index structure, use the near nearest neighbor search ANNS algorithm to generate a candidate set, and then filter the optimal results through similarity calculation.

10. The memory optimization system for large language models of claim 2, wherein, The user intent memory enhancement module is based on the fundamental function of constructing user preference vectors through user behavior sequence analysis. Its preference learning process is as follows: collect user interaction data within 30 days, extract preference features through the Transformer-XL model, generate user interest vectors, calculate preference matching degree through vector dot product during memory recall, and prioritize returning the memory content with the top 30% matching degree to improve user satisfaction.

Images