A kv cache memory strength management method based on a four-dimensional discrete space-time cognitive system

Abstract

This invention discloses a cross-domain memory strength management and key-value (KV) cache optimization method based on a four-dimensional discrete-time spatiotemporal cognitive system, applicable to scenarios such as human-computer interaction, machine interaction, neuroscience, and large-scale model reasoning. This invention uses a unified modeling of information units based on four-dimensional discrete-time spatiotemporal coordinates, employs a cognitive heuristic model that integrates repetition reinforcement, cue matching, encoding depth, structural disorder, time decay, and interference dynamics to calculate memory strength, and applies this model to KV cache eviction management. This invention extends the four-dimensional discrete-time spatiotemporal cognitive system from a single cache scenario to the fields of human-computer interaction, machine interaction, and neuroscience, achieving unified cross-domain cognitive strength control; when applied to KV caching, it can significantly improve the retention rate of key information and the performance of long-context reasoning. This invention has strong versatility and controllable computational overhead, and can be widely adapted to scenarios such as artificial intelligence interaction, distributed machine communication and neuromorphic computing, brain-computer interfaces, and multi-agent communication.

Description

Technical Field

[0001] This invention is a core component of the inventor's complete four-dimensional discrete spatiotemporal cognitive system and a key implementation of the next-generation large-scale model unified architecture in inference optimization and key-value (KV) cache management. It shares the same origin, isomorphism, and unified mathematical foundation with previously submitted patents in artificial intelligence, brain-computer interfaces, neural information element modeling, and cognitive quantitative evaluation. This invention uses four-dimensional discrete spatiotemporal coordinates to uniformly model token information units, employs a cognitively inspired memory strength model to calculate cache priority, combines proactive and retroactive interference dynamics to correct intensity, and performs KV cache eviction based on memory strength. This invention achieves fundamental theoretical interoperability between biological cognition, artificial intelligence, and cache management, significantly improving the retention rate of key information in long contexts, and can be widely applied to scenarios such as large-scale model inference, brain-like computing, and brain-computer interface information scheduling. Background Technology

[0002] Large language models require the maintenance of a key-value cache (KV Cache) when generating text. As sequence length increases, the cache size grows linearly, leading to high GPU memory pressure and reduced inference speed. Existing cache eviction strategies include LRU (Least Recently Used), FIFO (First-In-First-Out), sliding windows, and the attention-score-based H2O method. These methods only utilize access order, time intervals, or attention weights, ignoring cognitive mechanisms such as semantic content of information elements, repetition reinforcement effects, encoding depth, structural disorder, and proactive / retroactive interference. Furthermore, existing caching strategies lack a unified spatiotemporal cognitive framework, failing to achieve low-level interoperability with neuroscience cognitive models, human-computer interaction, and multi-agent communication, resulting in the easy loss of key information and instability in long contexts. Summary of the Invention

[0003] Technical problems to be solved This invention extends the four-dimensional discrete spatiotemporal cognitive system from caching scenarios to cross-domain scenarios such as human-computer interaction, machine interaction, and neuroscience, providing a unified method for memory strength management. It overcomes the shortcomings of traditional methods, such as reliance on access order, lack of cognitive modeling, and cross-domain incompatibility. This invention locates tokens based on four-dimensional discrete spatiotemporal coordinates and calculates cache priority using a memory strength model that includes repetition reinforcement, cue matching, encoding depth, structural disorder, time decay, and interference dynamics. This achieves content-aware, cognitively inspired cache eviction, improving the retention rate of key information and the stability of inference. Technical solution Step 1: Token localization based on four-dimensional discrete spatiotemporal coordinates: Assign four-dimensional coordinates to each token being processed. Automatic coordinate mapping, no additional storage required: Time coordinates The absolute timestamp generated by the token is based on a unified reference time source. Context coordinates Dialogue turn identifier: odd numbers represent user input, even numbers represent model output, and the number increments automatically. Logical coordinates : The position of the token sequence within a single input or output, starting from 1 and incrementing. Namespace : Device or session identifier used to distinguish different data sources. Step 2: Calculation of the cognitive-inspired memory strength model: For each token in the cache Its memory strength Calculate using the following formula: The parameters are defined as follows: Number of repetitions for reinforcement , For token embedding vectors, is the similarity threshold, which is an adjustable parameter. In this embodiment, the exemplary value is 0.7. Clue matching degree, current query and token Attention weights or dot product normalized values. Encoding depth ,in , This represents the maximum length of a single input, and is an adjustable parameter. In this embodiment, the exemplary value is 512. : The structure is disordered, and the degree of disorder before and after using Tokens The gzip compression ratio of a local window of a token. : Can be replaced with Shannon entropy; The absolute difference between the current time and the token generation time. . : This is an adjustable parameter; in this embodiment, the exemplary value is [value to be inserted here]. , , , . Step 3: Correction of disturbance dynamics Proactive interference: When a new token When written to the cache, its initial strength is suppressed by existing similar tokens: in For cosine similarity, , As a role factor, based on The parity is an adjustable parameter; for example, values ​​are 1.2 for the same role and 0.8 for different roles. For equipment factors, based on Whether they are the same is an adjustable parameter. In this embodiment, the exemplary values ​​are 1.2 for the same and 1.0 for different. Back-motion interference: when Token When a token is accessed for enhancement, its strength is reduced by weakening similar old tokens: in is the backflush interference attenuation coefficient, which is an adjustable parameter; in this embodiment, the exemplary value is 0.02. For Token The intensity increment, This is an adjustable parameter; in this embodiment, the example value is 10.0. Step 4: Cache Eviction Policy When the cache capacity reaches its limit, the token with the lowest memory strength is evicted. Update when access is hit. , , , And recalculate Simultaneously, back-feedback interference is applied. Periodic updates: In this embodiment, the exemplary value is every 10 steps. The memory strength of all tokens is updated over time by decay. Step 5: Reserve expansion interfaces A time decay extension interface is reserved to support the adjustment of the time decay factor. The time interval is replaced with one based on information density field correction to adapt to the upgrade requirements of high information density scenarios. This invention currently uses an approximation, but the interface is retained. Beneficial effects 1. Upgrade the four-dimensional discrete spatiotemporal cognitive system from a single cache to a universal cross-domain cognitive foundation, compatible with human-computer interaction, machine communication, brain science and brain-like computing; 2. The memory strength model integrates repetition reinforcement, encoding depth, structural disorder, exponential time decay, and proactive / retroactive interference dynamics. It draws on cognitive laws to construct a computational framework, while overcoming the shortcomings of human brain memory such as easy forgetting, susceptibility to interference, and unreliable retention of key information, to achieve stable and controllable retention of high-priority information. 3. A unified memory strength model and interference dynamics are adopted to achieve consistent memory regulation across different scenarios; 4. When applied to KV caching, it significantly improves the retention rate of key information in long contexts; 5. Computational overhead is controllable, and it can be seamlessly integrated into existing large-scale model inference frameworks; 6. It can be extended to high-value scenarios such as brain-computer interfaces, multi-agent communication, and distributed reasoning systems; Attached Figure Description Figure 1 Memory Strength Calculation Flowchart Figure 2 Diagram of forward-facing interference and backward-facing interference Figure 3 : Cache eviction policy flowchart Detailed Implementation Example 1: Cache Management in Long Document Digests Experimental environment: NVIDIA A100 graphics card, vLLM 0.4.2 inference framework, PyTorch 2.1, CUDA 12.1; Llama-3-8B model, arXiv paper abstract dataset, cache capacity set to 4096 tokens, parameters using default values. Results: Compared with the LRU strategy, ROUGE-L improved by 3.2%; compared with the H2O strategy, ROUGE-L improved by 1.8%; and the average recall rate of the first 1 / 3 of key entities in the document improved by 12.0%. Example 2: Role Consistency in Multi-Turn Dialogues A 100-round dialogue simulation test was conducted based on the PersonaChat dataset, and the parity of the context coordinate c was used to distinguish between users and model roles. Results: In the 100th round, the accuracy of the present invention in responding to entities mentioned in the 1st round reached 87%, LRU was 71%, and H2O was 76%, demonstrating a significant improvement in long-range memory retention.

Claims

1. Claim 1 (Independent Claim): A KV cache memory strength management method based on a four-dimensional discrete spatiotemporal cognitive system, characterized in that, include: Based on a unified four-dimensional discrete spatiotemporal coordinate system, four-dimensional coordinates are assigned to each token information unit. ; The memory strength of the token is calculated based on the coordinates, and the strength is determined by repetition reinforcement, clue matching, encoding depth, structural disorder, and time decay. Proactive and retroactive interference corrections are applied to memory strength. When the cache capacity reaches the threshold, the token with the lowest memory strength is evicted.

2. Claim 2 (dependent claim): The method according to claim 1, characterized in that, The four-dimensional discrete spatiotemporal coordinate system adopts the same origin, isomorphism, and unified mathematical framework as the inventor's patents in the fields of artificial intelligence, human-computer interaction, machine interaction, and brain science.

3. Claim 3 (dependent claim): The method according to claim 1, characterized in that, The memory strength is calculated using a logarithmic weighted method, and the calculation formula is as follows: ,in, For the number of repetitions of reinforcement, For clue matching degree, For encoding depth, For structural disorder, For time intervals, For parameters.

4. Claim 4 (dependent claim): The method according to claim 1, characterized in that, The four-dimensional coordinates are specifically: Generate timestamps for information units; Interaction rounds and role identifiers; : Local logical sequence position; : Device, session, or source namespace.

5. Claim 5 (dependent claim): The method according to claim 3, characterized in that, The number of repetitions ， For token embedding vectors, This is the similarity threshold.

6. Claim 6 (dependent claim): The method according to claim 3, characterized in that, The structural disorder The gzip compression ratio of a local window composed of k tokens before and after using the token, where k=5.

7. Claim 7 (dependent claim): The method according to claim 1, characterized in that, The proactive interference correction includes: When a new token is written to the cache, its initial memory strength is suppressed by the strength of existing similar tokens.

8. Claim 8 (dependent claim): The method according to claim 1, characterized in that, The backflip interference correction includes: When a token is accessed and strengthened, the strength of the execution is reduced against older tokens that are similar to it.

9. Claim 9 (dependent claim): The method according to claim 1, characterized in that, Cache eviction policies also include: Each time a hit occurs, the memory strength of the corresponding token is updated and a backseat interference correction is triggered; The memory strength of all tokens is updated periodically with time decay.

10. Claim 10 (dependent claim): The method according to claim 1, characterized in that, Also includes: A time decay extension interface is reserved to support the use of absolute time intervals in the time decay factor. Replace with time intervals based on information density field correction .

11. Claim 11 (dependent claim): The method according to claim 1, characterized in that, This method can be directly adapted to the neural information element and AIToken mapping system, and supports cache scheduling in neuromorphic computing and brain-computer interface scenarios.

12. Claim 12 (Independent Claim): A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the method described in any one of claims 1 to 11.

13. Claim 13 (Independent Claim): A KV cache management system, characterized in that, include: The module includes a coordinate generation module, a memory strength calculation module, an interference correction module, and a cache eviction module.

Images