Cache Replacement Using In-Vector LRU Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The LRU cache replacement algorithm incurs high runtime overhead due to the need for additional memory space to record access timestamps, which is inefficient and not scalable for large numbers of cache entries.
Innovation Solution
Implementing an in-vector LRU management technique using SIMD instructions, where cache entries are reordered within vector registers, allowing for efficient cache replacement without additional memory metadata, by using multiple vectors and a multi-step LRU algorithm that prioritizes frequently accessed items.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional LRU cache replacement algorithm is used with per-entry metadata, then accurate LRU ordering is achieved, but memory space overhead and runtime overhead increase significantly
Solution Approach 1:
The patent merges the cache data storage and LRU ordering information into a single unified structure. By using bitwise operations on the same cache entry fields to encode both data and access order information, it eliminates the need for separate metadata structures, thereby reducing memory overhead while maintaining LRU functionality
Solution Approach 2:
The patent changes the representation parameters of cache entries by using bit field encoding schemes. Instead of storing full timestamps or pointers, it uses modified bit patterns within the cache entry itself to represent access order, transforming the storage format to achieve both space efficiency and ordering information preservation
2Reliability
If traditional LRU cache replacement algorithm is used with additional bookkeeping operations, then accurate LRU ordering is maintained, but runtime overhead increases
Solution Approach 1:
The patent extracts the LRU ordering maintenance operations from the traditional complex bookkeeping system and replaces them with simplified bitwise operations. By removing the need for separate timestamp updates and linked list manipulations, it significantly reduces runtime overhead while preserving LRU ordering through more efficient bit-level manipulations
Solution Approach 2:
The patent substitutes the mechanical bookkeeping operations (linked list updates, timestamp comparisons, pointer manipulations) with electronic bitwise operations. This replacement leverages the processor's native bit manipulation capabilities to achieve faster, more efficient LRU maintenance without the overhead of traditional data structure operations
3Reliability
If cache entries are reordered frequently to maintain LRU, then cache hit ratio improves, but memory access performance degrades
Solution Approach 1:
The patent performs preliminary encoding of access order information at the time of cache insertion or access, storing the order information in a compressed bit representation. This preliminary action eliminates the need for frequent reordering operations later, as the ordering information is already captured and can be used directly for replacement decisions
Solution Approach 2:
Instead of reordering physical cache entries in memory (which degrades performance), the patent inverts the approach by maintaining order information in a separate encoded form that can be processed without moving data. The logical ordering is maintained through bit manipulation while physical memory positions remain stable
Data Source
AI summary
A method configures a cache to implement a LRU management technique. The cache has N entries divided into B buckets. Each bucket has a number of entries equal to P entries*M vectors, wherein N=B*P*M. Any P entry within any M vector is ordered using an in-vector LRU ordering process. Any entry within any bucket is ordered in LRU within the vectors and buckets. The LRU management technique moves a found entry to a first position within a same M vector, responsive to a lookup for a specified key, and permutes the found entry and a last entry in a previous M vector, responsive to the found entry already being in the first position within a vector and the same one of the M vectors not being a first vector in the bucket in the moving step.