Cuckoo Cache Eviction Without Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional caches require additional storage capacity for metadata to implement LRU and LFU eviction strategies, leading to inefficient use of space, especially when the cache is full and must evict data.
Innovation Solution
The cuckoo cache system eliminates the need for metadata by using an array of buckets with unique index values, where each entry is a key-value pair, and new items are written to one of two assignable buckets, allowing for efficient retention of data entries by moving them to alternate buckets when full, employing a modified LRU eviction strategy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If metadata is stored in the cache to implement LRU and LFU eviction strategies, then eviction functionality is improved, but storage capacity is reduced
Solution Approach 1:
The patent extracts the metadata storage function from the cache itself and relocates it to external structures. Counters tracking usage frequency are stored outside the cache buckets, allowing the cache to maintain full capacity for actual data while still implementing LFU eviction based on external counter information.
Solution Approach 2:
The patent introduces external counter structures as intermediaries between the cache and the eviction decision-making process. These counters mediate the tracking of usage frequency without consuming cache space, enabling the cache to distinguish between data storage and usage tracking functions.
2Quantity of substance
If cache capacity is increased to store more data, then storage capacity is improved, but the proportion of metadata overhead increases
Solution Approach 1:
By extracting metadata storage from the cache structure itself and placing it in external counter arrays, the patent enables the cache to scale its data storage capacity without proportionally increasing metadata overhead. The metadata lives separately and does not consume bucket space.
Solution Approach 2:
The patent separates the metadata dimension from the data storage dimension by using external counter structures. This dimensional separation allows independent scaling of cache capacity without being constrained by proportional metadata growth within the same storage space.
3Reliability
If LRU or LFU eviction strategies are implemented using metadata, then data retention accuracy is improved, but space efficiency deteriorates
Solution Approach 1:
External counter structures act as intermediaries that accurately track usage frequency without consuming cache space. The counters provide reliable LFU information for eviction decisions while the cache buckets remain entirely available for data storage, maintaining both accuracy and space efficiency.
Solution Approach 2:
The patent segments the cache system into distinct functional components: data storage buckets and external usage-tracking counters. This segmentation allows each component to optimize for its specific function without compromising the other, achieving accurate eviction tracking with minimal space overhead.
Data Source
AI summary
A cuckoo cache has plural buckets of plural cells each. The cells within a bucket are ranked to approximate relative usage recency. New items can be inserted into empty cells; when a bucket is full, room for a new item can be made by laterally transferring an older item to an alternative bucket. When empty cells and lateral transfers are unavailable, an item is selected for eviction based on the usage recency rank of the containing cell. When a match is found, depending on the embodiment, the hit item can be promoted within its bucket, to its alternative bucket, or to a separate tier of the cuckoo cache. The items can be key-value pairs. No metadata is required to track usage recency so that the cuckoo cache can be a very space efficient tool for finding cached values by their keys.