Hybrid Local Remote Memory Cache for Capacity Latency Trade-off
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory key-value caches face limitations in aggregate memory capacity, which can hinder low-latency and high-throughput access to unstructured objects, especially when relying solely on local volatile memory like DRAM.
Innovation Solution
Implementing a data caching system that utilizes both local and remote memory, where key lookups are processed in local memory using a hash table, and data values are stored in remote memory, allowing for faster access times and larger capacity while maintaining a remote copy for crash recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is cached in local volatile memory (DRAM) to achieve low-latency access, then access speed is improved, but aggregate memory capacity is limited
Solution Approach 1:
The memory system is segmented into two distinct parts: local volatile memory (DRAM) for high-speed access and remote memory for large capacity storage. The cache server divides its memory resources between these two segments, with local memory holding frequently accessed data and remote memory providing additional capacity for less frequently accessed but still needed data.
Solution Approach 2:
The patent introduces an intermediary mechanism (hash table in local memory) that mediates between the fast local memory and the larger remote memory. This hash table enables quick lookup of data locations, allowing the system to achieve fast access times comparable to pure local memory caching while utilizing the expanded capacity of remote memory.
2Quantity of substance
If remote memory is used to increase capacity, then memory capacity is improved, but access time increases
Solution Approach 1:
The system performs preliminary actions by pre-computing and storing hash values and data location information in the local memory hash table before actual data access occurs. This preliminary organization of data location metadata in fast local memory eliminates the need to scan remote memory during access, thereby maintaining fast access times despite using remote memory for storage.
3Loss of time
If local memory is used for fast access, then access latency is reduced, but capacity is limited and vulnerability to local failure increases
Solution Approach 1:
The patent implements a copying strategy where critical data location information (hash table entries) is maintained in both local memory and remote memory. This creates redundant copies of the indexing structure, allowing the system to recover quickly from local memory failures by using the remote copy to reconstruct the local hash table, thereby improving reliability without significantly impacting access latency.
Data Source
AI summary
A system and method for retrieving cached data are disclosed herein. The system includes a cache server including a local memory and a table residing on the local memory, wherein the table is used to identify data objects corresponding to cached data. The system also includes the data objects residing on the local memory, wherein the data objects contain pointers to the cached data. The system further includes a remote memory communicatively coupled to the cache server through an Input-Output (I/O) connection, wherein the cached data resides on the remote memory.