Distributed Metadata Cache Segmentation for Eviction and Insertion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed object stores face challenges in achieving sustained steady-state performance due to cache eviction and insertion operations that cause contention and inefficiencies, particularly when handling a large number of threads, leading to unpredictable performance.
Innovation Solution
A distributed metadata cache system with active and unreferenced sets manages cache entries, allowing efficient eviction and insertion by decoupling these operations, using independent synchronization processes and self-tuning mechanisms to minimize contention and ensure resources are available for new entries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional cache eviction and insertion operations are used in a distributed object store, then cache operations can be performed, but contention and inefficiencies occur leading to unpredictable performance
Solution Approach 1:
The cache is divided into two independent sets: an active set for cache lookups and an unreferenced set for eviction operations. This segmentation allows lookup operations to proceed without contention from eviction operations, as each set is managed independently with its own synchronization mechanisms.
Solution Approach 2:
The unreferenced set acts as an intermediary buffer between cache insertion and eviction. Entries are moved to this set when no longer actively referenced, allowing eviction operations to process them separately without blocking active cache lookups, thus improving overall system predictability and efficiency.
2Productivity
If a massive number of threads are employed to achieve performance, then throughput increases, but thread contention on shared resources increases causing bottlenecks
Solution Approach 1:
By segmenting the cache into active and unreferenced sets with independent synchronization, the patent reduces resource contention among multiple threads. Each set can be managed by different threads simultaneously without interfering with each other, allowing massive thread counts to scale effectively.
Solution Approach 2:
The system dynamically manages thread access to different cache sets based on operation type. Lookup operations access the active set while eviction operations process the unreferenced set, allowing threads to be dynamically allocated to different operations without global contention.
3Measurement precision
If cache eviction operations are performed frequently to maintain cache efficiency, then cache hit rate improves, but eviction overhead increases impacting frontend IO performance
Solution Approach 1:
Entries are preliminarily moved to the unreferenced set when they are no longer actively referenced, before actual eviction occurs. This preliminary action separates the identification of evictable entries from the eviction process itself, reducing the overhead impact on frontend IO operations.
Solution Approach 2:
The system maintains continuous cache efficiency by continuously populating the unreferenced set with candidates for eviction, while eviction operations process this set independently. This continuous separation of concerns ensures high cache hit rates without periodic performance dips from batched eviction operations.
Data Source
AI summary
A distributed metadata cache for a distributed object store includes a plurality of cache entries, an active-cache-entry set and an unreferenced-cache-entry set. Each cache entry includes information relating to whether at least one input/output (IO) thread is referencing the cache entry and information relating to whether the cache entry is no longer referenced by at least one IO thread. Each cache entry in the active-cache-entry set includes information that indicates that at least one IO thread is actively referencing the cache entry. Each cache entry in the unreferenced-cache-entry set is eligible for eviction from the distributed metadata cache by including information that indicates that the cache entry is no longer actively referenced by an IO thread.