Distributed Cache System Decoupling Storage and Compute
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed cache systems face low utilization and high latency due to the coupling of storage and computation requirements, where nodes with unused RAM lack CPU cycles, and those with spare CPU cycles experience contention, leading to inefficient resource utilization and increased latency.
Innovation Solution
A distributed cache system comprising a data storage portion, a data control portion, and a cache logic portion, where memory hosts with non-transitory memory and network interface controllers handle remote direct memory access requests, and a curator manages data striping and cache operations, including a cache service with a cache data layer and indexing layer, and a transaction interface for atomic operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If server jobs are co-located with storage resources on cluster nodes, then storage access is enabled, but CPU utilization becomes unpredictable and latency increases due to contention
Solution Approach 1:
The system segments cache server jobs into multiple instances distributed across different nodes in the cluster. Each node runs one or more cache server jobs that access storage resources on that node, while client jobs run on separate nodes. This segmentation prevents CPU contention by separating the compute and storage functions across different physical machines, eliminating the latency problem while maintaining storage access reliability.
2Loss of time
If nodes run at lower CPU utilization to reduce contention, then latency decreases, but storage resource utilization becomes inefficient
Solution Approach 1:
By segmenting the system into dedicated storage nodes and compute nodes, the invention enables both nodes to operate at optimal utilization levels simultaneously. Storage nodes can be fully utilized by cache server jobs without being idle due to CPU contention, while compute nodes handle client requests efficiently. This resolves the trade-off between latency and resource utilization.
Solution Approach 2:
The network connection acts as an intermediary between client jobs and storage resources. Instead of client jobs directly accessing storage on the same node (causing CPU contention), requests are transmitted through the network to appropriate cache server jobs on storage nodes. This intermediary approach eliminates contention while maintaining efficient resource utilization across the distributed system.
3Productivity
If multiple cache server jobs share the same node, then storage resources are utilized, but CPU cycles are contended and utilization becomes unpredictable
Solution Approach 1:
The system segments cache server jobs across multiple nodes rather than concentrating them on a single node. Each node hosts cache server jobs that access local storage resources, distributing the CPU load predictably across the cluster. This segmentation maintains high storage utilization while ensuring each node's CPU utilization remains predictable and manageable.
Data Source
AI summary
A distributed cache system including a data storage portion, a data control portion, and a cache logic portion in communication with the data storage and data control portions. The data storage portion includes memory hosts, each having non-transitory memory and a network interface controller in communication with the memory for servicing remote direct memory access requests. The data control portion includes a curator in communication with the memory hosts. The curator manages striping of data across the memory hosts. The cache logic portion executes at least one memory access request to implement a cache operation. In response to each memory access request, the curator provides the cache logic portion a file descriptor mapping data stripes and data stripe replications of a file on the memory hosts for remote direct memory access of the file on the memory hosts through the corresponding network interface controllers.