Distributed Head Node Storage System with Erasure Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Block-based storage systems face challenges in maintaining data durability and low latency due to single-point failures in server nodes or control planes, leading to significant performance degradation and high recovery latencies across multiple locations.
Innovation Solution
A data storage system with multiple head nodes and data storage sleds, where data is replicated across nodes and sleds, using erasure encoding and redundant networks to ensure durability and low latency, with a local control plane managing failovers and data distribution to maintain autonomy and reduce blast radii of failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored across multiple devices in multiple locations to increase durability, then data durability is improved, but recovery latency increases due to data being located across multiple locations
Solution Approach 1:
The system segments data into multiple parts and distributes them across different storage sleds within the same rack. Each part is stored in a separate sled, allowing parallel recovery operations while maintaining data durability through distribution. This segmentation enables faster recovery compared to centralized storage while preserving the benefits of distributed storage for durability.
Solution Approach 2:
The patent transitions from geographic distribution across multiple locations to spatial distribution within a single rack. By changing the dimension of data distribution from inter-location to intra-rack, the system maintains data durability through distribution while dramatically reducing recovery latency since all data resides in the same physical facility with high-speed interconnects.
2Device complexity
If a common control plane is used to manage data in multiple locations, then system management is simplified, but a failure of a control plane component impacts a large quantity of storage capacity
Solution Approach 1:
The control plane is segmented into distributed control logic residing in each head node and sled controller. Each component manages its local resources autonomously, eliminating the single point of failure in centralized control. This segmentation maintains manageable system operation through standardized interfaces while dramatically improving reliability by distributing control functions.
Solution Approach 2:
Each head node and sled controller operates autonomously to manage its local data and resources without requiring centralized control plane intervention for basic operations. The system components self-manage their own state, handling failures and recovery independently, which reduces the blast radius of control plane failures while maintaining systematic coordination through peer-to-peer communication.
3Adaptability or versatility
If extensive networks are used to move data between multiple locations, then data distribution capability is improved, but network complexity and cost increase
Solution Approach 1:
The patent merges multiple specialized network connections into a unified high-speed interconnect fabric within the rack. By combining storage, compute, and management traffic over a common infrastructure using standards-based protocols, the system achieves versatile data distribution capability while reducing network complexity and cost compared to dedicated networks for each function.
4Ease of operation
If block-based storage with server nodes is used to provide standardized storage calls, then storage accessibility is improved, but a failure of server nodes renders large amounts of storage capacity unusable
Solution Approach 1:
The critical control functions are extracted from centralized server nodes and distributed to head nodes and sled controllers. This extraction eliminates the single point of failure in server-based architectures, allowing storage capacity to remain accessible even when individual head nodes fail, while maintaining standardized storage interface accessibility for clients.
Solution Approach 2:
The architecture transitions from a two-tier server-node model to a multi-dimensional distributed architecture with head nodes and sled controllers operating at different levels. This dimensional change enables localized failure containment while preserving overall system accessibility through alternative paths and redundant control points.
Data Source
AI summary
A data storage system includes multiple head nodes and data storage sleds. A control plane of the data storage system designates, for a volume partition, one of the head nodes to function as a primary head node storing a primary replica of the volume partition and designates two or more other head nodes to function as reserve head nodes storing reserve replicas of the volume partition. Additionally, the primary head node causes volume data for the volume partition to be erasure encoded and stored on multiple mass storage devices in different ones of the data storage sleds.