Storage Control Plane Erasure Coding for Low-Latency Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing NVMe over Fabrics configurations face challenges in providing erasure code data protection due to increased latency and cost with RAID on a chip ASIC, which degrades performance, especially in Ethernet-attached SSDs with point-to-point connectivity.
Innovation Solution
A storage system with a data switch and a controller that routes commands to storage devices, generates erasure codes on dedicated devices invisible to the host, and redirects traffic to the controller upon device failure or removal, emulating the failed device's responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If RAID on a chip ASIC is used to provide data protection, then reliability is improved, but latency increases and performance degrades
Solution Approach 1:
A controller is introduced as an intermediary component that manages erasure code operations across multiple storage devices. The controller coordinates the generation and distribution of erasure codes, allowing data protection to be implemented through software-based control rather than hardware-based RAID ASIC, thereby reducing latency while maintaining reliability.
Solution Approach 2:
The storage system is segmented into multiple independent storage devices connected through a data switch, with erasure code functionality distributed across the system rather than centralized in a single RAID ASIC. This segmentation allows parallel processing of erasure code operations, improving performance and reducing latency.
2Reliability
If RAID on a chip ASIC is used to provide data protection, then reliability is improved, but cost increases
Solution Approach 1:
The controller serves as a mediator that implements erasure code functionality through existing storage devices and data switches, eliminating the need for expensive dedicated RAID ASIC hardware. This approach achieves the same data protection reliability using standard, more cost-effective components.
Solution Approach 2:
Existing storage devices and controllers are made multi-functional by enabling them to perform erasure code generation and management in addition to their primary storage functions. This eliminates the need for separate dedicated RAID hardware, reducing overall system cost while maintaining data protection capabilities.
3Device complexity
If point-to-point connectivity is used in Ethernet-attached SSDs, then device simplicity is improved, but erasure code implementation becomes more challenging
Solution Approach 1:
A controller is introduced as an intermediary that manages erasure code operations across multiple storage devices with point-to-point connectivity. The controller coordinates data distribution and erasure code generation, enabling erasure code functionality without requiring complex multi-ported device architecture.
Solution Approach 2:
Storage devices maintain their simple point-to-point connectivity while the controller enables erasure code functionality by having devices participate in distributed erasure code operations. Each device independently performs its portion of erasure code generation and storage, with coordination provided by the controller, thus maintaining device simplicity while achieving erasure code capability.
Data Source
AI summary
A system and method for providing erasure code protection across multiple storage devices. A data switch in a storage system connects a plurality of storage devices to a remote host. Each storage device is also connected to a controller, e.g., a baseboard management controller. During normal operation, read and write commands from the remote host are sent to respective storage devices through the data switch. When a write command is executed, the storage device executing the command sends a copy of the data to the controller, which generates and stores erasure codes, e.g., on a storage device that is dedicated to the storage of erasure codes, and invisible to the remote host. When a device fails or is removed, the controller reconfigures the data switch to redirect all traffic addressed to the failed or absent storage device to the controller, and the controller responds to host commands in its stead.