Erasure Coded Storage Fault Tolerance via Meta Object Batching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
RAID-6 storage systems face inefficiencies in write operations and fault tolerance due to the need for multiple read and write operations across primary and parity disks, leading to reduced capacity and increased latency, especially when disks become unavailable.
Innovation Solution
A storage system utilizing a meta object and a capacity object to perform full-stripe writes, where data is accumulated until a threshold is reached, and then written to the capacity object, which updates a logical map for data location and allows additional disks to be added for fault tolerance when original disks are unavailable, reducing the need for single-block writes and maintaining system integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If RAID-6 uses multiple read and write operations across primary and parity disks to maintain fault tolerance, then data reliability is improved, but write efficiency and latency deteriorate
Solution Approach 1:
The patent accumulates multiple write requests in a write buffer before executing them as a batch full-stripe write operation. This preliminary accumulation allows the system to perform all necessary read and write operations to parity and primary disks in a single coordinated action, rather than processing each write request individually. The write buffer holds requests temporarily, and when full, triggers a batch operation that updates all disks simultaneously, dramatically improving write efficiency while maintaining RAID-6 fault tolerance.
2Reliability
If RAID-6 performs full-stripe writes to maintain fault tolerance, then data redundancy is improved, but computational overhead and latency increase
Solution Approach 1:
The system accumulates write requests in a buffer before executing them as a batch operation. By waiting until the buffer is full or a threshold is reached, the system can perform all parity calculations and disk writes in a single coordinated full-stripe operation, rather than performing separate operations for each write request. This batching approach reduces the total time spent on computational overhead and disk I/O operations.
3Reliability
If RAID-6 uses block-level striping with multiple disks for fault tolerance, then system reliability is improved, but capacity efficiency deteriorates
Solution Approach 1:
The write buffer accumulates multiple write requests before triggering a batch full-stripe write operation. This allows the system to utilize the entire stripe across all primary and parity disks efficiently, ensuring that each write operation makes full use of the available disk capacity in the stripe. By batching writes, the system reduces the overhead associated with frequent small writes and maximizes the usable capacity of the RAID-6 array.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for maintaining fault tolerance for new writes in a storage system when one or more components of the storage system are unavailable. One of the methods includes determining that one or more first disks of a capacity object of a storage system are unavailable, wherein the storage system comprises a segment usage table identifying the plurality of segments of the capacity object; in response: identifying a plurality of available second disks, adding a plurality of new segments corresponding to the second disks to the capacity object, and adding data identifying the plurality of new segments to the segment usage table; and for each of one or more new write requests to the capacity object: identifying an available segment from the plurality of new segments, and writing data associated with the new write request to the identified available segment.