Distributed RAID Parallel Data Block Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data redundancy techniques, such as RAID, are time and resource intensive for rebuilding data sets from failed storage devices, and may not provide the most recent data due to sequential I/O performance limitations and the need for single-drive write speed.
Innovation Solution
The method involves dividing user data into data blocks and writing them into chunk zones across an ordered pool of storage devices, allowing for parallel data transfers, expanding the pool with additional devices, and reconstructing data in spare capacity, enabling fast rebuilds and handling partial failures without failing the entire device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional RAID techniques are used for data redundancy, then data integrity is maintained, but rebuild time becomes excessively long and system resources are overwhelmed
Solution Approach 1:
The patent divides the rebuild process into parallel streams, where data blocks are segmented and reconstructed simultaneously across multiple storage devices rather than sequentially on a single drive. This segmentation of the rebuild operation enables concurrent processing, dramatically reducing total rebuild time while maintaining data integrity through systematic reconstruction of all data blocks.
Solution Approach 2:
The system pre-allocates spare capacity on each storage device and maintains an ordered pool of storage devices ready for reconstruction operations. When a failure occurs, the rebuild process can immediately utilize pre-positioned spare capacity and pre-established parallel streams, eliminating the need for single-drive sequential writes and reducing rebuild time significantly.
2Device complexity
If data is written sequentially to storage devices, then simplicity is maintained, but sequential write performance becomes a bottleneck
Solution Approach 1:
The patent segments data into multiple data blocks that are distributed across an ordered pool of storage devices. Each storage device receives and processes data blocks in parallel streams, transforming the single sequential write operation into multiple concurrent write operations. This segmentation maintains relative system simplicity while dramatically improving sequential write performance through parallelism.
3Reliability
If traditional RAID rebuild processes are used, then data recovery is achieved, but system resources become overwhelmed and performance degrades
Solution Approach 1:
The rebuild operation is segmented into parallel streams that distribute the computational and I/O workload across multiple storage devices simultaneously. Instead of concentrating all rebuild resources on a single drive, the system divides the reconstruction task into manageable segments that can be processed concurrently, reducing peak resource consumption and preventing system overload while ensuring complete data recovery.
Data Source
AI summary
The technology disclosed herein includes a method for dividing a body of user data into a plurality of data blocks, and writing the plurality of data blocks into chunk zones in parallel streams, the chunk zones located in a first ordered pool of storage devices. In some implementations, the method includes adding additional storage devices to the first ordered pool making a second ordered pool, including the first ordered pool, and writing the plurality of data blocks across the second ordered pool of storage devices, such that each of the storage devices including spare capacity. The method includes determining if a storage device fails, and seeking data for the data blocks on the failed storage device from of the other storage devices.