Erasure-Coded Disk Allocation Across Flexible Drive Pools
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer storage systems are inflexible in allocating erasure coded data to disk storage, as they are tied to a fixed group of drives and a specific erasure coding algorithm, limiting I/O performance, CPU utilization, storage capacity, fault tolerance, and recovery time.
Innovation Solution
A dynamic allocation process that allows erasure coded data to be stored on any disk drive within a pool, using multiple erasure coding algorithms, with the ability to 'stack' data for reduced head seeks, and utilizing an allocation bitmask for efficient storage across multiple logical storage units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is allocated to a fixed group of drives with a specific erasure coding algorithm, then the system provides stable and predictable performance, but the system loses flexibility in adapting to different data sizes, failure scenarios, and performance requirements
Solution Approach 1:
The system transitions from static drive groupings to dynamic allocation where erasure coded data can be placed on any available drive in the pool. The allocation is determined at write time based on current system state and requirements, allowing the system to adapt to different data sizes, failure scenarios, and performance needs without being constrained by pre-defined drive groups
Solution Approach 2:
The patent creates a universal storage pool where all drives can serve multiple purposes and support multiple erasure coding algorithms. Instead of dedicating specific drives to specific algorithms, any drive in the pool can store data encoded with any algorithm, maximizing resource utilization and flexibility
2Speed
If erasure coded data is allocated contiguously across drives, then data access is simplified, but the number of head seeks increases when accessing data distributed across multiple drives
Solution Approach 1:
The system shifts from thinking about data placement in terms of contiguous physical locations to organizing data in a virtual address space that maps to physical drives. By introducing the dimension of virtual-to-physical mapping, the system can optimize for different access patterns and reduce head seeks through intelligent mapping strategies
3Reliability
If the system supports multiple erasure coding algorithms with different redundancy levels, then fault tolerance can be optimized for different scenarios, but the allocation complexity increases
Solution Approach 1:
The system allows dynamic selection of erasure coding parameters (such as redundancy level, code type) based on the data being stored and current system conditions. Different data blocks can use different erasure coding schemes, allowing optimization of fault tolerance for each scenario without requiring complex manual configuration
4Productivity
If data is written to a fixed drive grouping configuration, then the allocation process is simple and fast, but the system cannot adapt to drive failures or performance degradation
Solution Approach 1:
The system continuously monitors drive health, performance, and availability, using this feedback to dynamically adjust data allocation decisions. When drives fail or degrade, the system automatically redistributes data and recalculates erasure coding requirements to maintain fault tolerance and optimal performance
Data Source
AI summary
Allocation process that allows erasure coded data to be stored on any of a plurality of disk drives, in a pool of drives, so that the allocation is not tied to a fixed group of drives. Still further, the encoded data can be generated by any of multiple different erasure coding algorithms, where again storage of the encoded data is not restricted to a single group of drives based on the erasure algorithm being utilized to encode the data. In another embodiment, the encoded data can be “stacked” (aligned) on select drives to reduce the number of head seeks required to access the data. As a result of these improvements, the system can dynamically determine which one of multiple erasure coding algorithms to utilize for a given incoming data block, without being tied to one particular algorithm and one particular group of storage devices as in the prior art.