Hierarchical Erasure Coding for Multi-Array Failure Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face challenges in efficiently recovering erasures across multiple replicas, particularly in optimizing redundancy and correcting erasures in a hierarchical approach that maximizes minimum distance, especially in scenarios involving multiple storage devices and data centers.
Innovation Solution
A computer-implemented method using erasure-correcting codes that protect arrays with horizontal, vertical, and local parities, allowing for the recovery of erasures by determining the value of erased entries based on non-erased symbols, and incorporating shared parities to correct multiple erasures within and across devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hierarchical approach with global parities is used to correct multiple device failures, then reliability is improved, but device complexity increases
Solution Approach 1:
The system segments the storage array into multiple groups, where each group is protected by local parities. This segmentation allows independent correction of failures within each group while reducing the overall complexity compared to a single global parity system covering all devices.
Solution Approach 2:
The patent introduces a hierarchical structure with multiple dimensions of protection: local parities at the group level, and global parities at the array level. This multi-dimensional approach enables correction of multiple simultaneous failures by operating at different hierarchical levels, improving reliability without requiring a single complex global solution.
2Reliability
If multiple parities are used to maximize minimum distance, then erasure correction capability is improved, but redundancy increases
Solution Approach 1:
Different regions (groups) of the storage array are protected by different parity structures tailored to their specific failure patterns. Local parities provide targeted protection for sector failures within groups, while global parities handle device-level failures, optimizing the balance between correction capability and redundancy usage.
Solution Approach 2:
The system dynamically selects which parity structure to use based on the type and number of failures detected. For single failures, local parities are used; for multiple failures, global parities are invoked. This parameter-based selection optimizes the trade-off between correction capability and redundancy consumption.
3Speed
If first responder corrects sector failures using only vertical parity, then correction speed is improved, but correction capability is limited
Solution Approach 1:
Vertical parities are pre-computed and stored for each group, enabling immediate correction of single sector failures without invoking global parities. This preliminary preparation allows fast correction for common failure patterns while reserving global parities for more complex multi-failure scenarios.
Solution Approach 2:
The correction process is dynamic: the system first attempts correction using the faster local vertical parities, and only if that fails to correct all errors does it proceed to use the global parities. This dynamic approach optimizes correction speed for typical cases while maintaining full correction capability for edge cases.
Data Source
AI summary
A computer implemented method for recovering erased entries within a system of arrays includes identifying a system consisting of a plurality of arrays, wherein each array consists of m rows and n columns of entries, each entry is divided into p symbols consisting of a plurality of bits, protecting the m rows and n columns of entries in the system with an erasure-correcting code allowing the recovery of a number of erased entries in such rows and columns, detecting an erasure corresponding to an entry in the identified system, and, responsive to detecting an erasure, determining the value of the erased entry according to the p symbols of one or more non-erased entries.