Nested Erasure Coding for Flash Array Hard-Error Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Flash storage devices face challenges in data protection due to higher likelihood of hard errors compared to HDDs, especially near the end of their write endurance or data retention life, which can lead to catastrophic device failures and latent errors, making existing RAID systems like RAID 5 and RAID 6 inefficient in handling these errors.
Innovation Solution
A nested multiple erasure correcting code system is implemented, where data is encoded using horizontal and vertical erasure codes across multiple rows and columns of pages, allowing recovery from multiple erasures in any row, reducing the need for dedicated parity devices and enhancing storage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If RAID 6 is used to protect against double catastrophic disk failures, then data protection reliability is improved, but storage efficiency deteriorates due to dedicating an entire second device for parity
Solution Approach 1:
The patent segments the parity protection into multiple components distributed across different devices rather than concentrating it in a single dedicated parity device. Each device stores both data and parity information, with parity calculated across multiple devices, allowing flexible recovery without requiring a dedicated second device for parity storage.
Solution Approach 2:
The patent implements nested error correction codes where inner codes correct errors within individual devices and outer codes correct errors across devices. This nested structure allows multiple levels of protection to coexist, enabling recovery from multiple failures without proportionally increasing storage overhead.
2Quantity of substance
If RAID 5 is used for protecting information stored in flash devices, then storage efficiency is maintained, but reliability deteriorates when latent hard errors occur combined with catastrophic failures
Solution Approach 1:
The patent performs preliminary error correction by pre-calculating and storing parity information across multiple devices before failures occur. The nested code structure is pre-configured to handle both single device failures and combined hard error scenarios, enabling proactive protection rather than reactive recovery.
Solution Approach 2:
The patent changes the error correction parameters by implementing variable-rate parity distribution where different devices contribute different amounts of parity information based on their failure probability and impact. This allows optimization for the specific failure modes of flash storage while maintaining storage efficiency.
3Reliability
If nested multiple erasure correcting codes are used to protect against multiple erasures and hard errors, then data protection reliability is improved, but device complexity increases
Solution Approach 1:
The patent segments the complex error correction algorithm into independent inner and outer code modules. The inner codes handle local error correction within each device, while outer codes handle global error correction across devices. This segmentation allows each module to be implemented and optimized independently, reducing overall system complexity.
Solution Approach 2:
The patent introduces intermediate parity calculations that serve as mediators between the data and the final error correction. These intermediate parity values simplify the error correction process by breaking down complex multi-error scenarios into manageable steps, reducing the computational complexity required for recovery operations.
Data Source
AI summary
Embodiments of the invention relate to storing data in a storage array. An aspect of the invention includes receiving write data. The write data is arranged into “r” rows and “n” columns of pages, with each page including a plurality of sectors. The write data is encoded using a plurality of horizontal and vertical erasure correcting codes on the pages. The encoding allows recovery from up to tr erasures in any one of the r rows, up to tr-1 erasures in any one of the remaining r−1 rows, up to tr-2 erasures in any one of the remaining r−2 rows, and so on, such that the encoding allows recovery from up to t1 erasures in the last remaining row. Encoded write data is output from the encoding. The encoded write data is written as a write stripe across n storage devices in a storage array.