Parity Data Encoding for RAID Silent Corruption Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Erasure coding systems are impractical for RAID systems with more than two check drives due to computational complexity and hardware requirements, and they fail to detect and correct silent data corruptions effectively, especially in large-scale storage environments.
Innovation Solution
A practical erasure coding system that uses a precomputed master encoding matrix for efficient encoding and decoding, supports up to 256 drives, and incorporates a fast solution matrix algorithm for Reed-Solomon codes, along with maintaining parity data in a RAID cache for error detection and correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If erasure coding is implemented with more than two check drives, then data reliability and fault tolerance are improved, but computational complexity and hardware requirements increase significantly
Solution Approach 1:
The patent precomputes and stores encoding matrices (parity matrices) before actual encoding operations. These matrices are calculated in advance and stored in memory, allowing the encoding process to simply perform matrix multiplication rather than complex real-time calculations. This preliminary preparation resolves the contradiction by shifting computational burden from runtime to setup time.
Solution Approach 2:
The patent uses precomputed encoding matrices that can be reused across multiple encoding operations. Instead of recalculating complex encoding relationships each time, the system copies and applies the same parity matrices to different data sets, significantly reducing repeated computational complexity while maintaining reliability.
2Reliability
If traditional erasure coding is used, then forward error correction is provided, but silent data corruptions cannot be detected and corrected
Solution Approach 1:
The patent extends the traditional parity data structure to serve multiple functions simultaneously. The extended parity data not only provides forward error correction for lost blocks but also enables detection and correction of silent data corruptions in remaining blocks. This multi-functionality resolves the contradiction by making the same parity mechanism address both types of errors.
Solution Approach 2:
The patent implements a verification mechanism where decoded data is checked against regenerated parity information. When silent corruptions are detected, the system uses the parity data to identify and correct these errors. This feedback loop enables the system to detect and correct harmful factors that traditional erasure coding would miss.
3Reliability
If real-time encoding and decoding is performed, then data protection is provided, but computational overhead and processing time increase
Solution Approach 1:
The patent precomputes encoding matrices and stores them for reuse. This preliminary action eliminates the need for complex real-time matrix calculations during encoding and decoding operations, reducing processing time while maintaining data protection capabilities.
Solution Approach 2:
The system copies precomputed parity matrices and applies them to multiple data blocks without recalculation. This copying approach significantly reduces repeated computational overhead while maintaining real-time protection capabilities.
4Reliability
If parity data is maintained for all data blocks, then error detection and correction are enabled, but storage space requirements increase
Solution Approach 1:
The patent changes the structure and organization of parity data from traditional block-level redundancy to a more efficient matrix-based representation. By organizing parity information across multiple blocks using encoding matrices, the system achieves comparable error detection capability with reduced total parity storage requirements.
Data Source
AI summary
A system for software error-correcting code (ECC) protection or compression of original data using ECC data in a first memory is provided. The system includes a processing core for executing computer instructions and accessing data from a main memory, and a non-volatile storage medium for storing the computer instructions. The software ECC protection or compression includes: a data matrix for holding the original data in the first memory; a check matrix for holding the ECC data in the first memory; an encoding matrix for holding first factors in the main memory, the first factors being for encoding the original data into the ECC data; and a thread for executing on the processing core. The thread includes a Galois Field multiplier for multiplying entries of the data matrix by an entry of the encoding matrix, and a sequencer for ordering operations using the Galois Field multiplier to generate the ECC data.