Erasure Coding with Precomputed Matrices for Multi-Drive RAID
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Erasure coding systems are impractical for RAID systems with more than two check drives due to complex calculations and hardware requirements, limiting their effectiveness in handling multiple drive failures and increasing storage costs.
Innovation Solution
A practical erasure coding system that uses a single precomputed encoding matrix for up to 256 drives, incorporating a parity drive and a fast solution matrix algorithm for Reed-Solomon codes, allowing for efficient check data generation and data reconstruction with reduced computational overhead, leveraging parallel processing to handle larger numbers of drives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If erasure coding is used with more than two check drives, then reliability against multiple drive failures is improved, but computational complexity and hardware requirements increase significantly
Solution Approach 1:
The patent precomputes and stores encoding matrices in lookup tables before runtime. When encoding or decoding is needed, the system simply retrieves the precomputed matrix from memory rather than performing complex calculations, thus maintaining high reliability with minimal computational overhead during operation.
Solution Approach 2:
The patent replaces complex polynomial multiplication and matrix inversion operations with simple table lookup operations. By substituting the mechanical computation process with a memory access process, the system achieves the same reliability benefits without the associated computational complexity.
2Reliability
If erasure coding with multiple check drives is implemented, then data protection against failures is improved, but storage cost and processing time increase
Solution Approach 1:
Encoding matrices are precomputed and stored in lookup tables during system initialization or offline processing. During actual data encoding or recovery operations, the system performs rapid table lookups instead of complex real-time calculations, significantly reducing processing time while maintaining strong data protection capabilities.
Solution Approach 2:
The patent uses lookup tables that contain precomputed encoding matrices as copies of the computational results. Instead of recalculating matrices during operation, the system retrieves stored copies, enabling fast encoding and decoding operations with enhanced data protection.
3Reliability
If complex erasure coding algorithms are used for multiple check drives, then fault tolerance is improved, but ease of implementation deteriorates
Solution Approach 1:
The complex encoding matrices are precomputed during system setup or offline processing and stored in lookup tables. During runtime, the implementation becomes trivial - simply retrieving and applying the precomputed matrices from memory - thus achieving high fault tolerance with exceptional ease of implementation.
Solution Approach 2:
The patent substitutes complex polynomial arithmetic and matrix operations with simple memory access and vector multiplication operations. This replacement transforms a difficult-to-implement algorithm into a straightforward process that can be easily coded and maintained while providing robust fault tolerance.
Data Source
AI summary
An accelerated erasure coding system includes a processing core for executing computer instructions and accessing data from a main memory, and a non-volatile storage medium for storing the computer instructions. The processing core, storage medium, and computer instructions are configured to implement an erasure coding system, which includes: a data matrix for holding original data in the main memory; a check matrix for holding check data in the main memory; an encoding matrix for holding first factors in the main memory, the first factors being for encoding the original data into the check data; and a thread for executing on the processing core. The thread includes: a parallel multiplier for concurrently multiplying multiple entries of the data matrix by a single entry of the encoding matrix; and a first sequencer for ordering operations through the data matrix and the encoding matrix using the parallel multiplier to generate the check data.