Parallel Erasure Coding with Precomputed Matrices for Multi-Drive RAID
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Erasure coding systems are impractical for RAID systems with more than two check drives due to complex calculations and hardware requirements, limiting their effectiveness in modern disk drives with higher failure rates.
Innovation Solution
A practical erasure coding system that uses a precomputed master encoding matrix for byte-level RAID processing, supporting up to 256 drives, and implements a fast solution matrix algorithm for Reed-Solomon codes, along with a parity drive and parallel processing to efficiently generate and reconstruct check data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If erasure coding is implemented with more than two check drives, then data redundancy and reliability are improved, but calculation complexity and hardware requirements increase significantly
Solution Approach 1:
The patent precomputes encoding matrices and stores them in lookup tables before runtime. When encoding data, the system retrieves precomputed matrices from memory rather than performing complex calculations in real-time. This transforms the computationally intensive erasure coding operation into a simple memory retrieval and multiplication operation, resolving the contradiction between reliability and calculation complexity.
2Reliability
If erasure coding with multiple check drives is implemented, then fault tolerance is improved, but processing speed decreases due to complex polynomial calculations
Solution Approach 1:
The system precomputes and stores encoding matrices in lookup tables during system initialization. At runtime, encoding operations simply retrieve these precomputed matrices and perform efficient matrix-vector multiplication, avoiding the need for complex polynomial calculations during data processing. This dramatically improves processing speed while maintaining fault tolerance capabilities.
Solution Approach 2:
The patent replaces the traditional mechanical/mathematical approach of polynomial-based Reed-Solomon coding with a matrix-based approach using precomputed lookup tables. This substitution transforms complex algebraic operations into simple memory access and multiplication operations, significantly improving processing speed while maintaining the same fault tolerance properties.
3Reliability
If traditional erasure coding algorithms are used, then error correction capability is maintained, but hardware requirements and system cost increase
Solution Approach 1:
The patent uses software-based precomputed lookup tables stored in standard memory instead of requiring specialized hardware circuits for erasure coding. This approach replaces expensive, complex hardware with inexpensive memory and standard processor operations, significantly reducing hardware requirements and system cost while maintaining full error correction capability.
Data Source
AI summary
An accelerated erasure coding system includes a processing core for executing computer instructions and accessing data from a main memory, and a non-volatile storage medium for storing the computer instructions. The processing core, storage medium, and computer instructions are configured to implement an erasure coding system, which includes: a data matrix for holding original data in the main memory; a check matrix for holding check data in the main memory; an encoding matrix for holding first factors in the main memory, the first factors being for encoding the original data into the check data; and a thread for executing on the processing core. The thread includes: a parallel multiplier for concurrently multiplying multiple entries of the data matrix by a single entry of the encoding matrix; and a first sequencer for ordering operations through the data matrix and the encoding matrix using the parallel multiplier to generate the check data.