Local Repairable Encoding Matrices for Low-Complexity Data Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing erasure coding schemes for distributed storage systems face high computational complexity and I/O overhead, particularly in generating and performing local repairs, and require significant finite field operations, which complicates data recovery and encoding processes.
Innovation Solution
The method involves generating N×M encoding matrices with specific configurations, including an M×M first matrix, an 1×M second matrix with all elements as 1, and an M×M symmetric third matrix, allowing for local and global parity block generation without increasing complexity, and enabling efficient merging and recovery using exclusive OR operations, reducing the need for finite field operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If erasure coding scheme is used to reduce storage overhead, then storage efficiency is improved, but computational complexity and I/O overhead increase significantly
Solution Approach 1:
The patent divides the erasure coding system into local groups, where each group has its own local parity blocks. This segmentation allows repair operations to be performed locally within each group rather than requiring global participation, significantly reducing I/O overhead and computational complexity during repair operations while maintaining the storage efficiency benefits of erasure coding.
Solution Approach 2:
The patent implements local repair capability by creating local parity blocks for each local group. When data loss occurs, repair operations can be performed using only the local parity block and remaining data blocks within the same group, providing local quality of repair without requiring access to all global parity blocks, thus reducing both I/O and computational overhead.
2Reliability
If local repair codes are increased to improve repairability, then data recovery capability is improved, but complexity in generating encoding matrix increases significantly
Solution Approach 1:
The patent segments the encoding matrix generation into local group-level operations. Each local group generates its own local parity blocks using simplified local encoding matrices, rather than requiring generation of a single complex global encoding matrix. This segmentation of matrix generation reduces computational complexity while providing local repair capability.
Solution Approach 2:
The patent performs preliminary action by pre-generating local parity blocks and organizing data into local groups with assigned local parity blocks before any data loss occurs. This preliminary organization into local groups with pre-computed local parity blocks enables fast repair operations without requiring complex real-time matrix generation when repair is needed.
3Quantity of substance
If global parity blocks are merged to improve storage efficiency, then storage overhead is reduced, but recovery complexity increases
Solution Approach 1:
The patent segments parity functionality into local parity blocks (for local repair) and global parity blocks (for overall redundancy). This segmentation allows the system to use simplified local repair operations for most cases while maintaining global erasure coding properties, reducing both storage overhead compared to full replication and recovery complexity compared to pure global erasure coding.
Solution Approach 2:
The patent introduces local parity blocks as intermediary elements between data blocks and global parity blocks. These local parity blocks serve as intermediaries that enable fast local repair operations without requiring direct access to global parity blocks, thus reducing recovery complexity while maintaining storage efficiency.
4Manufacturing precision
If finite field operations are used for accurate encoding, then encoding precision is improved, but computational overhead increases
Solution Approach 1:
The patent extracts the essential parity generation functionality from complex finite field operations and implements it using simpler XOR operations for local parity block generation. This extraction of the core parity function and its simplification using XOR operations reduces computational overhead while maintaining sufficient precision for distributed storage applications.
Solution Approach 2:
The patent uses XOR operations as a computationally inexpensive alternative to finite field operations for generating local parity blocks. While XOR operations provide slightly less mathematical rigor than finite field operations, they are computationally much cheaper and sufficient for the redundancy requirements of distributed storage, effectively using a simpler, disposable computational approach.
Data Source
AI summary
Provided are methods and systems for storing data using locally repairable multiple encoding. A data storage method may include generating n N×M encoding matrices, each including an M×M first matrix, an 1×M second matrix in which all of elements have a value of 1, and an M×M third matrix that is a symmetric matrix in which respective columns are configured by changing a sequence of elements from an element set; arranging the encoding matrices into a plurality of groups; generating a data block through the first matrix, a local parity block through the second matrix, and a global parity block through the third matrix by encoding source data of a first group with a first encoding matrix among the encoding matrices arranged into the first group; and merging the global parity block with a global parity block of a second encoding matrix that is different than the first encoding matrix.