Overlapped Local Reconstruction Codes for Lower-Cost Fragment Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional erasure coding systems in zone redundant storage are not optimized for common cases of storage node failures, leading to inefficient reconstruction costs and read efficiency, despite being effective for uncommon cases of zone failures.
Innovation Solution
The implementation of overlapped local reconstruction codes (OLRC) that use a subset of data fragments to generate parity fragments, optimizing reconstruction for common cases while maintaining fault tolerance for uncommon cases, with horizontal or interlaced placement strategies to enhance fault tolerance and reduce reconstruction costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional erasure coding is used to tolerate zone failures, then fault tolerance for uncommon cases is provided, but reconstruction cost is not optimized for common cases of storage node failures
Solution Approach 1:
The system segments data into data fragments and generates parity fragments using only a subset of data fragments (local reconstruction codes). This segmentation allows independent reconstruction of lost fragments using only locally available parity fragments, avoiding the need to read all data fragments across zones. The segmentation principle resolves the contradiction by enabling fast local reconstruction for common storage node failures while maintaining the ability to tolerate zone failures through the overall erasure coding structure.
Solution Approach 2:
The patent implements local reconstruction codes where each parity fragment is generated from a specific subset of data fragments within the same zone or local group. This local quality approach means that reconstruction can be performed using only locally available fragments for common failure cases, reducing reconstruction cost and time. Meanwhile, the global erasure coding structure maintains fault tolerance for uncommon zone failure cases by distributing data and parity fragments across multiple zones.
2Reliability
If all data fragments are used to generate parity fragments, then comprehensive fault tolerance is achieved, but storage overhead and reconstruction cost increase
Solution Approach 1:
The system extracts and uses only a subset of data fragments to generate each parity fragment, rather than using all data fragments. This extraction principle reduces the computational overhead for parity generation and decreases storage overhead by creating more efficient local reconstruction codes. The subset-based approach maintains sufficient fault tolerance for common storage node failures while reducing the quantity of computation and storage required.
Data Source
AI summary
Various embodiments, methods, and systems for erasure coding with overlapped local reconstruction codes, are provided. An erasure coding scheme can be defined based on Overlapped Local Reconstruction Codes (OLRC) that achieve high storage efficiency by providing fault tolerance properties that optimize reconstruction for common cases of failures while maintaining the reconstruction costs for uncommon case of failures. In operation, a data chunk is divided into data fragments. The data fragments correspond to zones. A plurality of parity fragments is computed using the data fragments. A parity fragment is computed using a subset of the data fragments. The plurality of parity fragments are assigned to the zones comprising the data fragments, where the data fragments and the plurality of parity fragments define overlapped local construction codes having a plurality of local groups. An unavailable data fragment is recoverable from at least two local groups from the plurality of local groups.