Multiple Protection Group Codes for Low-Overhead Data Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems face challenges in balancing data reliability, storage cost, and access overhead, with existing redundancy schemes offering limited trade-offs between these factors, particularly lacking in erasure-resilient coding techniques that use multiple protection groups to protect multiple data chunks.
Innovation Solution
The development of multiple protection group (MPG) erasure-resilient coding methods, which construct generic classes of erasure-resilient codes by grouping data chunks into multiple protection groups with at least one parity chunk per group, allowing for lower access overhead while maintaining data protection, including basic MPG codes derived from Maximum Distance Separable (MDS) codes and generalized MPG codes with a Maximally Recoverable property.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If (n,k) ERC schemes are used to reduce storage cost, then storage cost decreases, but access overhead increases significantly due to requiring mathematical inverse and k other chunks
Solution Approach 1:
The patent segments the protection mechanism into multiple independent protection groups, where each group protects a specific subset of data chunks. This segmentation allows the system to recover failed chunks by accessing only the corresponding protection group rather than requiring all k chunks plus parity, thereby reducing access overhead while maintaining storage efficiency
Solution Approach 2:
The patent introduces a hierarchical dimension to the protection structure by organizing data and parity chunks into multiple levels of protection groups. This dimensional organization enables selective access to specific protection groups based on failure patterns, reducing the access overhead from O(k) to O(log k) or better in certain scenarios
2Loss of time
If replication schemes are used to ensure data availability, then access overhead decreases, but storage cost increases due to replicating data multiple times
Solution Approach 1:
The patent applies local quality by creating protection groups with different properties - some groups have higher redundancy for critical data while others have lower redundancy. This allows the system to optimize storage cost by applying appropriate protection levels to different data chunks based on their importance and access patterns
3Loss of time
If multiple protection groups are used to reduce access overhead, then access overhead decreases, but system complexity increases due to multiple parity groups and subgroups
Solution Approach 1:
The patent implements dynamic protection group configuration where the system can adaptively select which protection groups to access based on the failure pattern and data access requirements. This dynamic approach reduces the effective complexity by only activating the necessary protection groups rather than managing all groups simultaneously
Data Source
AI summary
A multiple protection group (MPG) erasure-resilient coding method for constructing MPG codes for encoding and decoding data. The MPG codes constructed herein protect data chunks of data in multiple protection groups and subgroups. In general, the MPG erasure-resilient codes are constructed by locating data chunks into multiple protection groups and assigning at least one parity chunk to each protection group. Basic MPG codes are constructed from existing Maximum Distance Separable (MDS) codes by splitting at least some of the parity chunks into local parities for each of the multiple protection groups and projecting local parities onto each of the groups. Generalized MPG codes have a Maximally Recoverable property that can be used to determine whether an erasure pattern is recoverable or unrecoverable. Generalized MPG codes can recover any erasure pattern that is recoverable.