Erasure Code Reconfiguration by Rearranging Encoded Data Segments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in dynamically changing erasure correction coding configurations as they scale, requiring recalculation of encoded data when adding or removing storage devices, which is computationally expensive and inefficient.
Innovation Solution
The introduction of a pliable encoded data management engine that allows for reconfiguration of erasure correction codes without recalculating the entire dataset, by rearranging and reorganizing encoded data blocks across failure domains, enabling efficient adaptation to changes in the number of storage devices without significant computational overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If erasure correction coding configuration is changed by recalculating encoded data, then data integrity and redundancy are maintained, but computational cost and time increase significantly
Solution Approach 1:
The patent pre-calculates and stores multiple encoded data sets corresponding to different erasure correction coding configurations. When a reconfiguration is needed, the system simply switches to a pre-computed encoded data set rather than recalculating from scratch, thereby maintaining data integrity while significantly reducing reconfiguration time.
Solution Approach 2:
The patent implements a dynamic encoding configuration that allows the erasure correction coding parameters to be changed on-the-fly. The system maintains multiple encoded data sets with different coding configurations and can dynamically switch between them based on storage device availability, enabling flexible adaptation without full recalculation.
2Reliability
If erasure correction coding configuration is changed by recalculating encoded data, then redundancy is maintained, but storage system scalability is reduced
Solution Approach 1:
The system pre-computes and stores multiple encoded data sets corresponding to different erasure correction coding configurations (e.g., different numbers of data blocks and parity blocks). When the storage system scales by adding or removing devices, the controller can switch to a pre-computed configuration that matches the new system size, maintaining data redundancy without requiring full recalculation and enabling seamless scalability.
Solution Approach 2:
The patent changes the parameters of the erasure correction coding (such as the number of data blocks n and parity blocks m) by selecting from pre-computed encoded data sets. This allows the system to adapt to different scales while maintaining optimal redundancy ratios, as each pre-computed set is optimized for specific coding parameters.
3Productivity
If multiple encoded data sets are maintained for different configurations, then reconfiguration efficiency is improved, but storage overhead increases
Solution Approach 1:
The patent applies different encoding configurations to different portions of the stored data. Instead of maintaining complete redundant copies of all encoded data sets, the system divides the data into segments and applies different erasure correction coding configurations to different segments, thereby reducing overall storage overhead while maintaining the ability to reconfigure efficiently.
Solution Approach 2:
The system maintains multiple encoded data sets with different erasure correction configurations, but discards the less frequently used configurations when they are not needed. When a reconfiguration is required, the system recovers by switching to the appropriate pre-computed encoded data set or regenerating it from the original data if necessary, thereby balancing storage overhead with reconfiguration efficiency.
Data Source
AI summary
In some examples, a system groups a plurality of blocks of encoded data into first segments, each first segment comprising multiple blocks of the plurality of blocks, and the encoded data being based on application of erasure correction coding on input data. The system stores the first segments in respective failure domains of a first quantity of failure domains in a first erasure correction configuration, where the first erasure correction configuration enables error recovery in response to lost encoded data in a specified number of failure domains. The system changes from the first erasure correction configuration to a second erasure correction configuration by reorganizing the first segments into second segments, each second segment comprising a different quantity of blocks of encoded data than a first segment.