Erasure Code Stripe Merging with Hash-Based Parity Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for converting narrow erasure code stripes to wide stripes result in significant bandwidth overhead due to data chunk relocation and parity chunk regeneration.
Innovation Solution
A stripe merging method and system based on erasure codes that preprocesses erasure code stripes, determines alignment, and applies a greedy strategy to select and merge stripes with low merging overhead, using hash tables to optimize the process and reduce unnecessary computing overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If erasure code stripes are re-encoded and merged into wide stripes using traditional methods, then wide stripe generation is achieved, but a large amount of bandwidth overhead is triggered due to data chunk relocation and parity chunk regeneration
Solution Approach 1:
The patent applies preliminary action by pre-processing erasure code stripes before merging, including determining storage locations of parity chunks and establishing hash tables in advance. This preparation work is done before the actual merging process, allowing for optimized stripe selection and reduced bandwidth overhead during the merge operation itself.
Solution Approach 2:
The patent introduces hash tables as an intermediary data structure to manage and track parity chunk locations across multiple erasure code stripes. This intermediary mechanism enables efficient identification of aligned stripes and facilitates the merging process without requiring extensive data relocation, thereby reducing bandwidth overhead.
2Ease of operation
If traditional stripe merging methods are used, then merging functionality is provided, but considerable computing overhead is incurred during data relocation and parity regeneration
Solution Approach 1:
The system performs preliminary processing of erasure code stripes by pre-determining parity chunk storage locations and building hash tables that map stripe identifiers to their parity locations. This advance preparation eliminates the need for complex real-time computations during the merging operation, significantly reducing computing overhead.
Solution Approach 2:
The hash table structure enables the system to self-determine stripe alignment and merging compatibility without requiring extensive external computation. The pre-organized data structures allow the merging process to automatically identify suitable stripe pairs and execute merges with minimal computational intervention.
Data Source
AI summary
A stripe merging method and system based on erasure codes are provided. A StripeMerge-P algorithm is used first to determine alignment information of parity chunks of erasure code stripes based on a preprocessed hash table. Through a greedy strategy, erasure code stripe pairs to be merged are selected for merging. Through the hash table, location information of the parity chunks is directly looked up, so that no additional computing overhead is required, and the overhead of selecting and merging the erasure code stripe pairs is further reduced through the combination with the greedy strategy.