Distributed Storage Reconstruction Using Effective Redundancy Priority
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face challenges in efficiently prioritizing data reconstruction during maintenance or failures, leading to inefficiencies in data availability and recovery processes.
Innovation Solution
A method and system that determine an effective redundancy value for each stripe in a distributed storage system, allowing for immediate reconstruction of high-availability chunks with low redundancy and delayed reconstruction of high-availability chunks with higher redundancy, based on system domain states and hierarchy levels, to prioritize data recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all inaccessible chunks are reconstructed immediately, then data availability is improved, but system resource consumption and reconstruction time increase
Solution Approach 1:
The patent applies local quality by differentiating reconstruction priorities based on stripe characteristics. High-availability chunks with low effective redundancy values are reconstructed immediately, while other chunks are reconstructed after a threshold period. This selective approach optimizes resource allocation by focusing reconstruction efforts on the most critical data segments.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring effective redundancy values and system domain states. The curator process dynamically adjusts reconstruction priorities based on current system conditions, ensuring that reconstruction resources are allocated to chunks that most need recovery while avoiding unnecessary immediate reconstruction of chunks with sufficient redundancy.
2Reliability
If high redundancy chunks are reconstructed immediately, then data safety is improved, but system efficiency deteriorates due to unnecessary reconstruction operations
Solution Approach 1:
The patent applies partial action by reconstructing only the necessary subset of chunks immediately. Specifically, only high-availability chunks with effective redundancy values below the threshold are reconstructed immediately, while chunks with sufficient redundancy are deferred. This avoids the excessive action of reconstructing all inaccessible chunks immediately, thereby maintaining system efficiency.
3Reliability
If data reconstruction prioritizes high-availability chunks with low redundancy, then data loss risk is reduced, but complexity of reconstruction management increases
Solution Approach 1:
The system manages complexity by dynamically calculating and using the effective redundancy value parameter to prioritize reconstruction. This quantitative metric allows the curator process to automatically determine reconstruction priorities based on stripe characteristics and system domain states, providing a systematic approach that reduces management complexity compared to ad-hoc decision-making.
Data Source
AI summary
A method of prioritizing data for recovery in a distributed storage system includes, for each stripe of a file having chunks, determining whether the stripe comprises high-availability chunks or low-availability chunks and determining an effective redundancy value for each stripe. The effective redundancy value is based on the chunks and any system domains associated with the corresponding stripe. The distributed storage system has a system hierarchy including system domains. Chunks of a stripe associated with a system domain in an active state are accessible, whereas chunks of a stripe associated with a system domain in an inactive state are inaccessible. The method also includes reconstructing substantially immediately inaccessible, high-availability chunks having an effective redundancy value less than a threshold effective redundancy value and reconstructing the inaccessible low-availability and other inaccessible high-availability chunks, after a threshold period of time.