Distributed Storage Recovery Priority by Effective Redundancy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face inefficiencies in data recovery during maintenance or failures, as they lack a method to prioritize data reconstruction based on availability and redundancy, leading to potential data loss and operational disruptions.
Innovation Solution
A method is introduced to determine an effective redundancy value for each stripe in a distributed storage system, allowing for immediate reconstruction of high-availability chunks and delayed reconstruction of low-availability chunks, based on system domain states and hierarchy levels, ensuring quicker recovery of critical data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data reconstruction is performed uniformly for all chunks in distributed storage systems, then data integrity is maintained, but system efficiency deteriorates due to unnecessary reconstruction of low-risk data during maintenance or failures
Solution Approach 1:
The patent applies local quality by differentiating reconstruction priority based on the specific characteristics of each stripe. The system calculates an effective redundancy value for each individual stripe considering its unique chunk distribution across system domains, rather than applying a uniform reconstruction policy to all data. This allows high-risk stripes to be reconstructed immediately while low-risk stripes are deferred, optimizing system efficiency without compromising data integrity.
Solution Approach 2:
The patent changes the parameter of reconstruction timing from a uniform fixed policy to a dynamic variable based on effective redundancy value. By introducing the effective redundancy value as a calculated parameter that reflects the actual risk level of each stripe, the system can adjust reconstruction priority dynamically. This parameter change enables the system to respond adaptively to different failure scenarios and maintenance schedules.
2Reliability
If immediate reconstruction is performed for all inaccessible chunks, then data availability is improved, but resource consumption increases unnecessarily for chunks with sufficient redundancy
Solution Approach 1:
The patent applies partial action by performing immediate reconstruction only for high-risk stripes where it is truly necessary, rather than reconstructing all inaccessible chunks immediately. The effective redundancy value calculation identifies which stripes require urgent attention versus those that can safely wait. This partial action approach maintains data availability for critical data while avoiding unnecessary resource consumption on redundant reconstruction operations.
3Measurement precision
If the system tracks system domain states for redundancy calculation, then reconstruction priority accuracy is improved, but system complexity increases due to hierarchical domain tracking
Solution Approach 1:
The patent applies segmentation by dividing the storage system into a hierarchical structure of system domains (e.g., rack domains, node domains). Each domain is tracked independently with its active/inactive state, allowing the effective redundancy value to be calculated based on the specific domain hierarchy. This segmentation makes the complexity manageable by organizing tracking at discrete hierarchical levels rather than requiring continuous monitoring of all individual components.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
A method (600) of prioritizing data (312) for recovery in a distributed storage system (100) includes, for each stripe (320) of a file (310) having chunks (330), determining whether the stripe comprises high-availability chunks (330a) or low-availability chunks (330b) and determining an effective redundancy value (ER) for each stripe. The effective redundancy value is based on the chunks and any system domains (402) associated with the corresponding stripe. The distributed storage system has a system hierarchy (400) including system domains. Chunks of a stripe associated with a system domain in an active state are accessible, whereas chunks of a stripe associated with a system domain in an inactive state are inaccessible. The method also includes reconstructing substantially immediately inaccessible, high-availability chunks having an effective redundancy value less than a threshold effective redundancy (ERT) value and reconstructing the inaccessible low-availability and other inaccessible high-availability chunks, after a threshold period of time.