Distributed Chunk Recovery in Asymmetric Clustering File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing asymmetric clustering file systems rely on metadata servers for data recovery, which is inefficient and can be bottlenecked by the failure of a single data server, leading to suboptimal recovery processes.
Innovation Solution
The method involves distributing data recovery across multiple data servers, where each server group processes and recovers erroneous chunks by requesting necessary data from other server groups, utilizing a chunk list and recovery information structure to manage the process, and notifying the metadata server upon completion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data recovery is centralized at the metadata server, then the recovery process can be coordinated centrally, but the recovery speed is slow and the metadata server becomes a bottleneck
Solution Approach 1:
The patent divides the centralized recovery function into distributed recovery units across multiple data servers. Each data server group independently performs recovery operations for failed servers within its group, segmenting the monolithic recovery process into parallel distributed tasks that eliminate the metadata server bottleneck while maintaining coordinated recovery through group-based organization
2Device complexity
If a single metadata server handles all recovery operations, then recovery coordination is simplified, but the system cannot scale efficiently with multiple data servers
Solution Approach 1:
The patent segments the recovery coordination function by organizing data servers into groups with designated recovery servers. Each recovery server coordinates recovery for its group independently, dividing the coordination complexity into manageable segments while enabling parallel recovery operations across groups, thus scaling efficiency without proportionally increasing coordination complexity
Solution Approach 2:
The patent introduces a hierarchical dimension to recovery coordination by adding group-level organization above individual server level. Recovery servers operate at the group level while metadata servers maintain global oversight, creating a two-dimensional coordination structure that distributes complexity vertically rather than concentrating it at a single point
3Quantity of substance
If parity-based distributed storage is used, then storage efficiency is improved compared to mirroring, but data recovery requires reading multiple data and parity chunks
Solution Approach 1:
The patent segments the recovery process into parallel operations where multiple data servers simultaneously retrieve and process different parity chunks needed for reconstruction. By dividing the recovery workflow into independent parallel tasks executed by different servers, the system maintains parity-based storage efficiency while reducing total recovery time through concurrent processing
Data Source
AI summary
The present invention relates to an apparatus and method of performing an error recovery process in an asymmetric clustering file system that has higher efficiency in data recovery when a data server in an asymmetric clustering file system fails than a method of processing the data recovery in the metadata server. The present invention includes receiving a chunk list requiring recovery by a data server included in the other data server groups than a data server group including a failed data server among a plurality of data server groups, requesting chunk data necessary for recovering an erroneous chunk from a data server in the other data server groups to the other data servers than the failed data server in the data server group, and recovering the erroneous chunk based on the chunk data by the data server in the other data server group.