Distributed Metadata Servers for Power Failure Resilience
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional distributed storage systems face limitations in scalability and power failure resilience, as they rely on centralized metadata management and limited numbers of metadata servers, which can lead to data loss and prolonged recovery times in the event of a power failure.
Innovation Solution
The system employs a distributed virtual file system architecture that utilizes non-volatile memory and a network of nodes to spread metadata functionality across many servers, allowing for load distribution and independent operation of buckets within the system, enabling efficient recovery and data integrity maintenance through journaling and memory scrubbing processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If centralized metadata management is used, then system simplicity is maintained, but scalability and power failure resilience deteriorate
Solution Approach 1:
The patent segments the centralized metadata management function into distributed metadata servers across multiple nodes. Each metadata server manages a portion of the metadata, allowing the system to scale horizontally while maintaining metadata availability during power failures through redundancy and distribution.
2Device complexity
If the number of metadata servers is limited, then system complexity is reduced, but scalability and recovery speed deteriorate
Solution Approach 1:
The system divides metadata management across multiple independent servers, allowing parallel recovery operations. When a power failure occurs, multiple metadata servers can simultaneously restore their portions of metadata, significantly accelerating overall recovery compared to a single centralized server.
Solution Approach 2:
The patent transitions from a single-dimension centralized metadata server to a multi-dimensional distributed architecture. This dimensional expansion allows the system to handle larger datasets and faster recovery by distributing metadata across spatial (multiple servers) and functional (different metadata types) dimensions.
3Device complexity
If centralized control is used, then coordination is simplified, but power failure impact increases
Solution Approach 1:
By segmenting metadata management into distributed servers, the patent eliminates the single point of failure inherent in centralized control. Each metadata server operates semi-independently, so a power failure affecting one node does not compromise the entire system's metadata availability.
Solution Approach 2:
The patent introduces distributed coordination mechanisms and metadata replication as intermediaries between nodes. These intermediaries enable automatic failover and recovery without requiring complex centralized coordination during power failure events, reducing both coordination complexity and failure impact.
Data Source
AI summary
A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. One or more of the computing devices and/or the storage devices may be used to rebuild data that may be lost due to a power failure.