Metadata Server Non-Persistent Storage Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed data storage systems in cloud computing environments face performance issues due to the slow nature of permanent storage used by metadata servers, which affects system durability and recovery processes, leading to delays in client operations during server failures.
Innovation Solution
A metadata server configured to store storage assignment mappings in non-persistent storage, allowing for efficient updates and client operations during recovery processes by rebuilding mappings from server storage assignments and managing replacement servers, enabling clients to continue operations before the recovery process is complete.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the metadata server stores the metadata table in permanent storage to ensure durability across reboots, then system reliability is improved, but system performance deteriorates due to the slow access speed of permanent storage
Solution Approach 1:
The patent segments the storage system into two distinct layers: permanent storage for durability and non-persistent storage for speed. The metadata table is stored in non-persistent storage (fast) while maintaining durability through replication to servers' persistent storage. This segmentation allows the metadata server to achieve both fast access and durability without relying solely on slow permanent storage.
Solution Approach 2:
The patent introduces servers as intermediaries that maintain copies of the metadata table in their own persistent storage. This intermediary layer allows the metadata server to use fast non-persistent storage while ensuring durability through the servers' persistent storage copies, resolving the contradiction between speed and reliability.
2Manufacturing precision
If the metadata server delays responding to client requests during the recovery process to ensure completeness, then data consistency is improved, but system performance deteriorates due to client waiting time
Solution Approach 1:
The patent implements preliminary action by having servers proactively send heartbeats to report their status and by allowing the metadata server to provide incomplete metadata tables during recovery. The system prepares for potential inconsistencies in advance and handles them through version comparison mechanisms, enabling clients to operate without waiting for complete recovery.
Solution Approach 2:
The patent makes the system dynamic by allowing the metadata server to transition between different operational states: providing full metadata when complete, and providing partial metadata during recovery. The system adapts its behavior based on recovery status, enabling clients to continue operations with available information rather than waiting statically for recovery completion.
3Productivity
If the metadata server uses non-persistent storage for the metadata table to improve access speed, then system performance is improved, but reliability deteriorates as data may be lost during server failures
Solution Approach 1:
The patent merges two storage approaches: the metadata server uses non-persistent storage for fast access while servers simultaneously maintain persistent storage copies for durability. This combination allows the system to achieve both the speed of non-persistent storage and the reliability of persistent storage through distributed replication.
Solution Approach 2:
The patent implements copying by replicating the metadata table from the metadata server's non-persistent storage to multiple servers' persistent storage. This copying mechanism ensures that even if the metadata server loses its data, reliable copies exist on servers, resolving the reliability concern while maintaining fast access through the primary non-persistent storage.
4Manufacturing precision
If the system waits for the complete recovery process before allowing client operations, then data consistency is improved, but productivity deteriorates due to system downtime
Solution Approach 1:
The patent makes the system dynamic by allowing it to operate in different modes: full consistency mode when recovery is complete, and partial availability mode during recovery. The metadata server dynamically adjusts what information it provides to clients based on recovery status, enabling continuous operation with progressively improving consistency as recovery progresses.
Solution Approach 2:
The patent implements preliminary action by establishing version tracking and comparison mechanisms before recovery completes. This allows the system to accept client operations during recovery and later resolve any inconsistencies through version comparison, rather than requiring complete recovery before operations resume.
Data Source
AI summary
A metadata server configured to maintain storage assignment mappings in non-persistent storage is described herein. The tract storage assignment mappings associate servers with storage assignments, the storage assignments representing the data stored on the servers. Responsive to a failure, the metadata server receives the storage assignments from the servers and rebuilds the storage assignment mappings from the storage assignments. The metadata server is also configured to enable clients to operate during a recovery process for a failed server by providing the storage assignment mappings to the clients during the recovery process. Also during the recovery process, the replacement server for the failed server conditionally overwrites stored data with other data received from other servers as part of the recovery process. The replacement server conditionally overwrites based on version information associated with the data and version information associated with the other data, the version information being associated with one or more versions of the storage assignment mappings.