Persistent Memory Tiering for Deduplicated File System Failover
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deduplicated file systems face challenges in achieving fast failover and recovery due to the need to access large amounts of metadata stored on hard disk drives, leading to lengthy recovery times that can result in client timeouts and disruptions to business operations.
Innovation Solution
The implementation of a deduplicated file system architecture that utilizes memory tiers, including huge pages and persistent memory devices, to store and recover metadata quickly, reducing the reliance on hard disk drives and minimizing recovery time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a deduplicated file system maintains a large amount of metadata to facilitate deduplication operations, then deduplication functionality is improved, but recovery time increases significantly
Solution Approach 1:
The patent segments the metadata storage into different tiers: a first memory tier for frequently accessed metadata and a second memory tier for less frequently accessed metadata. This segmentation allows the system to maintain comprehensive metadata for deduplication while reducing recovery time by loading only necessary metadata from the first tier during failover events.
Solution Approach 2:
The patent introduces a hierarchical memory structure with multiple tiers, transitioning from a single storage dimension to a multi-dimensional storage architecture. This dimensional change enables the system to balance between maintaining complete metadata for deduplication operations and reducing recovery time through selective metadata loading from appropriate tiers.
2Reliability
If the file system restarts and recovers metadata from hard disk drives, then data integrity is maintained, but recovery time becomes lengthy causing client timeouts
Solution Approach 1:
The patent implements preliminary action by pre-loading and caching frequently accessed metadata in the first memory tier before failures occur. During failover events, this pre-positioned metadata eliminates the need to read from slow hard disk drives, significantly reducing recovery time while maintaining data integrity through the persistent storage architecture.
Solution Approach 2:
The patent introduces memory tiers as intermediary storage between the hard disk drives and the active file system operations. This intermediary layer filters and provides frequently accessed metadata quickly, reducing the burden on hard disk drives during recovery operations and minimizing client timeouts.
3Quantity of substance
If metadata is stored on hard disk drives, then storage capacity is sufficient, but access speed during recovery is too slow
Solution Approach 1:
The patent segments metadata storage across different memory tiers with different performance characteristics. The first memory tier provides high-speed access for frequently accessed metadata, while the second memory tier and hard disk drives provide capacity for less frequently accessed metadata. This segmentation resolves the contradiction between storage capacity and access speed.
Solution Approach 2:
The patent applies local quality by assigning different storage locations to different types of metadata based on their access patterns. Frequently accessed metadata resides in the high-speed first memory tier, while less frequently accessed metadata is stored in the second memory tier or on disk. This localized optimization ensures fast access for critical metadata while maintaining sufficient overall capacity.
Data Source
AI summary
A memory tier including persistent memory (PMEM) devices is established in nodes of a cluster system having a deduplicated file system. At least a portion of metadata generated by the deduplicated file system is persisted to the memory tier. The portion of metadata includes an index of fingerprints corresponding to data segments stored by the deduplicated file system to a storage pool. A determination is made that an instance of the deduplicated file system has failed. A new instance of the deduplicated file system is started to recover file system services by loading the index of fingerprints from the memory tier.