Logical Mirroring in Memory-Based File Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mirroring solutions are inefficient for memory-aware file systems and hybrid storage systems, as they are designed for block-based storage and assume symmetry between nodes, leading to high latency and inefficiency when dealing with byte-addressable storage media and mixed storage types.
Innovation Solution
Implementing a system call-level mirroring process that mirrors byte-granular data and metadata in memory-based file systems, utilizing multiple network channels in parallel to reduce latency and enabling tiering of data to conserve expensive persistent memory resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If block-based storage mirroring solutions (DRBD, RAIN) are used for memory-based file systems, then data redundancy is achieved, but latency increases and efficiency decreases due to the mismatch between block-based and byte-addressable storage interfaces
Solution Approach 1:
The patent implements a file system level copy mechanism that operates on byte-granular data structures rather than block-based copying. The mirror file system creates and maintains a duplicate copy of the source file system's data and metadata in memory, using byte-addressable operations that match the native interface of memory-based storage, thereby avoiding the latency overhead of block interface emulation
Solution Approach 2:
The patent introduces a mirror file system layer that acts as an intermediary between the source file system and the persistent storage medium. This intermediary operates at the file system level rather than the block device level, enabling efficient byte-granular mirroring operations that are native to memory-based storage while maintaining data redundancy
2Productivity
If byte-granular mirroring is implemented in memory-based file systems, then network utilization is maximized, but the cost of persistent memory resources increases
Solution Approach 1:
The patent applies local quality by differentiating between hot data (frequently accessed) and cold data (infrequently accessed), storing hot data in expensive persistent memory to maximize network utilization and performance, while tiering cold data to cheaper storage media. This selective approach optimizes both network utilization and resource costs by matching data placement with access patterns
3Device complexity
If symmetric storage devices are assumed on both nodes, then implementation is simplified, but adaptability to hybrid storage systems is reduced
Solution Approach 1:
The patent embraces asymmetry by designing a mirroring system that explicitly supports heterogeneous storage configurations on source and mirror nodes. The system can handle cases where the source node uses fast persistent memory while the mirror node uses slower storage media, or where storage capacities differ. This asymmetric design achieves adaptability to hybrid storage systems while maintaining implementation feasibility through a unified file system level interface
4Ease of manufacture
If mirroring is implemented at the block layer, then existing RAID technologies can be used, but efficiency is reduced due to the need to wrap byte-addressable storage with a block interface
Solution Approach 1:
The patent inverts the traditional approach by implementing mirroring at the file system level rather than the block device level. Instead of wrapping byte-addressable storage with a block interface (the conventional approach), the patent creates a mirror file system that operates natively on byte-granular data structures and memory-based storage, eliminating the efficiency loss associated with interface emulation while maintaining implementation feasibility through standard file system operations
Data Source
AI summary
A method and system for logical mirroring between nodes includes maintaining a log of a state modifying operation received at a memory-based file system of an initiator node; writing attributes of the state modifying operation from the memory-based file system to a target node memory, and using the written attributes to process the state modifying operation at the target node according to the order represented by the log, to obtain logical mirroring between the initiator node and the target node.