Persistent Memory Tier Mapping for Fresh Data Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage file systems fail to leverage the benefits of persistent memory due to its byte-addressable nature, leading to stale or missing data issues when managing data operations across both persistent memory and file system tiers.
Innovation Solution
Implement a persistent memory tier with a tailored file system that frames blocks of up-to-date data from the persistent memory tier to the file system, allowing data management operations to operate upon up-to-date data across both tiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a storage file system uses traditional block-addressable storage media, then data management operations can be performed, but data operations across persistent memory and file system tiers result in stale or missing data issues
Solution Approach 1:
The storage system is divided into separate tiers: a persistent memory tier for high-performance byte-addressable storage and a file system tier for traditional block-addressable storage. This segmentation allows each tier to be optimized for its specific characteristics while maintaining overall system reliability through coordinated data management across both tiers.
Solution Approach 2:
A data management layer acts as an intermediary between the persistent memory tier and file system tier. This intermediary coordinates data operations, ensures consistency, and manages the complexity of interacting with both storage types, preventing stale or missing data issues by mediating data access and synchronization.
2Speed
If persistent memory is used to provide lower latency, then access speed improves, but the byte-addressable nature creates complexity in data management operations
Solution Approach 1:
The system segments storage operations into byte-addressable operations for persistent memory (providing high speed) and block-addressable operations for the file system tier. This allows the high-speed persistent memory to be leveraged for latency-sensitive operations while the file system handles structured data management, dividing complexity across manageable layers.
Solution Approach 2:
The data management layer changes the addressing parameters and access patterns to bridge persistent memory's byte-addressable interface with the block-addressable file system. By transforming and coordinating these parameter differences, the system achieves high-speed access without overwhelming complexity in data management operations.
3Quantity of substance
If data is stored in both persistent memory and file system tier, then data redundancy and accessibility improve, but determining which data is up-to-date becomes problematic
Solution Approach 1:
The data management layer implements feedback mechanisms to track data freshness and synchronization status between the persistent memory tier and file system tier. This feedback enables the system to identify which data is up-to-date and which requires synchronization, ensuring data freshness while maintaining redundancy for availability.
Solution Approach 2:
The system performs preliminary actions to establish and maintain data synchronization metadata before operations are needed. By pre-tracking data freshness and sync status, the system can quickly determine which data is up-to-date during operations, avoiding information loss while maintaining both redundancy and availability.
Data Source
AI summary
Techniques are provided for data management across a persistent memory tier and a file system tier. A block within a persistent memory tier of a node is determined to have up-to-date data compared to a corresponding block within a file system tier of the node. The corresponding block may be marked as a dirty block within the file system tier. Location information of a location of the block within the persistent memory tier is encoded into a container associated with the corresponding block. In response to receiving a read operation, the location information is obtained from the container. The up-to-date data is retrieved from the block within the persistent memory tier using the location information for processing the read operation.