Adaptive Memory File System Data Placement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file systems are not adaptive to changing workload requirements and do not efficiently manage data across different storage types, leading to suboptimal performance and cost inefficiencies in using both fast and slow storage devices.
Innovation Solution
Implementing a method that maintains data in two lists, Lr and Lf, based on access patterns, allowing pages to be moved between these lists and a pool of free pages, enabling dynamic allocation and protection of transparent pages to optimize data placement across fast and slow access storage devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional file systems use block devices (HDD/SSD) with volatile memory caching, then cost efficiency is improved, but access speed deteriorates due to the slow and cumbersome caching and memory map services
Solution Approach 1:
The patent extracts the VFS caching and memory map services from the traditional file system path, allowing PM-aware file systems to directly access PM without going through the slow VFS layer. This separation enables fast PM access while maintaining cost efficiency by using PM only for hot data.
Solution Approach 2:
The patent introduces a new caching layer architecture where volatile memory acts as an intermediary between PM and traditional storage. This mediator manages data movement between PM, volatile cache, and slow storage, optimizing both speed and cost by placing frequently accessed data in fast PM and less frequent data in slower, cheaper storage.
2Speed
If PM-aware file systems directly access PM avoiding VFS layer, then access speed is improved, but adaptability to changing workload requirements deteriorates because they assume entire data set resides in homogenous PM space
Solution Approach 1:
The patent segments the storage system into multiple layers: PM for hot data, volatile memory cache for frequently accessed data, and traditional slow storage for cold data. This segmentation allows the system to adapt to changing workloads by dynamically moving data between layers based on access patterns, while maintaining fast access speeds for hot data in PM.
Solution Approach 2:
The patent implements dynamic data placement and movement between storage layers based on real-time access patterns and workload changes. The system continuously monitors and adjusts which data resides in PM versus slower storage, enabling adaptability to changing workload requirements while maintaining optimal access speeds.
3Ease of operation
If volatile memory is used to support memory-mapped file I/O, then ease of operation is improved, but access speed deteriorates due to the slow and cumbersome caching services
Solution Approach 1:
The patent extracts the caching and memory map services from the VFS layer and implements them directly in PM-aware file systems. This removal eliminates the slow intermediary services while preserving the ease of operation through direct PM access using memory machine instructions and memory APIs.
Solution Approach 2:
The patent enables PM-aware file systems to self-manage their own caching and memory mapping without relying on the VFS layer. The file systems directly use volatile memory and PM with memory machine instructions, making the system easier to operate while achieving fast access speeds by eliminating the cumbersome VFS caching services.
Data Source
AI summary
Managing pages in a memory based file system by maintaining a memory into two lists, an Lr list and an Lf list, moving pages from the Lr list to the Lf list based on a repeated access pattern, and moving a page out of the Lr list or the Lf list arbitrarily, thereby enabling the two lists to re-grow according to current workload.