Lazy Virtual Filesystem Instantiation via Metadata Proxies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Shared computing environments face memory footprint issues due to the need for numerous virtual filesystems, which consume significant memory resources and lead to decreased compute density and hardware utilization.
Innovation Solution
Implementing lazy virtual filesystem instantiation and caching, where only actively used filesystems are stored in a specialized cache, and metadata-based filesystem proxies are used to respond to requests, instantiating filesystems only when necessary, thereby conserving memory and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If numerous virtual filesystems are instantiated in shared computing environments, then file access functionality is provided for multiple guests, but host memory usage increases significantly
Solution Approach 1:
Filesystem proxies are pre-created and stored in the proxy cache with all necessary metadata (file names, directories, permissions, inode numbers) before actual filesystem instantiation. This preliminary preparation allows the system to respond to file access requests without immediately instantiating the full filesystem, thereby reducing memory usage while maintaining file access functionality.
Solution Approach 2:
A filesystem proxy acts as an intermediary between the guest and the actual filesystem. The proxy contains metadata that enables file location and access control without requiring the entire filesystem to be loaded into memory. When a file access request occurs, the proxy mediates by providing file locations and managing permissions, while the actual filesystem remains instantiated only when necessary.
2Speed
If all virtual filesystems are kept in memory for fast access, then file access speed is improved, but compute density and hardware utilization decrease
Solution Approach 1:
The system implements local quality by providing fast metadata access through filesystem proxies stored in memory, while allowing the actual filesystem data to reside in slower storage media. The proxy cache contains only essential metadata (file names, directories, permissions, inode numbers) that is frequently accessed, while the bulk filesystem data remains on disk or other persistent storage, achieving a balance between speed and memory utilization.
3Quantity of substance
If filesystems are instantiated only when needed, then memory usage is reduced, but filesystem instantiation overhead occurs
Solution Approach 1:
The invention extracts the essential filesystem metadata (file names, directories, permissions, inode numbers) from the complete filesystem structure and places it in the proxy cache. This extraction allows the system to handle file access requests using only the lightweight proxy data structure, avoiding the need to instantiate the entire filesystem in memory. The full filesystem is instantiated only when actual file content access is required, minimizing both memory usage and instantiation overhead.
Data Source
AI summary
A host with a memory, which stores a guest, a filesystem cache, and a proxy cache, has a filesystem manager executing on a processor that receives a first request to locate a file in a first filesystem. In response to receiving the first request, a filesystem proxy in the proxy cache associated with the first filesystem is queried. The first request is responded to based on metadata retrieved from the filesystem proxy. A second request is received to modify the file. In response to receiving the second request, the first filesystem is instantiated in the filesystem cache, where a second filesystem is evicted from the filesystem cache to reclaim storage capacity for the first filesystem. The file in the first filesystem is modified.