Dual Inode File System for Asynchronous Cache Overlay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file systems face inefficiencies due to the limitations of single inode associations, which can lead to lock conditions and metadata management challenges, especially when files are tiered to external storage, resulting in increased process expenses and inefficiencies.
Innovation Solution
Implementing a dual data stream approach with a cache overlay layer and a storage layer, where each file has a cache overlay inode and a storage layer inode, allowing asynchronous processing of file system operations and enabling data augmentation, compression, encryption, and tiering, thereby reducing the need for user-space metadata management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single inode is used to store file metadata and block locations, then the file system structure remains simple, but lock conditions occur during file operations and metadata management becomes inefficient when files are tiered to external storage
Solution Approach 1:
The patent divides the traditional single inode into two separate inodes: a cache overlay inode that stores file metadata and chunk state information, and a storage layer inode that stores block locations and external storage references. This segmentation allows independent management of cache and storage operations, eliminating lock conditions and improving operational efficiency.
2Loss of information
If metadata is stored in user space with kernel space inodes, then additional context information can be maintained, but process expenses and inefficiencies increase due to maintaining separate metadata stores
Solution Approach 1:
The patent merges the cache overlay inode and storage layer inode into a unified dual-inode structure within the kernel space, eliminating the need for separate user-space metadata stores. This integration maintains all necessary context information while improving metadata management efficiency through centralized kernel-space handling.
3Device complexity
If the inode size is limited to store basic file information, then the inode structure remains manageable, but additional metadata for tiered storage cannot be stored without external references
Solution Approach 1:
The patent adds a second inode dimension to the traditional single-inode structure. The storage layer inode provides an additional dimension for storing block locations and external storage references, while the cache overlay inode handles file metadata. This dimensional expansion allows comprehensive metadata storage without increasing the complexity of individual inodes.
Data Source
AI summary
Implementations are provided herein for having at least two data streams associated with each file in a file system. The first, a cache overlay layer, can store additional state information on a per block basis that details whether each individual block of file data within the cache overlay layer is clean, dirty, or indicates that a write back to the storage layer is in progress. The second, a storage layer, can be a use case defined repository that can transform data using data augmentation methods or store unmodified raw data in local storage. File system operations directed to the cache overlay layer can be processed asynchronously from file system operations directed to the storage layer.