File Mapping Extents for Large File I/O Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large files in file systems require multiple levels of block mapping, leading to delayed access due to the need to read multiple tree levels, which increases I/O operations and reduces efficiency in storage management.
Innovation Solution
Implementing a file mapping structure using extents of contiguous indirect blocks, allowing for fine-grained allocation of data blocks and reducing the number of I/O operations required to obtain block mapping, while also supporting tree-based mapping for other files, enabling efficient read and write operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If tree structures (Indirect Block Tree or B-tree) are used for file mapping, then fine-grained allocation of data blocks is achieved, but the number of I/O operations increases due to multiple tree levels that must be read
Solution Approach 1:
The patent segments the file mapping structure into two distinct approaches: tree structures for files requiring fine-grained allocation, and extent-based mapping for large files. This segmentation allows each file type to use the most appropriate mapping method, avoiding the I/O overhead of multi-level trees for large files while preserving fine-grained allocation capabilities where needed.
Solution Approach 2:
The patent dynamically selects the file mapping method based on file characteristics. The file system can choose between tree-based mapping and extent-based mapping depending on the specific file's size and access patterns, making the system adaptive rather than static. This dynamic approach optimizes performance for each file individually.
2Device complexity
If large extents are used for storage, then the number of managed entities is reduced, but storage efficiency decreases due to inability to support thin provisioning, block sharing, or block de-duplication
Solution Approach 1:
The patent changes the parameter of extent size from fixed large extents to variable-sized extents. By allowing extents to be sized appropriately for each file and access pattern, the system can use larger extents to reduce management overhead when appropriate, while still supporting smaller extents for files that require thin provisioning, block sharing, or de-duplication operations.
3Adaptability or versatility
If multiple levels of block mapping are used, then large files can be addressed, but access time increases due to reading multiple tree levels from storage
Solution Approach 1:
The patent extracts the intermediate tree levels from the file mapping path for large files by using extent-based mapping. Instead of traversing multiple tree levels, the extent mapping provides a more direct path from the file inode to the data blocks, taking out the unnecessary intermediate lookup steps that slow down access.
4Stability of the object's composition
If tree structures are used for all files, then consistent mapping method is maintained, but performance degrades for large files due to unnecessary I/O operations
Solution Approach 1:
The patent applies local quality by allowing different mapping methods in different parts of the file system based on file characteristics. Rather than enforcing a single mapping method globally, the system applies tree-based mapping locally to files that benefit from fine-grained allocation while using extent-based mapping locally to large files that require performance optimization.
Data Source
AI summary
Large files in a file system are mapped by extents of contiguous indirect blocks in order to reduce the time for read or write access to the large files while allowing allocation of data blocks one at a time. The inode of a file includes an indication of whether file mapping metadata of the file includes either an extent of contiguous indirect blocks or a tree of blocks. In a preferred mapping scheme, an inode contains an array of block pointer entries, and each entry includes a flag indicating whether the block pointer is pointing to either an extent of indirect blocks or a tree of blocks. For sharing of data blocks between files, the block pointer fields of the indirect blocks each contain an ownership flag indicating whether the pointed-to block is shared or not.