File System Metadata Compression for Sparse Indirect Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional file system metadata management leads to inefficient memory usage due to the flat layout of Indirect Blocks (IBs), which results in significant memory wastage, especially when dealing with large data sets.
Innovation Solution
A method and apparatus for compressing metadata in a file system by encoding storage addresses of data blocks into a first indirect block, allowing for more efficient memory utilization without affecting data reading performance, and dynamically allocating and dividing IBs as needed to manage memory space effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If conventional flat layout IBs are used to store data block addresses, then data access performance is maintained, but memory space is wasted significantly
Solution Approach 1:
The patent segments the flat IB structure into hierarchical levels. Instead of using a single large flat IB array, the system divides IBs into multiple levels where upper-level IBs store addresses of lower-level IBs. This segmentation allows sparse data block addresses to be represented more compactly, reducing memory waste while maintaining access capability through hierarchical navigation.
Solution Approach 2:
The patent transitions from a one-dimensional flat IB layout to a multi-dimensional hierarchical structure. By organizing IBs in levels with parent-child relationships, the system adds dimensional complexity that enables more efficient address encoding. The hierarchical arrangement allows sharing of upper-level IBs among multiple data files, reducing total memory consumption compared to the flat structure.
2Quantity of substance
If more IBs are allocated to handle large data sets, then data storage capacity increases, but memory consumption increases proportionally
Solution Approach 1:
The patent merges multiple data block address spaces into shared hierarchical IB structures. Upper-level IBs can be shared among multiple lower-level IBs, and ultimately among multiple data files. This merging eliminates redundant address storage that would occur in separate flat IB arrays for each data file, allowing the system to handle large data sets with reduced memory consumption.
Solution Approach 2:
The patent implements a nested hierarchical structure where upper-level IBs contain addresses of lower-level IBs, which in turn contain addresses of data blocks. This nesting allows the system to represent large address spaces efficiently by only allocating IBs for actually used address ranges, rather than pre-allocating entire flat IB arrays for maximum potential capacity.
3Loss of energy
If hierarchical IB structure is implemented to reduce memory usage, then memory efficiency improves, but IB access complexity increases
Solution Approach 1:
The patent performs preliminary organization of data block addresses into hierarchical IB structures during data allocation. By pre-building the hierarchical address structure as data is written, the system avoids the need for complex runtime computations to navigate flat IB arrays. The preliminary action of structuring addresses hierarchically simplifies subsequent access operations while maintaining memory efficiency.
Data Source
AI summary
Embodiments of the present disclosure relate to a method and an apparatus for compressing metadata in a file system. The method comprises, in response to receiving a first request for writing first data to a file, determining whether the first request is for an initial write to a storage area associated with a second indirect block in the first group of indirect blocks, the first group of indirect blocks at least including a first indirect block and the second indirect block. The method further comprises, in response to the initial write, allocating a first group of data blocks for writing the first data on a storage device. In addition, the method further comprises compressing the first group of indirect blocks by encoding a first group of storage addresses corresponding to the first group of data blocks into the first indirect block.