Virtual Block Metadata for Partial File System Block Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In file systems that share and compress data blocks among files, there is a need for an efficient method to identify and utilize partially used data blocks to maximize storage efficiency by sharing these blocks among files that do not have identical data.
Innovation Solution
A method is implemented in a file server that compresses new data blocks and allocates them to partially used file system data blocks with sufficient free space, linking the compressed data to the appropriate files using virtual block metadata and sector bitmaps to manage sector allocation across multiple files.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data blocks are shared among files with identical content, then storage efficiency is improved through de-duplication, but storage space is wasted when blocks are partially used and cannot be fully utilized
Solution Approach 1:
The patent divides data blocks into smaller units called sectors (typically 512 bytes each), allowing flexible allocation within a block. The sector bitmap tracks which sectors are in use, enabling partial block utilization where only occupied sectors are allocated to specific files while remaining sectors can be allocated to other files or marked as free.
Solution Approach 2:
The patent implements dynamic block allocation where the allocation state of each sector can change over time. The sector bitmap is updated dynamically when files are written, deleted, or truncated, allowing the file system to adapt block usage patterns and reallocate sectors to maximize storage efficiency.
2Quantity of substance
If compression is applied to data blocks, then storage requirements are reduced, but the complexity of managing block allocation and metadata increases
Solution Approach 1:
The patent combines de-duplication and compression operations into a unified block-level processing pipeline. Both operations work on the same data blocks, and their effects are coordinated through the block allocation structure and sector bitmap, reducing overall storage requirements while managing complexity through integrated processing.
Solution Approach 2:
The sector bitmap acts as an intermediary data structure that tracks the allocation state of individual sectors within blocks. This mediator enables efficient management of complex allocation scenarios by providing a simple bit-based representation of which sectors are in use, by which files, and their allocation status through the allocation table.
3Volume of stationary object
If partial block allocation is implemented, then storage space utilization is improved, but the complexity of tracking and managing sector-level allocation increases
Solution Approach 1:
The patent uses a bitmap data structure where each bit represents the allocation state of a sector (similar to color coding). This visual/abstract representation simplifies tracking complexity by converting complex allocation information into simple bit patterns that can be efficiently stored, searched, and manipulated to determine sector usage and allocate space.
Solution Approach 2:
The patent uses pointers and references to track block and sector allocation relationships. Instead of storing complete allocation information for every sector, the system uses pointer structures that reference allocation tables and sector bitmaps, reducing the complexity of tracking while maintaining accurate allocation state information.
Data Source
AI summary
A file server has a data compression facility and also a snapshot copy facility or a data de-duplication facility that shares data blocks among files. Compression of the file data on a file system block basis leads to a problem of partially used file system data blocks when the data blocks are shared among files. This problem is solved by partial block allocation so that file system data blocks are shared among files that do not share identical data. Block pointers in the file mapping metadata point to virtual blocks representing the compressed data blocks, and associated virtual block metadata identifies portions of file system data blocks that store the compressed data. For example, a portion of a file system data block is identified by a sector bitmap, and the virtual block metadata also includes a reference count to indicate sharing of a compressed data block among files.