Sparse File Storage Using Fill Counts and Segment Trees
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file systems face inefficiencies in storing and managing sparse files, which contain contiguous ranges of zero bytes, leading to wasted storage space and difficulties in locating holes within these files.
Innovation Solution
The method involves using fill counts to store and manage sparse files by referencing data blocks and indirect blocks within a storage pool, allowing for efficient allocation and location of holes through a hierarchical data structure and fill count tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If sparse files are stored using traditional file system methods, then the file can be created and accessed, but storage space is wasted due to allocation of blocks for zero-byte regions
Solution Approach 1:
The patent segments the file's address space into regions that are either allocated (containing data) or unallocated (containing zeros). Instead of allocating physical blocks for every byte of the file, the system divides the virtual address space and only allocates storage for non-zero regions, thereby eliminating waste of storage space while maintaining a manageable structure through the segment tree organization.
Solution Approach 2:
The patent introduces a hierarchical dimension to file storage by implementing a segment tree structure that operates at the address space level rather than the block level. This dimensional shift allows the system to track allocated and unallocated regions abstractly, resolving the contradiction between storage efficiency and structural complexity by adding a layer of virtualization.
2Productivity
If traditional file systems allocate blocks for entire file ranges, then data access is straightforward, but locating holes (zero-byte regions) becomes inefficient
Solution Approach 1:
The patent performs preliminary organization of the address space into a segment tree structure during file creation or modification operations. By pre-establishing the hierarchical segmentation of allocated and unallocated regions, the system enables rapid hole location later without requiring time-consuming scans, thus improving productivity while the initial setup time is incurred only once.
Solution Approach 2:
The segment tree acts as an intermediary data structure between the file's virtual address space and the physical storage blocks. This intermediary layer provides efficient pathways to identify hole regions by traversing the tree and locating unallocated segments, dramatically reducing the time required to find holes compared to traditional methods that would require scanning allocated blocks.
3Quantity of substance
If the file system manages every byte of sparse files, then complete data integrity is maintained, but storage overhead increases significantly
Solution Approach 1:
The patent extracts the management of zero-byte regions from the traditional file system allocation mechanism. By identifying and separating unallocated segments (holes) from allocated data blocks through the segment tree, the system eliminates storage overhead for zero bytes while maintaining data integrity through selective allocation—only regions containing actual data are allocated and managed, reducing overhead without compromising reliability.
Data Source
AI summary
A method for storing a first block and a second block, the includes storing the first block and the second block in a storage pool, and storing a third block in the storage pool, wherein the third block references the first block and second block and comprises a first fill count and a second fill count, wherein the first fill count corresponds to one selected from the group consisting of a number of data blocks directly referenced by the first block, a number of data blocks indirectly referenced by the first block, if the first block is an indirect block, wherein the first fill count is one if the first block is a data block.