File System Storage Reclamation Using Version Sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern data storage systems face challenges in efficiently managing storage resources, particularly in virtualization environments with high dynamics, where thousands of storage objects and their snapshots require efficient block sharing and space reclamation, complicating the process of identifying and updating shared data blocks for optimal storage usage.
Innovation Solution
The method involves assigning storage objects and their snapshots to version sets, where unchanged blocks are shared across files within the same version set, using block metadata to identify and update pointers for relocated blocks, thereby simplifying the reclamation of storage space by limiting the scope of examination to files within the same version set.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If block sharing is used to avoid unnecessary duplication of data, then storage efficiency is improved, but the complexity of identifying and updating shared blocks increases
Solution Approach 1:
The file system is segmented into version sets, where each version set contains a primary object and its snapshots. This segmentation allows the system to limit the scope of block sharing and pointer updates to within a single version set, reducing the complexity of managing shared blocks while maintaining storage efficiency through block sharing within each segmented unit.
Solution Approach 2:
A version set identifier is introduced as an intermediary metadata field in the block metadata structure. This identifier acts as a mediator that links blocks to their specific version set, enabling efficient identification of shared blocks without requiring examination of all files in the file system. The intermediary simplifies the process of tracking and updating shared blocks.
2Quantity of substance
If the number of primary objects and snapshots increases, then the storage system can handle more data, but the task of identifying shared blocks for space reclamation grows in size
Solution Approach 1:
By segmenting the file system into version sets and limiting block sharing to within each version set, the system can handle a large total number of objects while keeping the identification task manageable. When reclaiming space, the system only needs to examine blocks within the specific version set being processed, not across all version sets, thus reducing the time required despite handling many objects.
Solution Approach 2:
The version set identifier in block metadata serves as an intermediary that enables rapid filtering and identification of relevant shared blocks. When a block needs to be moved during space reclamation, the system uses the version set identifier to quickly identify all files within that version set that may share the block, avoiding a full file system scan and reducing identification time.
3Productivity
If live data blocks are moved around to free up larger contiguous chunks of storage space, then space reclamation efficiency is improved, but it is necessary to identify and update pointers of all objects sharing such blocks
Solution Approach 1:
The segmentation into version sets confines the pointer identification and update task to a limited scope within each version set. When live data blocks are moved to free contiguous space, the system only needs to track and update pointers for files within that specific version set, making the reclamation process more efficient and manageable despite the complexity of block moving operations.
Solution Approach 2:
The version set identifier acts as an intermediary that facilitates efficient pointer management during block moving operations. By including this identifier in block metadata, the system can quickly determine which files share a block that is being moved, update only the necessary pointers within the relevant version set, and maintain data integrity without examining the entire file system.
Data Source
AI summary
Reclaiming storage from a file system hosting of storage objects includes assigning objects and respective files to version sets, each version set containing a primary object and respective snapshots. Each block of storage is associated with the version set of the file(s) that contain the block. Blocks residing in an extent that is to be reclaimed are identified, and for each block (i) an offset is obtained that specifies a location of the block in a file, (ii) files whose objects are assigned to the version set associated with the block are identified, (iii) a block pointer at the location in each of the identified underlying files is examined, and for each file whose block pointer points to the block, updating the block pointer to point to a new block location outside of the extent, to which the block is moved.