File System Metadata Region Segmentation for Backup Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current file-based backup systems experience poor deduplication ratios when storing metadata files across multiple iterations, leading to inefficient use of storage space due to unchanged attributes being stored alongside changing attributes.
Innovation Solution
The method involves segmenting metadata files into attribute regions based on the likelihood of attribute changeability, allowing for targeted deduplication and resegmentation to optimize storage efficiency by separating constant and variable attributes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If metadata files are stored without segmentation across multiple backup iterations, then storage space is consumed for all attributes including unchanged ones, but deduplication ratio remains poor and storage efficiency is reduced
Solution Approach 1:
The metadata file is segmented into multiple attribute regions based on changeability characteristics. Constant attributes (e.g., file name, size) are separated from variable attributes (e.g., modification time, content hash). This segmentation enables differential deduplication where constant regions are deduplicated across backup iterations while variable regions are updated, significantly improving deduplication ratios and storage efficiency
2Reliability
If all metadata attributes are treated uniformly in deduplication, then processing is simple, but deduplication ratio is poor due to mixing constant and changing attributes
Solution Approach 1:
Different attribute regions within the metadata file are assigned different quality characteristics based on their changeability. Constant attribute regions are marked for aggressive deduplication while variable attribute regions are marked for update. This local differentiation improves deduplication ratios without requiring complete reprocessing of entire metadata files, balancing reliability and complexity
3Loss of information
If metadata files are completely rewritten in each backup iteration, then all changes are captured, but storage space is wasted on unchanged attributes
Solution Approach 1:
Constant attributes that do not change between backup iterations are extracted and deduplicated separately from variable attributes. Only the variable attribute regions are rewritten in each backup iteration, while constant regions reference previously deduplicated versions. This extraction approach ensures complete metadata updating while eliminating waste of storage space on unchanged attributes
Data Source
AI summary
A method for managing file based backups (FBBs) includes obtaining, by a backup agent, a backup request for a FBB, in response to the backup request, generating a FBB, generating a FBB metadata file corresponding to the FBB, wherein the FBB metadata file comprises a set of attribute regions, performing, using the set of attribute regions, a deduplication on the FBB metadata file to obtain a deduplicated FBB metadata file, and storing the deduplicated FBB metadata file in a backup storage system.