Hybrid Backup for Large File Systems Using Metadata Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Very large file systems face challenges in restoring data efficiently, as the time required to completely restore a file system is proportional to its size, forcing administrators to choose between manual restoration of specific files or delaying access until all data are restored, which can impact business operations.
Innovation Solution
A hybrid backup method combining metadata image backup and traditional backup, where metadata is backed up to create a point-in-time image of the file system, and resident files are identified and backed up using a traditional backup system, allowing for optimized restoration of immutable files.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional complete file system backup is performed, then complete data protection is achieved, but restoration time becomes proportional to file system size causing business delay
Solution Approach 1:
The backup process is segmented into two independent parts: metadata image backup (capturing file system structure, permissions, and references) and data backup (capturing actual file contents). This segmentation allows the metadata to be backed up quickly without waiting for large data transfers, thereby reducing restoration time while maintaining complete data protection through the combined approach.
Solution Approach 2:
A metadata image copy is created that contains references to data blocks without copying the actual data contents. This copy can be restored quickly to重建 the file system structure, and only the necessary data blocks are then restored from the data backup, significantly reducing the time required for complete restoration while ensuring all data is protected.
2Loss of time
If manual restoration of specific files is performed, then access to important data is enabled quickly, but data loss risk increases for immutable files not yet migrated
Solution Approach 1:
The system performs preliminary identification of immutable files (files marked with immutable attribute indicating they cannot be modified) before the backup process. These files are then included in the data backup even though they have not been migrated to the archive storage tier, ensuring they are protected before any restoration operation occurs.
Solution Approach 2:
The metadata image acts as an intermediary that stores references to data blocks including immutable files. During restoration, the metadata image is restored first to重建 the file system structure, and then the data backup is used to restore the actual file contents, including those of immutable files that were identified beforehand. This intermediary approach ensures complete data protection without requiring manual intervention.
3Productivity
If HSM file system is used with metadata image backup, then backup efficiency is improved, but resident files not yet migrated are not fully protected
Solution Approach 1:
The system performs a partial data backup action by identifying and backing up only the resident files that are not yet migrated to the archive tier. This partial action ensures that these vulnerable files are included in the backup without requiring a complete scan and backup of the entire file system, thereby maintaining backup efficiency while improving protection of resident files.
Data Source
AI summary
A mechanism performs a metadata image backup of a hierarchical storage management (HSM) file system. Concurrent with performing the metadata image backup of the HSM file system, the mechanism identifies a set of resident files that are not yet stored off-line in the HSM file system. The mechanism identifies immutable files within the set of resident files to form an optimized set of resident files and passes the optimized set of resident files to a traditional backup system. The traditional backup system backs up file data for the set of resident files. Responsive to detecting an inability to restore a migrated managed state of a file, the mechanism unsets an immutable bit of the file attribute, restores file data for the file using the traditional backup system, and sets the immutable bit attribute of the file.