Deduplication File System Synthetic Restore Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deduplication file systems face challenges in restoring files efficiently due to the scattering of file data across the storage system, leading to lengthy restore times and high network bandwidth consumption, especially when dealing with large files.
Innovation Solution
The implementation of a deduplication system that generates virtual synthetic files during backups, allowing only unique regions to be retrieved during restores, using metadata and cached data on the client system to reduce bandwidth and disk accesses, and employing advanced prefetching techniques for improved performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire file is read during restore operations, then complete file restoration is achieved, but network bandwidth consumption and restore time increase significantly
Solution Approach 1:
The patent extracts and transmits only the unique data blocks that differ between the base file and the synthetic file, rather than transmitting the entire synthetic file. The deduplication system identifies unique regions and retrieves only those specific blocks from storage, significantly reducing network bandwidth consumption while maintaining restoration completeness.
Solution Approach 2:
The patent segments the file into multiple data blocks and identifies which specific blocks need to be retrieved during restore operations. By dividing the file into manageable segments and selectively retrieving only the unique portions, the system reduces overall bandwidth usage while ensuring complete file restoration.
2Quantity of substance
If file data is scattered throughout the storage system due to deduplication, then storage efficiency is improved, but restore time and compute resources increase
Solution Approach 1:
The patent performs preliminary actions during the backup phase by creating synthetic files that reference base files already stored in the deduplication system. This preliminary structuring of data with metadata information about unique blocks enables efficient retrieval during restore operations, reducing restore time despite the scattered nature of deduplicated data.
3Productivity
If metadata is generated during backup to track file regions, then restore efficiency is improved, but backup processing overhead increases
Solution Approach 1:
The patent creates a metadata copy or reference structure during backup that maps file regions to their storage locations. This metadata copying approach provides the information needed for efficient restore operations without requiring complex processing during actual file restoration, as the metadata serves as a pre-computed lookup guide.
Data Source
AI summary
During a current backup of a file system of a client, a determination is that a file in the current backup includes first data regions found in a file from a previous backup, and second data regions that are new. A virtual synthetic file is generated by writing the second data regions, and synthesizing the first data regions by referencing the file from the previous backup. A request received to restore the particular file includes identifications of other files still present at client. A determination is that the particular file corresponds to the virtual synthetic file and that the other files still present at client include the file from the previous backup. The second data regions are read to send to client. A complete version of the particular file is assembled at client using the file from the previous backup, still present at client, and the second data regions.