Multiphase Deduplication Change Tracking for Backup Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing backup technologies face inefficiencies and redundancy issues when backing up multiple similar storages, leading to increased storage size and bandwidth overhead due to redundant data, particularly in image backups that do not account for file system metadata and file locations.
Innovation Solution
Implementing multiphase deduplication methods that track changes in allocated blocks between snapshots, using a change tracking phase to temporarily store and hash changed blocks, and an analysis phase to identify and eliminate duplicates, thereby reducing redundancy and fragmentation in incremental backups.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If image backup is used to back up multiple similar storages, then backup speed is improved, but storage size increases due to redundant data
Solution Approach 1:
The patent combines multiple similar storages into a single consolidated backup by identifying and merging identical blocks across different source storages. The deduplication process merges duplicate blocks into single stored copies, reducing overall storage size while maintaining backup speed through efficient block-level operations.
Solution Approach 2:
The patent discards redundant duplicate blocks during the backup process by storing only unique blocks. When restoring, the system recovers the original data structure by referencing the consolidated backup and reconstructing the multiple storages from the deduplicated block set.
2Speed
If image backup is used to back up multiple similar storages, then backup speed is improved, but bandwidth overhead increases due to redundant data
Solution Approach 1:
The patent merges identical blocks from multiple source storages into a single consolidated backup, eliminating redundant data transmission. This combining approach reduces the total volume of data that needs to be transported across the network, thereby reducing bandwidth overhead while maintaining efficient backup operations.
Solution Approach 2:
The system discards redundant block copies during the backup process, transmitting only unique blocks across the network. This selective transmission reduces bandwidth consumption, and the original multi-storage structure is recovered during restoration by reconstructing the data layout from the deduplicated backup.
3Ease of repair
If file backup is used, then individual file restoration is improved, but backup overhead increases due to file system tracking
Solution Approach 1:
The patent segments the backup process into block-level operations rather than file-level operations. By working with fixed-size blocks independent of file system structures, the system reduces the complexity of tracking and managing individual files while maintaining the ability to restore specific data blocks efficiently.
Solution Approach 2:
The patent extracts the backup operation from file system dependencies by performing deduplication and backup at the block level without requiring file system metadata tracking. This extraction reduces backup overhead by eliminating the need to track file locations, permissions, and other file system attributes.
Data Source
AI summary
Change tracking for multiphase deduplication. In one example embodiment, a method of tracking changes to a source storage of a source system for multiphase deduplication includes a change tracking phase that includes performing various steps for only allocated blocks in the source storage that are changed between a prior point in time and a subsequent point in time. These steps include temporarily storing a copy of the changed block in a volatile memory of the source system prior to writing the changed block to the source storage, performing a hash function only once on the copy of the changed block, while the copy is temporarily stored in a volatile memory of the source system, to calculate a hash value, writing the changed block to the source storage, and tracking, in a change log, a location in the source storage of the changed block and the corresponding hash value.