NTFS Deduplication Volume Replication via Chunk Reference Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data protection methods for NTFS deduplication volumes face challenges in efficiently synchronizing and replicating deduplicated data across systems, particularly in minimizing bandwidth usage and maintaining deduplication integrity during data transfer and storage.
Innovation Solution
The system implements volume level and file level synchronization, as well as replication, by retrieving and sending deduplicated files and chunk stores without rehydration, and enabling deduplication on replicas using matching settings, ensuring consistent deduplication across master and replica systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deduplicated files are rehydrated before transfer, then file integrity is maintained, but bandwidth usage increases
Solution Approach 1:
The patent transfers deduplicated file references (pointers to chunk IDs) instead of rehydrating complete files before transfer. This copying approach maintains file integrity through reference validation while minimizing bandwidth consumption by transmitting only metadata rather than full file contents.
Solution Approach 2:
The patent extracts only the necessary deduplication metadata (chunk IDs, file references, deduplication settings) from the source system and transfers this extracted information to the replica. This extraction eliminates the need to transfer redundant file data while preserving the essential information needed for deduplication integrity.
2Productivity
If deduplication settings are not synchronized, then transfer speed increases, but deduplication consistency deteriorates
Solution Approach 1:
The patent performs preliminary synchronization of deduplication settings (chunk size, hash algorithms, retention policies) from the source system to the replica before initiating file transfer. This preliminary configuration ensures that both systems use identical deduplication parameters, guaranteeing consistency without requiring repeated verification during the transfer process.
3Reliability
If complete files are transferred, then data consistency is ensured, but storage requirements increase
Solution Approach 1:
The patent transfers deduplication metadata (chunk references, file pointers, deduplication settings) instead of complete file contents. This copying strategy ensures data consistency by maintaining accurate references to the original data chunks while reducing storage requirements on the replica system, as the actual file data can be retrieved from the source or shared storage.
Solution Approach 2:
The patent merges the deduplication storage architecture between source and replica systems, allowing both systems to share common data chunks. By combining their deduplication databases and chunk stores, the systems maintain data consistency while eliminating redundant storage of identical file contents across both locations.
Data Source
AI summary
Various embodiments illustrated and described herein include systems, methods, and computer program products to protect NTFS deduplicated volumes. Some embodiments select a data protection mode from various modes to protect deduplicated volumes. Some data protection modes retrieve the deduplicated files and associated data chunks without rehydrating the files and send them to a replica. Some data protection modes rehydrate deduplicated files as they are retrieved, send the rehydrated files to the replica, and turn on deduplication at the replica so the files can be deduplicated on the replica. Deduplication settings can also be transferred to the replica so that deduplication on the replica mimics deduplication on the source. Some data protection modes replicate the deduplicated files to the replica.