Deduplication Layer for Asynchronous Remote Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current remote replication techniques in storage arrays inefficiently consume network bandwidth by unnecessarily replicating deduplicated data, leading to high storage overhead and costs.
Innovation Solution
Implementing a deduplication layer on remote replication protocols that identifies and removes duplicate data at a logical unit number (LUN) level, using deduplication fingerprints to replicate only unique data, thereby reducing network bandwidth consumption and storage requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional remote replication techniques are used to replicate all data, then data redundancy and storage overhead increase, but network bandwidth consumption increases unnecessarily
Solution Approach 1:
The patent extracts and removes duplicate data from the replication process by implementing a deduplication layer that identifies and eliminates redundant data blocks before transmission. Only unique data blocks are replicated to the remote system, thereby maintaining data redundancy for reliability while significantly reducing network bandwidth consumption.
Solution Approach 2:
The patent changes the parameter of data representation by using deduplication fingerprints (hash values) instead of replicating actual data blocks. This parameter transformation allows the system to identify and compare data uniqueness efficiently, enabling selective replication based on fingerprint matches rather than transmitting redundant data.
2Reliability
If all data is replicated to remote system, then storage overhead increases, but storage capacity requirements increase
Solution Approach 1:
The patent extracts duplicate data blocks from the replication stream by comparing deduplication fingerprints between source and remote systems. When a fingerprint match is found, the corresponding data block is identified as redundant and excluded from replication, thereby maintaining backup reliability while reducing storage overhead at the remote system.
Solution Approach 2:
The patent performs preliminary deduplication analysis by generating and comparing fingerprints before actual data replication occurs. This preliminary action identifies which data blocks need to be replicated and which are redundant, preventing unnecessary storage allocation and reducing overall storage overhead while ensuring critical data is backed up.
3Loss of energy
If deduplication fingerprints are generated for each IO request, then data uniqueness is identified, but processing overhead increases
Solution Approach 1:
The patent replaces the mechanical process of comparing actual data blocks with a computational hashing mechanism. By substituting complex byte-by-byte data comparison with fingerprint generation and comparison, the system achieves efficient data uniqueness identification with reduced processing overhead, as hash computation is computationally lighter than full data comparison.
Data Source
AI summary
The present disclosure relates to applying a data deduplication layer on top of an asynchronous remote replication protocol. In embodiments, a remote replication group including one or more of a storage array's logical unit numbers (LUNs) can be established. Further, at a remote system, —the remote replication group can be remotely replicated on a per LUN basis using a deduplication fingerprint of each LUN's data tracks corresponding to the remote replication group.