Hash-Based Replication Optimization for CAS Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face inefficiencies in replication processes, particularly in comparing and transferring data between source and target sites, which can be costly and prone to data corruption, especially when assuming data identity without verification.
Innovation Solution
The system optimizes replication by analyzing differences between Content Addressable Storage (CAS) systems using hash-based methods, sending only necessary data and leveraging state information to expedite replication, especially for nearly identical data sets, thereby reducing latency and minimizing unnecessary operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full data comparison is performed between source and target CAS systems, then data accuracy is improved, but replication time and processing overhead increase significantly
Solution Approach 1:
The patent extracts only the essential verification element (hash value) from the complete data set, comparing hashes instead of full data contents. This extraction principle reduces the comparison workload from entire data sets to compact hash representations, maintaining accuracy while dramatically reducing replication time.
Solution Approach 2:
The patent transforms the data representation parameter from raw data contents to hash values. By changing the parameter used for comparison from full data to condensed hash representations, the system achieves both high accuracy (through cryptographic hash verification) and efficiency (through reduced comparison size).
2Loss of information
If all data is transferred during replication, then data completeness is improved, but network bandwidth consumption and transfer costs increase
Solution Approach 1:
The patent performs preliminary hash comparison before initiating data transfer. By pre-identifying which data blocks need replication through hash mismatch detection, the system avoids transferring data that already exists at the target, thereby reducing network bandwidth consumption while ensuring completeness of necessary transfers.
Solution Approach 2:
The patent applies partial action by transferring only the specific data blocks that are identified as different through hash comparison, rather than transferring all data. This selective partial transfer maintains data completeness for divergent blocks while minimizing unnecessary network usage.
3Productivity
If replication assumes data identity without verification, then replication speed is improved, but data corruption risk increases
Solution Approach 1:
The patent introduces hash values as an intermediary verification mechanism between source and target data. Instead of directly assuming data identity or comparing full data contents, the system uses hash values as intermediaries to verify data equivalence, thereby maintaining both speed (through efficient hash comparison) and reliability (through cryptographic verification).
Data Source
AI summary
A system, computer program product, and computer-executable method of optimizing replication between a first Content Addressable Storage (CAS) system and a second CAS system, the system, computer program product, and computer-executable method including analyzing the first CAS system and the second CAS system to determine an amount of differences between the first CAS system and the second CAS system, upon determining that the amount of differences are below a threshold amount, sending a set of replication information to the second CAS system, wherein the set of information includes a first address and a first hash for a portion of data stored on the first CAS system, analyzing, at the second CAS system, the set of replication information to determine whether to request that the first CAS system send data associated with the set of replication information; upon a negative determination, sending the data associated with the set of replication information.