Resynchronization of Replicated Data Using Hash Tuples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data resynchronization methods are resource-intensive and time-consuming, often taking hours or days for large replicated data sets, as they require traversing and comparing all data segments between source and target nodes, which is wasteful and inefficient.
Innovation Solution
The solution involves maintaining sequence numbers and hash tuples for data chunks at both source and target nodes, allowing for efficient determination of synchronization by comparing current and stored sequence numbers and hashes, enabling resynchronization without reading all data, and updating these values during replication to identify and correct out-of-sync data in a single pass.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional resynchronization methods traverse and compare all data segments, then data consistency is ensured, but processing time and resource consumption increase dramatically
Solution Approach 1:
The patent extracts only the essential synchronization metadata (sequence numbers and hash values) from the complete data sets, maintaining synchronization verification capability while eliminating the need to process entire data segments. This extraction approach enables rapid comparison without reading all data, resolving the contradiction between thorough verification and time efficiency.
Solution Approach 2:
The system performs preliminary actions by pre-computing and storing sequence numbers and hash values for data segments during the replication process. These pre-computed metadata are maintained in synchronization with the replicated data, allowing future resynchronization operations to proceed rapidly by comparing only these pre-prepared values rather than re-processing entire data sets.
2Measurement precision
If conventional methods compute hashes for all data segments, then accurate synchronization status is determined, but processing resources are excessively consumed
Solution Approach 1:
The patent extracts only the hash values (without re-computing them) from the original data segments and stores these extracted hash values as metadata. This extraction eliminates the computationally expensive hash computation step during resynchronization, maintaining measurement precision through stored hash values while dramatically reducing processing resource consumption.
Solution Approach 2:
The system creates and maintains copies of hash values and sequence numbers as metadata alongside the replicated data. These copied metadata values enable synchronization verification without requiring re-computation or re-reading of the original data segments, preserving accuracy while minimizing resource usage.
3Reliability
If complete data resynchronization is performed, then data consistency is guaranteed, but network bandwidth is wasted transferring unchanged data
Solution Approach 1:
The patent extracts and compares only sequence numbers and hash values to determine which data segments require resynchronization. This extraction-based verification approach identifies precisely which segments are out of sync without transferring complete data sets, ensuring data consistency for modified segments while eliminating unnecessary network transmission of unchanged data.
Solution Approach 2:
The system performs partial resynchronization by transferring only the specific data segments that are determined to be out of sync, rather than performing excessive full data set transfers. This partial action approach maintains data consistency where needed while optimizing network bandwidth utilization by avoiding redundant transfers.
Data Source
AI summary
Mechanisms are provided for efficient resynchronization of replicated data. A hash value is generated for a chunk of data replicated from a source node to a target node. The chunk of data may be a file deduplicated and compressed at both a source node and a target node. A current sequence number is determined and a sequence number and hash tuple is maintained for the chunk of data at both the source node and the target node. Sequence numbers are modified whenever the data is modified. Current sequence numbers and sequence number and hash values in the sequence number hash tuples at the source node and the target node may be compared to determine whether data is still synchronized at a later point in time or whether data requires resynchronization.