Storage System Stale Data Detection via Bidirectional Signature Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data replication systems face challenges in recovering from failures without excessive data transfer and effectively eliminating stale data, which can prolong replication cycles and hinder the achievement of desired recover point objectives (RPOs).
Innovation Solution
Implementing storage systems with stale data detection and clean-up functionality, using bidirectional content-based signature comparisons to identify and eliminate stale data during replication failure recovery, while conserving network bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional data replication systems perform full data transfer during failure recovery, then data consistency is restored, but network bandwidth resources are excessively consumed and replication cycle length increases
Solution Approach 1:
The patent extracts only the necessary data elements (stale data identified through bidirectional signature comparisons) rather than transferring all data during failure recovery. By comparing content-based signatures bidirectionally between source and target systems, the system identifies and transfers only the specific data pages that differ, significantly reducing network bandwidth consumption while ensuring data consistency is restored.
2Reliability
If conventional replication systems transfer all data during failure recovery, then data consistency is ensured, but replication cycle length is prolonged
Solution Approach 1:
The system extracts and transfers only stale data identified through bidirectional signature comparisons rather than performing full data transfers. This selective approach significantly reduces the time required for failure recovery while maintaining data consistency, as evidenced by the reduced replication cycle length achieved through targeted data page transfers.
3Reliability
If bidirectional signature comparisons are performed, then stale data is effectively identified and eliminated, but detection complexity increases
Solution Approach 1:
The patent uses content-based signatures (hash values) as copies or representations of the actual data pages. Instead of comparing entire data pages bidirectionally, the system compares compact signature representations, which dramatically simplifies the detection mechanism while maintaining the ability to effectively identify and eliminate stale data through the bidirectional comparison process.
4Ease of manufacture
If conventional systems perform unidirectional data transfer during recovery, then implementation is simpler, but stale data on target system is not effectively cleaned up
Solution Approach 1:
The patent inverts the conventional unidirectional approach by implementing bidirectional signature comparisons and selective data transfers. Instead of simply transferring data from source to target, the system also identifies and cleans up stale data on the target system that does not exist on the source, achieving effective stale data elimination while maintaining reasonable implementation complexity through the use of signature-based comparisons.
Data Source
AI summary
An apparatus in one embodiment is configured to detect a failure in a data verification operation performed on at least a portion of at least one replicated storage volume. Responsive to the detected failure, the apparatus is further configured to perform bidirectional content-based signature comparisons between data pages of a source storage system and data pages of a target storage system. For one or more data pages of the source storage system for which respective content-based signatures are not found in the target storage system, the one or more data pages are sent from the source storage system to the target storage system. For one or more data pages of the target storage system for which respective content-based signatures are not found in the source storage system, the one or more data pages are designated as stale data and a clean-up operation is initiated for the stale data.