Replication Progress Tracking for Multi-Site Data Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-site data replication environments, existing technologies face challenges with costly and inefficient data replication due to constant network data transfer and lossy network conditions, which can lead to delayed protection and unnecessary re-replication of data.
Innovation Solution
A replication progress tracking technique that persistently stores and presents the progress tracking state of replicated data as a recovery point (RP) at the target site, independent of the source, using a publicly callable API. This allows clients to identify and re-replicate dropped data ranges without re-replicating successfully replicated data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data replication is performed constantly to ensure data protection, then data reliability is improved, but network bandwidth consumption increases and replication becomes costly
Solution Approach 1:
The system performs preliminary actions by tracking replication progress and identifying dropped data ranges before initiating re-replication. The target site maintains knowledge of which data blocks have been successfully replicated and which have been dropped, allowing selective re-replication only of necessary data portions rather than constant full-data replication.
Solution Approach 2:
The system applies local quality by treating different data ranges differently based on their replication status. Successfully replicated data ranges are not re-transmitted, while dropped data ranges are identified and re-replicated selectively. This localized approach optimizes network bandwidth by transmitting only the specific portions of data that need replication.
2Reliability
If ordered replication is used to maintain data sequence integrity, then data reliability is improved, but replication speed decreases when data loss occurs
Solution Approach 1:
The replication process is segmented into independent data blocks or ranges. When data loss is detected in specific segments, only those segmented portions are re-replicated rather than restarting the entire replication sequence. This segmentation allows parallel processing of different data ranges and maintains overall replication speed while ensuring sequence integrity through selective re-transmission of only affected segments.
3Reliability
If replication is restarted from the beginning after data loss, then data completeness is improved, but time consumption increases
Solution Approach 1:
The system implements feedback mechanisms where the target site monitors replication progress and identifies dropped data ranges. This feedback information is used to determine which specific data blocks need re-replication, allowing the system to avoid restarting from the beginning. The feedback loop ensures data completeness by tracking which blocks are missing while minimizing time loss by targeting only the necessary re-replication operations.
4Device complexity
If progress tracking is performed only at the source, then system complexity is reduced, but measurement precision of replication status decreases in multi-site environments
Solution Approach 1:
The target site serves as an intermediary that maintains local tracking of replication progress. Instead of relying solely on source-site tracking, the target site independently records which data blocks have been successfully received and which have been dropped. This intermediary tracking mechanism provides precise measurement of replication status at the target, enabling accurate identification of dropped ranges without significantly increasing overall system complexity.
Data Source
AI summary
A replication progress tracking technique is configured to present progress tracking state relating to data replicated as a recovery point (RP) from a source site to one or more target and/or remote sites over a network in a multi-site data replication environment. The progress tracking state of the replicated RP, i.e., RP replication progress, is persistently stored as a progress tracking recording at the target site independent of the source site of the replication and presented (i.e., viewed) using a well-defined, published application programming interface that is publicly callable by one or more clients of the environment.