Asynchronous Journaling Policy for Continuous Data Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data protection systems face challenges in maintaining continuous data replication under dynamically changing I/O rates, leading to potential production site shutdowns due to backlog buildup at the backup site, as they struggle to keep pace with high data transaction rates without slowing down the production site.
Innovation Solution
An optimized asynchronous journaling policy that dynamically adapts by switching between nominal and accelerated journaling modes based on backlog size, and employs a near real-time policy when transaction rates allow, ensuring the backup site can manage data transactions efficiently without causing production site downtime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the backup site processes all data transactions synchronously (5-stage journaling), then data integrity is maintained, but the backup site cannot keep pace with high data transaction rates at the production site
Solution Approach 1:
The system dynamically adjusts the journaling process by implementing multiple processing modes (nominal 5-stage journaling, accelerated journaling, and near real-time journaling) that can be switched based on the current data transaction rate and backlog conditions, allowing the backup site to adapt its processing speed to match production site demands
2Productivity
If the backup site operates asynchronously with nominal journaling, then it can handle higher data transaction rates, but backlog of un-logged data transactions builds up when production site I/O rate increases
Solution Approach 1:
The system implements a feedback mechanism that continuously monitors the backlog of un-logged data transactions at the backup site and dynamically adjusts the journaling mode in response. When backlog exceeds a threshold, the system transitions from nominal asynchronous journaling to accelerated journaling to clear the backlog, creating a closed-loop control system that prevents excessive backlog accumulation
3Productivity
If the system forces the production site to slow down to match backup site capacity, then backlog is prevented, but production site downtime increases
Solution Approach 1:
The system segments the journaling process into distinct stages (5-stage nominal journaling, 3-stage accelerated journaling) and allows different stages to operate at different speeds. This segmentation enables the backup site to process transactions faster when needed without requiring the production site to slow down, as the segmented process can absorb variable processing rates
4Productivity
If the system uses accelerated journaling to clear backlog, then data transaction throughput increases, but data protection capability is reduced by omitting journaling stages
Solution Approach 1:
The system dynamically switches between full 5-stage journaling mode (providing complete data protection) and accelerated 3-stage journaling mode (providing faster throughput). The mode selection is based on real-time conditions, ensuring that the more protective mode is used when data integrity is critical while the accelerated mode is used when catching up on backlog, thus balancing protection and throughput
Data Source
AI summary
Method and apparatus to provide a data replication system to receive a data transfer request from a first component at a transmitter module in a continuous data replication system having a production site and a backup site, the production site having a transmitter module and a transmitter credit mechanism, the transmitter module to transmit data over a network for replication in the backup site, detect a high-load condition at the transmitter module, and modify a flow of credits from the transmitter credit mechanism to the first component to reduce or stop data flow to the transmitter module.