Journaling Process for Continuous Data Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data replication systems face challenges in keeping pace with high data transaction rates at production sites, leading to potential shutdowns due to backlog of un-logged transactions and limitations in minimizing downtime and enabling recovery to any specified point in time.
Innovation Solution
The method involves managing journaling processes to track changes across consistency groups, allowing for the addition or removal of LUNs without disrupting replication, ensuring that data is synchronized and accessible only at specific points in time, and utilizing a five-state journaling process to maintain data integrity and availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data replication uses journaling to track changes at the backup site, then data integrity and recovery capability are improved, but the backup site cannot keep pace with high data transaction rates at the production site, leading to system shutdowns
Solution Approach 1:
The consistency group is divided into multiple sub-consistency groups, each handled by separate journaling processes. This segmentation allows parallel processing of journal entries across multiple threads, enabling the backup site to keep pace with high transaction rates while maintaining data integrity through organized, concurrent journaling operations.
Solution Approach 2:
The system dynamically adjusts the number of journaling processes and threads based on the data transaction rate. When transaction rates increase, the system activates additional journaling processes to maintain backup performance without forcing production site slowdowns, thereby preventing system shutdowns while preserving reliability.
2Loss of time
If data replication keeps pace with production site transactions, then downtime during recovery is minimized, but the system forces production site shutdown when backlog builds up
Solution Approach 1:
The system performs preliminary actions by pre-configuring multiple journaling processes and establishing thread pools before high transaction rates occur. This allows the backup site to be ready to handle increased transaction volumes without forcing production shutdowns, thereby minimizing recovery downtime while maintaining system availability.
Solution Approach 2:
The system implements feedback mechanisms that monitor the backlog of un-logged transactions at the backup site. When the backlog approaches thresholds that would force shutdowns, the system activates additional journaling processes to clear the backlog, ensuring continuous operation and minimizing recovery downtime without sacrificing system availability.
3Measurement precision
If the backup site processes all data transactions sequentially, then journaling accuracy is maintained, but the backup site cannot handle high transaction rates without slowing down the production site
Solution Approach 1:
The journaling process is segmented into multiple parallel threads, each handling a portion of the transaction log. This allows the system to process high volumes of transactions concurrently while maintaining journaling accuracy through proper synchronization and ordering mechanisms within each thread, eliminating the need to slow down the production site.
Solution Approach 2:
The system maintains continuous journaling operations across multiple parallel processes, ensuring that no transactions are lost or missed. By keeping all journaling processes actively writing to the journal simultaneously, the system achieves both high transaction processing speed and journaling accuracy without interrupting production site operations.
Data Source
AI summary
In a first embodiment, computer implemented method and computer program product for executing an action selected from the group consisting of removing a LUN from a consistency group on a production site and adding a LUN to a consistency group on a production site; wherein the action is performed without loss of a journal tracking the changes to the consistency group at a replication site, the replication site replicating the consistency group on the production site.