Data Replication Engine Switchover via Audit Trail Tokens
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for replacing or upgrading data replication engines in mission-critical applications often require shutting down the application, leading to potential data loss and system unavailability, as they cannot handle ongoing transactions and maintain target database consistency during the switchover process.
Innovation Solution
A method involving writing a first token into the Audit Trail to track ongoing transactions, allowing the original data replication engine to be stopped after they are completed, and the new engine to start while skipping already replicated transactions, ensuring no data loss and maintaining target database consistency by only replicating new transactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the application is shut down to replace the data replication engine, then the replacement can be performed cleanly without ongoing transactions, but the application experiences downtime and potential data loss
Solution Approach 1:
The system performs preliminary actions by capturing the list of active transactions before the switchover begins. The original data replication engine writes a token containing the list of active transactions to the audit trail, allowing the new engine to identify which transactions are already being handled and skip them during initialization, thereby enabling replacement without application shutdown
Solution Approach 2:
The audit trail token acts as an intermediary mechanism between the old and new data replication engines. It stores the list of active transactions and serves as communication medium that allows the new engine to understand the current state of transactions without interrupting the application, enabling seamless switchover
2Loss of time
If the new data replication engine starts processing transactions immediately during switchover, then no application downtime is needed, but already replicated transactions may be replayed causing data inconsistency
Solution Approach 1:
The new data replication engine uses feedback from the audit trail token (which contains the list of active transactions captured during switchover) to determine which transactions to skip. This feedback mechanism prevents the new engine from replaying transactions that were already replicated by the old engine, maintaining target database consistency during immediate switchover
Solution Approach 2:
The system creates a copy of the active transaction list in the audit trail token before switchover. The new data replication engine reads this copy and uses it to identify transactions that should be skipped, effectively copying the state information needed to prevent duplicate processing while enabling immediate startup
3Loss of information
If all ongoing transactions are replicated by the new data replication engine during switchover, then no transactions are lost, but transactions already replicated by the old engine are replayed causing data loss
Solution Approach 1:
The system extracts the list of active transactions from the running data replication engine and stores it in the audit trail token before switchover. The new engine then extracts this information to identify which transactions were already replicated by the old engine, allowing it to skip those transactions and avoid duplicate processing while ensuring no transaction data is lost
Data Source
AI summary
An automated method is provided for use when replacing a currently operating data replication engine in a first system with a new data replication engine in the first system in a bidirectional data replication environment. The currently operating data replication engine in the first system and the new data replication engine in the first system replicates first database transactions from an audit trail of a first database in the first system to a second database in a second system. The new data replication engine in the first system generating a list of active database transactions in the first system, and sends the list of active database transactions to the new data replication engine in the second system as a first token. The new data replication engine in the second system receives the first token, fetches a transaction event from an audit trail of second database, and replicates the fetched transaction event to the new data replication engine of the first system when the fetched transaction event does not match a transaction on the list in the first token. These steps are repeated during operation of the new data replication engine of the second system. The currently operating data replication engine in the first system is stopped from replicating first database transactions when all of the transactions on the list of active database transactions that were generated have been replicated to the second system.