Amortizing Replication Log Updates for Database Transactions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database replication techniques are costly and result in lengthy downtime during recovery from failures, especially for large transactions, due to the time-consuming process of copying transaction data to replication logs, which can lead to extended downtime and performance impacts on client applications.
Innovation Solution
Implementing amortizing replication log updates for transactions, where replication log records are stored locally until a transaction size threshold is reached, then transferred to remote storage, reducing the time required for commit and minimizing the work needed to recover from failures by optimizing log updates and reducing data copying.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If replication log updates are performed synchronously for all transactions, then data consistency is ensured, but commit time increases and productivity decreases
Solution Approach 1:
The patent segments the replication log update process into two phases: a local phase where replication records are stored in a local data store during transaction commit, and a remote phase where these records are asynchronously transferred to remote storage. This segmentation allows the commit operation to complete quickly without waiting for remote replication, while still ensuring eventual consistency through the asynchronous transfer mechanism.
Solution Approach 2:
The patent implements preliminary action by pre-storing replication records in the local data store before the asynchronous remote transfer completes. The local data store captures replication records immediately upon transaction commit, ensuring consistency is prepared and maintained locally while the remote replication occurs in the background, thus decoupling commit time from replication time.
2Loss of time
If replication logs are stored locally for all transactions, then recovery time is reduced, but storage resources are consumed and device complexity increases
Solution Approach 1:
The patent extracts the replication log storage function from the primary database storage system by introducing a separate local data store specifically for replication records. This extraction allows the database to maintain a simple storage architecture for transactional data while using a dedicated, optimized storage location for replication logs, improving recovery performance without complicating the core database structure.
Solution Approach 2:
The local data store acts as an intermediary between the database transaction system and the remote replication storage. It temporarily holds replication records during the transaction commit process and serves as the source for asynchronous remote transfers. This intermediary layer simplifies the overall architecture by providing a buffered, decoupled storage mechanism that improves recovery without requiring complex synchronous replication infrastructure.
3Reliability
If synchronous replication is used for large transactions, then data consistency is maintained, but throughput decreases and loss of time increases
Solution Approach 1:
The patent implements periodic action through the asynchronous transfer mechanism, where replication records are transferred to remote storage in batches rather than immediately after each transaction. The system periodically flushes the local data store to remote storage, allowing large transactions to commit quickly while maintaining consistency through periodic synchronization, thus improving throughput without sacrificing data consistency.
Data Source
AI summary
Updates for transactions to a replication log for a database may be amortized. As updates are received as part of an active transaction, replication log records may be generated. When the size of replication log records exceeds a transaction size threshold, the replication log records may be stored in a remote data store instead of a local data store. When a request to commit the active transaction is received, an replication log for the database may be updated to include the replication log records from a transaction data object in the remote data store.