Distributed Database Log Replay Parallelism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed database systems, synchronizing log backups across multiple nodes is challenging due to performance issues, especially when logs are stored across multiple servers, and ensuring transactional consistency during data recovery is difficult, especially when reading from tertiary media like tape.
Innovation Solution
The solution involves recording log entries in a data storage application with a node hierarchy comprising a master node and slave nodes, where the master node replays its log entries until a first replay position and then initiates parallel replay of subsequent log entries with slave nodes, allowing for snapshotting and error handling to maintain transactional consistency, and discarding unrecoverable log entries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If log backups are synchronized across all nodes to ensure transactional consistency, then reliability is improved, but productivity deteriorates due to performance overhead
Solution Approach 1:
The log replay process is segmented into multiple phases: first replaying logs on the primary node to a checkpoint position, then replaying on secondary nodes to the same position, and finally replaying remaining logs in parallel. This segmentation allows nodes to operate independently at different stages, improving overall productivity while maintaining consistency through coordinated checkpoints.
Solution Approach 2:
The primary node performs preliminary log replay actions before secondary nodes begin their replay. By advancing the primary node's replay position first and establishing checkpoint positions, the system prepares a foundation that enables secondary nodes to catch up efficiently, reducing the synchronization overhead that would otherwise hinder productivity.
2Reliability
If the entire log is replayed on all nodes to ensure consistency, then reliability is improved, but loss of time worsens due to redundant replay operations
Solution Approach 1:
The log replay workload is segmented by dividing it into two portions: logs up to the checkpoint position are replayed on all nodes, while logs after the checkpoint are replayed only on the primary node. This segmentation eliminates redundant replay operations on secondary nodes for the second portion, significantly reducing total replay time while maintaining consistency through the checkpoint mechanism.
Solution Approach 2:
The system discards redundant log replay operations on secondary nodes for log portions that will be replayed on the primary node. By identifying the checkpoint position and determining which logs can be skipped on secondary nodes, the system recovers time that would otherwise be spent on unnecessary replay operations, while still ensuring consistency through the primary node's complete replay.
3Reliability
If log replay is performed sequentially on all nodes, then reliability is improved through proper ordering, but productivity deteriorates due to lack of parallelism
Solution Approach 1:
The replay process is segmented into sequential and parallel phases. The first phase (replaying logs to checkpoint position) is performed sequentially on each node to establish proper ordering. The second phase (replaying logs after checkpoint) is performed in parallel on the primary node, maximizing productivity while maintaining reliability through the established ordering from the first phase.
Solution Approach 2:
The system dynamically adjusts the replay approach based on the checkpoint position. Secondary nodes dynamically determine their replay scope by comparing the checkpoint position with their current log position, replaying only the necessary portion. This dynamic adaptation enables parallel execution where possible while maintaining the required sequential ordering for consistency.
Data Source
AI summary
Log entries are recorded in a data storage application (such as an in-memory database, etc.) for a plurality of transactions among nodes in a node hierarchy. The node hierarchy comprises master node having a plurality of slave nodes. Thereafter, at least a portion of the master node log entries are replayed until a first replay position is reached. Next, for each slave node, at least a portion of its respective log entries are replayed until the first replay position is reached (or an error occurs). Subsequently, replay of at least a portion of the log entries of the master node that are subsequent to the first replay position is initiated by the master node in parallel to at least a portion of the replaying by the slave nodes. Related apparatus, systems, techniques and articles are also described.