Database Log Replay Concurrency via Page Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional log replay methods in database systems are slow, leading to prolonged downtime when a primary server fails, as they require sequential replay of transaction logs and synchronization between threads, which limits the speed at which secondary servers can take over and provide data services.
Innovation Solution
The method involves using multiple threads to replay transaction commit logs and page operation logs independently, where all page operation logs for a page are replayed by a single thread, allowing for concurrent processing and reducing the need for synchronization, and reading logs from a buffer instead of data memory to enhance speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple threads are used to replay logs concurrently, then log replay speed is improved, but thread synchronization complexity increases
Solution Approach 1:
The patent divides log replay into two independent segments: transaction commit log replay (single thread) and page operation log replay (multiple threads). This segmentation allows the page operation logs to be replayed in parallel without requiring synchronization with the transaction commit log replay process, thus improving log replay speed while avoiding thread synchronization complexity.
Solution Approach 2:
The patent introduces an intermediary mechanism where page operation logs are processed independently through multiple threads without needing to synchronize with the main transaction commit log replay thread. This intermediary approach allows concurrent processing while maintaining data consistency, resolving the contradiction between speed and synchronization complexity.
2Device complexity
If all page operation logs for a page are replayed by a single thread, then synchronization requirements are reduced, but thread utilization efficiency may decrease
Solution Approach 1:
The patent segments page operation logs by page, assigning all logs for a specific page to a single thread. This segmentation reduces synchronization requirements while maintaining parallelism across different pages, effectively balancing thread utilization efficiency and synchronization complexity.
3Productivity
If logs are read from data memory, then data accuracy is ensured, but input/output operations increase reducing speed
Solution Approach 1:
The patent performs preliminary action by pre-reading and buffering log data before the replay process. Logs are read from data memory in advance and stored in a buffer, reducing I/O operations during replay while ensuring data accuracy is maintained through the use of committed and durable log data.
Data Source
AI summary
A method for replaying a log on a data node, a data node, and a database system are described. The method is used to improve a log replay speed, and includes: the data node obtaining a plurality of logs, wherein the plurality of logs includes at least one transaction commit log and at least one page operation log; and a first thread of a plurality of threads replaying the at least one page operation log, and a same second thread replaying all page operation logs including an operation on a same page. As such, log replay progress does not need to be synchronized between threads. A process in which the first thread replays the transaction commit log is independent of a process in which at least one second thread replays the at least one page operation log.