Log Replay Queue Segmentation for Error-Isolated Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing log replay methods in database systems waste system resources when encountering errors, particularly when an in-situ retry policy is employed for errors that cannot be resolved automatically, leading to occupation of replay threads and affecting other log streams.
Innovation Solution
Classify to-be-replayed logs into multiple log queues based on log streams and transaction numbers, allocate replay threads to these queues, and reassign threads to following queues if errors occur, isolating errors within a single log stream to avoid resource waste and maintain efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If an in-situ retry policy is employed to repeatedly replay a log with error until the error is eliminated, then the log replay reliability is improved, but system resources are occupied and other log streams are affected
Solution Approach 1:
The patent segments the log replay process by introducing a global replay queue that divides logs into multiple queues. When an error occurs in one log queue, only that specific queue is affected and can be retried independently, while other queues continue processing without interruption. This segmentation isolates errors to prevent system-wide resource occupation.
Solution Approach 2:
The patent changes the retry mechanism from in-situ retry (replaying the same log repeatedly in place) to cross-queue retry (replaying failed logs in different queues). This parameter change allows the system to maintain retry reliability while improving overall throughput by utilizing available replay threads in other queues.
2Loss of energy
If logs are replayed sequentially without parallel processing, then resource occupation is minimized, but log replay efficiency is reduced
Solution Approach 1:
The patent introduces dynamic resource allocation where replay threads are allocated to log queues based on current system state and error conditions. The global replay queue dynamically adjusts which queues receive replay thread allocation, allowing parallel processing when resources are available while adapting to error conditions that may require sequential retry of specific queues.
Solution Approach 2:
The global replay queue serves multiple functions: it manages parallel replay of multiple log queues, tracks error states across different queues, and coordinates retry operations. This universal structure enables the system to achieve both parallel processing efficiency and controlled resource occupation.
Data Source
AI summary
Implementations of this specification provide methods and apparatuses for replaying logs. One method includes: classifying a plurality of logs to be replayed into log queues, sending the log queues to a global replay queue, allocating one or more replay threads to the log queues based on rankings of the log queues in the global replay queue, wherein the one or more replay threads are configured to replay one or more current log queues of the log queues, and in response to identifying an error of a current log queue of the one or more current log queues in a replay process, allocating a replay thread of the one or more replay threads allocated to the current log queue to a log queue that immediately follows the current log queue in the global replay queue.