Database Log Parallelization via Round-Robin Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database recovery methods are inefficient, particularly in update-intensive applications, as they require complex algorithms to reorder log entries and increase recovery time with the size of the database, affecting commit durability and processing time.
Innovation Solution
A round-robin distribution scheme for writing log entries to multiple log partitions, allowing for sequential assembly of a recovery log with I/O flags and commit callbacks to ensure correct ordering and efficient replay, reducing processing consumption and recovery time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional database recovery methods are used to ensure commit durability, then data integrity is maintained, but recovery time increases linearly with database size and processing complexity increases
Solution Approach 1:
The log is divided into multiple log partitions distributed across different storage devices. Each partition contains a subset of log entries, allowing parallel recovery operations. The round-robin distribution scheme segments the log writing process across multiple partitions, enabling concurrent log entry processing during recovery without requiring sequential reordering of all log entries.
Solution Approach 2:
The patent introduces a new dimension to log organization by using round-robin distribution across multiple log partitions instead of traditional sequential logging. This dimensional change allows log entries to be organized by partition index rather than strict chronological order within a single log file, enabling parallel recovery while maintaining commit durability through the distributed partition structure.
2Reliability
If sequential log processing is used to ensure correct ordering, then commit durability is maintained, but processing time and complexity increase
Solution Approach 1:
The log processing system is segmented into multiple independent log partition processors that can operate in parallel. Each processor handles a specific log partition independently, eliminating the need for sequential processing of all log entries. The segmentation maintains commit durability through proper partition indexing while significantly improving processing throughput.
Solution Approach 2:
Log entries are pre-distributed into multiple partitions using round-robin distribution during the logging phase. This preliminary organization eliminates the need for complex reordering operations during recovery, as log entries within each partition are already in the correct sequential order for that partition. The preliminary action of distributed sorting enables parallel processing without sacrificing correctness.
3Productivity
If multiple log partitions are used to improve parallel processing, then productivity increases, but system complexity and difficulty of managing log ordering increase
Solution Approach 1:
The round-robin distribution scheme provides a universal mechanism that handles both log writing and log organization simultaneously. The same round-robin logic applies to distributing log entries across partitions and to determining the correct processing order, simplifying the management complexity. This multi-functional approach eliminates the need for separate complex indexing and reordering mechanisms.
Solution Approach 2:
The patent changes the organizational parameter from single-sequential-log to multi-partition-distributed-log with round-robin indexing. This parameter change transforms the complexity from managing sequential order in one large log to managing independent sequential orders in multiple smaller logs with a simple round-robin distribution pattern, reducing overall management complexity while enabling parallel processing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A plurality of transactions in a data storage application are executed during which log entries according to the plurality of transactions are stored in a plurality of log buffers. The log entries are sequentially written into a current log buffer and the plurality of log buffers are written to a plurality of log partitions according to a round-robin distribution. Thereafter, in response to a data recovery event, a sequential list of the log entries in the log buffers are assembled to generate a log that can be used for data recovery by the data storage application. Related apparatus, systems, techniques and articles are also described.