Private Log Cache Batching for Database Transaction Throughput

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The logging system in database management systems (DBMS) presents a bottleneck to system scalability due to increased contention for logging resources as every transaction involves the logging system, leading to reduced throughput with growing transaction volume.

Innovation Solution

Implementing a private log cache (PLC) and a queue to temporarily hold log records, reducing contention on the transaction log lock by buffering log records in memory and flushing them in batches, thereby minimizing direct access to the transaction log.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If log records are written to the transaction log for every transaction operation, then data consistency and recovery capability are ensured, but logging resource contention increases and system throughput decreases

Engineering Contradiction:

Improvedata consistencyVSAvoidsystem throughput

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent applies preliminary action by pre-writing log records to a private log cache in memory before they are permanently written to the transaction log on disk. This allows transactions to commit and proceed without waiting for disk I/O operations, thereby maintaining data consistency through eventual log flushing while significantly improving system throughput by decoupling transaction commit from physical log writing.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces a private log cache as an intermediary layer between the transaction processing system and the transaction log. This intermediary buffer in memory absorbs the logging resource contention by temporarily holding log records, allowing multiple transactions to write to the cache simultaneously without blocking each other, while only periodic flush operations contend for the actual transaction log resources.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If log records are flushed frequently to the transaction log, then log contention is reduced, but I/O operations increase and performance deteriorates

Engineering Contradiction:

Improvetransaction throughputVSAvoidI/O operations

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent implements periodic action by flushing log records from the private log cache to the transaction log at scheduled intervals or when the cache reaches a threshold, rather than after every transaction. This periodic flushing strategy reduces the frequency of I/O operations compared to traditional immediate flushing, thereby improving transaction throughput while still maintaining acceptable log durability through controlled periodic synchronization to disk.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS11487740B2Reduce log contention by batching log record transfers to the log

Publication Date: 2022.11.01 SYBASE INC

AI summary

A system for managing database logging, the comprises a processor; and a user task executing in a database server process and executable by the processor, the user task to: receive in a database management system on a database server, a command to manipulate a portion of a database managed by the database management system; obtain a lock on the portion of the database; create a first log record in a first private log cache associated with the user task, the first log record recording a data manipulation to the portion of the database; enqueue the first log record to a queue; and release the lock on the portion of the database after copying the first log record to the queue.