Persistent Redo Buffering for OLTP Commit Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current write-ahead logging (WAL) techniques introduce latency during transaction commit due to log file sync, which affects online transaction processing (OLTP) throughput, and using persistent memory (PMEM) to reduce this latency is hindered by the lack of fine granularity in sleep time programming and increased task switching latency.

Innovation Solution

Implementing persistent redo buffering in PMEM to persist redo records before appending them to the online redo log, allowing database sessions to asynchronously persist redo records using remote direct memory access (RDMA), thus eliminating the need for log file sync and reducing commit latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If WAL is used to ensure atomicity and durability, then transaction reliability is improved, but commit latency increases due to log file sync

Engineering Contradiction:

Improvetransaction durabilityVSAvoidcommit latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by persisting redo records to PMEM in advance before actual commit. The system writes redo records to a persistent redo buffer in PMEM during transaction execution, so that when commit is called, the durability guarantee is already in place without requiring additional sync operations. This separates the persistence action from the commit action, eliminating the wait time.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent segments the logging process into two independent parts: (1) writing redo records to the persistent redo buffer in PMEM, and (2) the actual commit operation. This segmentation allows the commit process to proceed without waiting for the log writer to complete, as the persistence to PMEM happens asynchronously in the background.

Inventive Principle:

Principle #1Segmentation

2Speed

If PMEM is used to replace disk storage for redo logs, then write speed is improved, but sleep time control becomes difficult due to insufficient granularity

Engineering Contradiction:

Improvewrite speedVSAvoidsleep time programming

Core Design Contradiction:

SpeedVSEase of operation

Solution Approach 1:

The patent applies self-service by making the database session itself responsible for persisting its own redo records to PMEM using RDMA, rather than relying on a separate log writer process. This eliminates the need for the session to sleep and wait for external operations, as the persistence is performed directly and immediately by the session that needs it.

Inventive Principle:

Principle #25Self-service

3Reliability

If the log writer completes before commit, then durability is ensured, but OLTP throughput decreases due to waiting processes

Engineering Contradiction:

Improvedurability guaranteeVSAvoidOLTP throughput

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system performs the durability-guaranteeing persistence action preliminarily during transaction execution by writing redo records to PMEM. This way, when commit is called, the durability is already established and no waiting is needed, allowing the OLTP process to continue immediately and maintain high throughput.

Inventive Principle:

Principle #10Preliminary action

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach significantly reduces transaction commit latency and increases OLTP throughput by avoiding log file sync and allowing immediate processing after commit, while maintaining durability and reliability through PMEM-based buffering.

Implementation Method 1

Persistent memory (PMEM) can be written at nearly dynamic random access memory (DRAM) speeds but, unlike DRAM, PMEM has the advantage of persistence.

Methodology Applied

Scientific EffectPersistent memory:

Implementation Method 2

allowing database sessions to asynchronously persist redo records using remote direct memory access (RDMA)

Methodology Applied

Scientific EffectRemote direct memory access:

Data Source

PatentUS12086132B2Increasing OLTP throughput by improving the performance of logging using persistent memory storage

Publication Date: 2024.09.10 ORACLE INT CORP

AI summary

In an embodiment, before modifying a persistent ORL (ORL), a database management system (DBMS) persists redo for a transaction and acknowledges that the transaction is committed. Later, the redo is appended onto the ORL. The DBMS stores first redo for a first transaction into a first PRB and second redo for a second transaction into a second PRB. Later, both redo are appended onto an ORL. The DBMS stores redo of first transactions in volatile SRBs (SLBs) respectively of database sessions. That redo is stored in a volatile shared buffer that is shared by the database sessions. Redo of second transactions is stored in the volatile shared buffer, but not in the SLBs. During re-silvering and recovery, the DBMS retrieves redo from fast persistent storage and then appends the redo onto an ORL in slow persistent storage. After re-silvering, during recovery, the redo from the ORL is applied to a persistent database block.