Persistent Redo Buffering for OLTP Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current write-ahead logging (WAL) techniques introduce latency during transaction commit due to log file sync, which affects online transaction processing (OLTP) throughput, and using persistent memory (PMEM) to reduce this latency faces challenges in fine-grained sleep time management and task switching latency.
Innovation Solution
Implementing persistent redo buffering in PMEM to persist redo records before appending them to the online redo log, allowing database sessions to asynchronously persist redo records using remote direct memory access (RDMA), thereby eliminating the need for log file sync wait and reducing commit latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If persistent memory (PMEM) is used to store redo logs, then log file sync latency is reduced, but fine-grained sleep time management becomes difficult and task switching latency increases
Solution Approach 1:
The patent applies preliminary action by buffering redo records in memory before they are appended to the online redo log. This allows the database session to persist redo asynchronously to PMEM without waiting for the log writer to complete, thereby reducing log file sync latency while avoiding the need for fine-grained sleep time management
Solution Approach 2:
The patent introduces an intermediary mechanism where redo records are first buffered in memory and then asynchronously persisted to PMEM. This intermediary buffering layer decouples the database session from the log writer process, eliminating the need for the session to wait for log file sync and avoiding task switching latency issues
2Reliability
If database sessions wait for log writer to complete during transaction commit, then data durability is ensured, but OLTP throughput decreases due to commit latency
Solution Approach 1:
The patent applies preliminary action by buffering redo records in memory before appending them to the online redo log. This allows the database session to persist redo asynchronously to PMEM without waiting for the log writer to complete, thereby reducing log file sync latency while avoiding the need for fine-grained sleep time management
Solution Approach 2:
The patent enables continuous useful action by allowing database sessions to asynchronously persist redo records to PMEM without waiting for the log writer. This asynchronous persistence continues the useful action of data durability maintenance without blocking the transaction commit path, thereby increasing OLTP throughput
Data Source
AI summary
In an embodiment, before modifying a persistent ORL (ORL), a database management system (DBMS) persists redo for a transaction and acknowledges that the transaction is committed. Later, the redo is appended onto the ORL. The DBMS stores first redo for a first transaction into a first PRB and second redo for a second transaction into a second PRB. Later, both redo are appended onto an ORL. The DBMS stores redo of first transactions in volatile SRBs (SLBs) respectively of database sessions. That redo is stored in a volatile shared buffer that is shared by the database sessions. Redo of second transactions is stored in the volatile shared buffer, but not in the SLBs. During re-silvering and recovery, the DBMS retrieves redo from fast persistent storage and then appends the redo onto an ORL in slow persistent storage. After re-silvering, during recovery, the redo from the ORL is applied to a persistent database block.