Persistent Change Log Buffer for Database Transaction Commit Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Log file synchronization delay occurs during transaction commit time in database systems, leading to increased latency and reduced throughput in OLTP workloads, despite existing solutions like using non-volatile memory, which do not effectively address this issue.
Innovation Solution
Implementing a persistent change log buffer in non-volatile memory that allows change records to be persisted independently without synchronization, enabling deferred writing to disk and eliminating the need for synchronization between processes, thus allowing transaction commits to proceed without waiting for log file writes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If change records are written to persistent log file on disk during transaction commit, then durability is ensured, but transaction commit latency increases due to log file synchronization delay
Solution Approach 1:
The system separates the change log buffer into multiple regions, with each region independently writable by different processes. This segmentation allows parallel writes without synchronization, eliminating the log file synchronization delay while maintaining durability through the write-ahead logging mechanism.
Solution Approach 2:
Change records are pre-written to the change log buffer in non-volatile memory before being flushed to the persistent log file on disk. This preliminary action in a non-volatile buffer allows the transaction commit to proceed without waiting for the slower disk I/O, reducing commit latency while ensuring durability through the buffered records.
2Productivity
If multiple processes concurrently write change records to in-memory change log buffer, then throughput is improved, but synchronization delay occurs when gathering records from multiple regions
Solution Approach 1:
The change log buffer is divided into multiple independent regions that can be concurrently written to by different processes without requiring synchronization. Each process owns specific regions, eliminating the need to wait for other processes to finish writing to the same buffer, thus maintaining high throughput while avoiding synchronization delays.
Solution Approach 2:
The change log buffer in non-volatile memory acts as an intermediary between the concurrent transaction processes and the single writer process that flushes to disk. Multiple processes can write to the buffer concurrently without blocking each other, and the writer process can gather records at its own pace without impacting transaction commits, thus eliminating synchronization delay while maintaining throughput.
Data Source
AI summary
A method and apparatus for elimination of log file synchronization delay at transaction commit time is provided. One or more change records corresponding to a database transaction are generated. One or more buffer entries comprising the one or more change records are entered into a persistent change log buffer. A commit operation is performed by generating a commit change record corresponding to the database transaction and entering a commit buffer entry comprising the commit change record into the persistent change log buffer. The commit operation returns without waiting for the commit change record to be recorded in a change record log file.