Non-Volatile Memory Transaction Commit Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Log file synchronization delays occur during transaction commits in database systems due to the need to write change records to persistent storage, which hinders concurrency and increases latency in OLTP workloads.
Innovation Solution
Implementing non-volatile memory as a side buffer to hold change records, allowing transactions to commit without waiting for records to be persisted to disk, while maintaining compatibility with existing log writing models by operating in conjunction with volatile memory buffers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If change records are written to persistent storage before transaction commit, then data durability is ensured, but transaction commit latency increases due to log file synchronization delays
Solution Approach 1:
The system performs preliminary actions by writing change records to the persistent change log file in advance, before the transaction actually commits. The write-ahead logging mechanism ensures that all modifications are recorded on disk prior to commit, allowing the transaction to commit quickly without waiting for I/O completion, thus resolving the contradiction between durability and commit latency
Solution Approach 2:
The patent introduces an intermediary mechanism where change records are first written to an in-memory change log buffer, then asynchronously flushed to the persistent change log file by a separate writer process. This intermediary buffer allows the transaction commit to proceed without directly blocking on slow disk I/O, while still ensuring durability through the write-ahead logging guarantee
2Productivity
If multiple processes concurrently write change records to in-memory buffers, then transaction concurrency is improved, but log file synchronization delays increase due to writer process coordination overhead
Solution Approach 1:
The system segments the change log buffering into multiple independent in-memory change log buffers, one for each process. Each process writes to its own dedicated buffer without contention, and the writer process segments the work of flushing buffers to disk. This segmentation eliminates write contention while maintaining concurrency, resolving the contradiction between productivity and synchronization delay
Solution Approach 2:
Each process serves itself by maintaining its own private change log buffer and managing its own change records without needing to coordinate with other processes for buffer access. The writer process independently gathers and flushes each buffer without requiring synchronization with the processes that wrote to it, enabling self-service operation that improves concurrency while reducing synchronization overhead
Data Source
AI summary
Described is an improved approach to implement parallel queries where session states are saved for parallelization resources. When work needs to be performed in the parallel query system for a given session, a search can be performed to identify a resource (from among the pool of available resources) that had previously been used by that session, and which had saved a session state object for that previous connection to the session. Instead of incurring the entirety of setup costs each time workload is assigned to a resource, the saved session state can be used to re-set the context for the resource to the configuration requirements for that session.