Local Page Writes Using Pre-Staging Buffers in Buffer Pool Extensions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Compute nodes in large-scale database implementations face performance degradation due to the need to flush data pages to permanent storage when the local buffer pool becomes full, leading to delays and inefficiencies in write transactions, especially with restrictive latching mechanisms.
Innovation Solution
Implement pre-staging buffers to temporarily store data pages before writing to the buffer pool extensions, allowing encryption at the staging buffers and reducing the need for exclusive latches, enabling less restrictive update latching and batched writes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data pages are flushed to permanent storage when the local buffer pool becomes full, then the buffer pool capacity is maintained, but system performance degrades due to write delays and restrictive latching
Solution Approach 1:
A pre-staging buffer is introduced as an intermediary component between the buffer pool and permanent storage. This staging buffer temporarily holds data pages that need to be written, allowing the buffer pool to maintain its capacity while deferring the actual write operation to permanent storage. The intermediary staging buffer absorbs the write pressure, preventing direct performance degradation in the buffer pool operations.
Solution Approach 2:
The system performs preliminary actions by pre-staging data pages in the staging buffer before they are actually written to permanent storage. This preliminary staging phase allows the system to prepare writes in advance, batch them efficiently, and reduce the critical path delay for buffer pool operations while maintaining data integrity through proper latching mechanisms.
2Reliability
If exclusive latches are used for writing data pages to permanent storage, then data integrity is ensured, but write throughput decreases due to restrictive latching
Solution Approach 1:
The write operation is segmented into multiple phases: staging in the pre-staging buffer, latching for integrity, and batched writing to permanent storage. By segmenting the operation, the exclusive latch is held only during the critical staging phase rather than during the entire write process, allowing other operations to proceed in parallel and improving overall write throughput while maintaining data integrity.
Solution Approach 2:
Multiple data pages are merged into batched write operations from the pre-staging buffer to permanent storage. By combining multiple individual write operations into a single batched operation, the system reduces the frequency of latch acquisition and release, thereby improving write throughput while the exclusive latch still ensures data integrity for each batch.
Data Source
AI summary
Methods for local page writes via pre-staging buffers for resilient buffer pool extensions are performed by computing systems. Compute nodes in database systems insert, update, and query data pages maintained in storage nodes. Data pages cached locally by compute node buffer pools are provided to buffer pool extensions on local disks as pre-copies via staging buffers that store data pages prior to local disk storage. Encryption of data pages occurs at the staging buffers, which allows a less restrictive update latching during the copy process, with page metadata being updated in buffer pool extensions page tables with in-progress states indicating it is not yet written to local disk. When stage buffers are filled, data pages are written to buffer pool extensions and metadata is updated in page tables to indicate available/valid states. Data pages in staging buffers can be read and updated prior to writing to the local disk.