Database Storage Engine NVM Buffer Pool Dynamic Array
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional relational database systems face performance bottlenecks when transitioning to non-volatile memory (NVM) due to inefficient components and technologies in the disk-oriented storage hierarchy, leading to suboptimal performance in online transaction processing (OLTP) under varying load characteristics.
Innovation Solution
A database storage engine is redesigned with a hierarchical storage architecture using NVM for mirrored data pages, mini-transactions, and a buffer pool with dynamic arrays for undo and modification logs, allowing only modified data to be flushed to NVM, reducing write amplification and enhancing concurrency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional disk-oriented database architecture is used with NVM, then storage capacity and non-volatility are improved, but performance bottleneck remains due to inefficient components and technologies
Solution Approach 1:
The patent extracts and removes inefficient components from the traditional database architecture when deployed on NVM. Specifically, it eliminates the redo log buffer and traditional write-ahead logging mechanisms that were necessary for disk-based systems but become bottlenecks on NVM. The invention also removes unnecessary buffering layers and optimization techniques designed for slow disk I/O, allowing the database to operate more efficiently directly on NVM's capabilities.
Solution Approach 2:
The patent changes key operational parameters of the database system to match NVM characteristics. This includes modifying write operation parameters to exploit NVM's persistent write capabilities, adjusting transaction management parameters to leverage NVM's durability, and tuning memory management parameters to utilize NVM's large capacity and fast access speeds. These parameter changes enable the system to achieve optimal performance on NVM hardware.
2Reliability
If redo log buffer is maintained for crash recovery, then reliability is improved, but startup time and write amplification increase
Solution Approach 1:
The patent removes the traditional redo log buffer component from the database architecture when running on NVM. Instead of maintaining a separate buffer for crash recovery logging, the invention leverages NVM's inherent persistence capabilities to provide crash recovery functionality. This extraction eliminates the overhead of maintaining and managing the redo log buffer, reducing both startup time and write amplification while preserving reliability through NVM's native durability features.
3Ease of operation
If traditional buffering and concurrency control mechanisms are used, then transaction processing is supported, but CPU time is wasted on auxiliary work
Solution Approach 1:
The patent modifies concurrency control parameters and buffering strategies to reduce CPU overhead. This includes adjusting lock granularity parameters, changing transaction isolation level settings, and optimizing buffer management parameters to reduce the frequency and complexity of auxiliary operations. These parameter changes allow the system to maintain transaction processing capabilities while minimizing the ratio of auxiliary CPU work to useful work.
Data Source
Figure 1
Figure 2
Figure 3~5
AI summary
Provided are a data page access method, a storage engine, and a computer-readable storage medium. The method includes: storing (S11) mirrored data pages of a buffer pool on a nonvolatile memory; and modifying (S12) a data page in the buffer pool according to a write access to the data page, and storing a page number of, undo log information and modification log information about the data page in a preset dynamic array.