Database Journaling via Non-Volatile System Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database systems face inefficiencies in persisting log records due to the limitations of off-board storage devices, which can lead to increased recovery times and vulnerability during failures, as they require higher latency and are costly.
Innovation Solution
Implementing a hybrid system memory with a non-volatile portion that allows direct storage and retrieval of log records, reducing latency and increasing durability by using non-volatile system memory for journaling, and managing the allocation and use of both volatile and non-volatile memory to optimize performance and recovery processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If log records are persisted to off-board storage devices, then durability is improved, but latency increases and performance deteriorates
Solution Approach 1:
The patent transitions from traditional off-board storage devices to non-volatile system memory, changing the storage dimension from external to integrated. This allows log records to be persisted in memory space that is both fast (system memory speed) and durable (non-volatile persistence), resolving the contradiction between durability and latency.
Solution Approach 2:
The patent creates a copy mechanism where log records are first written to non-volatile system memory for durable storage, and then selectively flushed to off-board storage devices. This copying approach allows the system to achieve durability through the non-volatile memory while avoiding the latency penalty of direct writes to slow storage devices.
2Reliability
If log records are flushed to persistent storage devices, then reliability is improved, but device limitations reduce journaling effectiveness
Solution Approach 1:
The patent segments the log record storage function into two parts: non-volatile system memory for active journaling operations and off-board storage devices for archival persistence. This segmentation allows the high-speed non-volatile memory to handle frequent journaling writes while the slower storage devices handle periodic flushing, thereby maintaining journaling effectiveness while ensuring reliability.
Solution Approach 2:
The patent introduces non-volatile system memory as an intermediary between the database system and off-board storage devices. This intermediary layer absorbs the performance limitations of storage devices by providing a fast, non-volatile buffer that maintains journaling effectiveness while periodically flushing data to persistent storage for reliability.
3Loss of time
If non-volatile system memory is used for log records, then latency is reduced and durability is improved, but memory management complexity increases
Solution Approach 1:
The patent implements dynamic memory management for non-volatile system memory, where the allocation and flushing of log records is adaptive based on system conditions. The system dynamically decides when to flush logs to off-board storage based on factors like memory availability and system state, reducing management complexity while maintaining performance benefits.
Data Source
AI summary
A database system may implement efficient database journaling using non-volatile system memory. An update to a database may be received, and log records indicating the update may be generated. A non-volatile portion of the system memory may be directly accessed to store the generated log records such that in the event of a failure of the database, the log records may be still maintained. Log records stored to the non-volatile portion of the system memory may be identified as durably persisted, in some embodiments. Log records may be identified and sent to block-based storage, in some embodiments. Recovery from a database system failure, in some embodiments, by obtaining log records from the non-volatile portion of system memory.