Persistent Memory Write-Ahead Logging with Checksums
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file system logging techniques are inefficient when applied to persistent memory, as they often rely on volatile RAM, leading to data loss during crashes and inconsistent log states, which complicates crash recovery and metadata table integrity.
Innovation Solution
Writing log data directly to persistent memory, using checksums to differentiate good and bad log pages during recovery, and employing non-temporal copies to bypass CPU caches and ensure immediate data hardening, thereby enhancing crash resilience and recovery efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If log data is staged in volatile RAM before writing to disk, then writing speed is fast, but data loss occurs during crashes
Solution Approach 1:
The patent creates a copy of the log data structure in persistent memory that mirrors the in-memory log buffer. This copy serves as a durable backup that survives crashes, allowing the system to recover without data loss while maintaining fast in-memory writing performance.
Solution Approach 2:
The patent introduces persistent memory as an intermediary layer between volatile RAM and disk storage. This intermediary provides both the speed characteristics of memory and the durability of persistent storage, eliminating the trade-off between writing speed and crash resilience.
2Reliability
If log data is written directly to persistent memory, then crash resilience is improved, but write performance may be reduced
Solution Approach 1:
The patent segments the logging system into two independent components: an in-memory log buffer for fast writes and a persistent memory log structure for durability. Each segment operates independently with optimized characteristics, allowing the system to achieve both high write performance and crash resilience simultaneously.
Solution Approach 2:
The patent performs preliminary actions by pre-allocating and initializing the persistent memory log structure before it is needed for crash recovery. This includes setting up the log segment directory, allocating pages, and establishing the initial state, so that during normal operation only incremental updates are needed, maintaining high performance.
3Reliability
If checksums are inserted in log pages, then recovery reliability is improved, but storage space is consumed
Solution Approach 1:
The patent changes the parameter of checksum placement from traditional locations to specific reserved fields within the log page structure. By optimizing the placement and size of checksum fields, the system achieves reliable recovery verification while minimizing the space consumed by redundancy data.
4Reliability
If CPU caches are bypassed for log writes, then data hardening is immediate, but CPU performance is reduced
Solution Approach 1:
The patent extracts the logging write operations from the main CPU execution path by using dedicated persistent memory write commands that bypass the CPU cache hierarchy. This separation allows critical log data to be hardened immediately without interfering with general CPU performance for other operations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The techniques disclosed herein improve performance of file system logging by writing log data to persistent memory instead of staging in RAM before writing to disk. In one embodiment, while the log is being written, checksums are inserted, such that during recovery, the checksums can be used to distinguish good log pages from bad log pages. In this way, good log pages can be evaluated to determine whether to roll a portion of a file system transaction forward, backward, or do nothing, while bad log pages can be safely ignored. Additionally or alternatively, non-temporal copies are employed when writing data to persistent memory, thereby reducing an amount of time log data is exposed to be lost in a crash.