Persistent Memory File Store for Lockless Database Block Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Byte addressable persistent memory (PMEM) poses data integrity risks due to lack of block atomicity, leading to torn database blocks when readers and writers concurrently access database blocks, which can result in reduced throughput and deadlocks in relational database management systems (RDBMS).
Innovation Solution
The FsDirect file store allocates new database blocks on write, copies new content into the block, and atomically reassigns metadata pointers to ensure data integrity by always accessing the latest version, allowing concurrent reading and writing without waiting or deadlocks, using a lockless and non-blocking approach.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a writer waits for exclusive use of a database block to prevent torn blocks, then data integrity is improved, but RDBMS throughput is reduced and deadlocks may occur
Solution Approach 1:
The patent segments the database block access into two independent parts: metadata (pointers) and data content. By making the metadata byte-addressable and separately addressable from the data, the system allows concurrent access where readers access metadata to locate data blocks while writers modify data blocks, eliminating the need for writers to wait for exclusive access and preventing deadlocks while maintaining data integrity
Solution Approach 2:
The patent introduces metadata as an intermediary layer between the reader and the data block. The metadata contains pointers that indirect access to the actual data blocks. This intermediary allows readers to obtain block locations without blocking writers, and enables the system to track and manage concurrent accesses, resolving the contradiction between data integrity and throughput
2Reliability
If a reader always reads a copy of the database block to avoid torn blocks, then data integrity is improved, but RDBMS throughput is reduced
Solution Approach 1:
The patent applies preliminary action by having readers obtain metadata (pointer information) in advance before accessing data blocks. The metadata is read and cached, allowing readers to locate and access data blocks without needing to wait for writers or read entire block copies. This preliminary metadata access enables efficient concurrent access patterns that maintain data integrity while preserving throughput
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques herein store database blocks (DBBs) in byte-addressable persistent memory (PMEM) and prevent tearing without deadlocking or waiting. In an embodiment, a computer hosts a DBMS. A reader process of the DBMS obtains, without locking and from metadata in PMEM, a first memory address for directly accessing a current version, which is a particular version, of a DBB in PMEM. Concurrently and without locking: a) the reader process reads the particular version of the DBB in PMEM, and b) a writer process of the DBMS replaces, in the metadata in PMEM, the first memory address with a second memory address for directly accessing a new version of the DBB in PMEM. In an embodiment, a computer performs without locking: a) storing, in PMEM, a DBB, b) copying into volatile memory, or reading, an image of the DBB, and c) detecting whether the image of the DBB is torn.