Database Dictionary Structures in Non-Volatile Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional persistence methods for non-volatile memory (NVM) are inefficient in loading data due to the need to persist pointers, leading to increased time requirements for loading persisted data into DRAM, especially when dealing with large volumes of data and complex data structures like database column fragments.
Innovation Solution
The solution involves writing data to NVM in larger binary data blocks with alignment gaps to ensure direct access and mapping to a virtual memory space, eliminating the need to persist pointers within NVM, and using a delta and main column fragment structure for database columns, where each fragment is written sequentially with one entry point, facilitating efficient pointer reconstruction during deserialization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is persisted in NVM using conventional tree format with pointers, then data durability is maintained, but loading time increases due to need to reload all pointer structures
Solution Approach 1:
The patent extracts pointers from the persisted data structure in NVM, storing only data values without associated pointers. This eliminates the need to reload and reconstruct pointer structures during data loading, directly reducing loading time while maintaining data durability through the persistent nature of NVM.
Solution Approach 2:
The patent segments the data storage into two distinct parts: (1) pointer structures kept in volatile DRAM for fast access during operations, and (2) pure data values persisted in NVM without pointers. This segmentation allows the system to maintain durability benefits of persistence while avoiding the overhead of persisting and reloading pointer metadata.
2Productivity
If larger binary data blocks are written to NVM, then data access efficiency improves through direct mapping, but memory alignment requirements increase complexity
Solution Approach 1:
The patent performs preliminary alignment of data blocks to memory boundaries (e.g., 64-byte alignment) before writing to NVM. By pre-aligning data during the persistence operation, the system enables direct memory-mapped access without requiring complex alignment logic during data retrieval, thus improving access efficiency while managing alignment complexity proactively.
Solution Approach 2:
The patent introduces an intermediary alignment layer that sits between the raw NVM storage and the application's virtual address space. This intermediary handles the translation and alignment of data blocks, allowing applications to access data efficiently through direct mapping while the intermediary manages the complexity of memory alignment requirements.
3Stability of the object's composition
If pointers are persisted in NVM, then data structure integrity is maintained, but storage space is wasted and loading performance deteriorates
Solution Approach 1:
The patent extracts and removes pointers from the persisted data structure in NVM, storing only the essential data values. This extraction eliminates wasted storage space on pointer metadata while maintaining data structure integrity through the use of columnar organization and dictionary structures that provide logical relationships without requiring physical pointers in the persisted format.
Solution Approach 2:
The patent applies local quality by maintaining different data representation strategies in different locations: pointers are maintained in volatile DRAM where they provide fast structural navigation, while only data values are persisted in NVM to save space. The columnar format and dictionaries provide local organization that maintains integrity without requiring global pointer structures.
Data Source
AI summary
A system includes storage, in a volatile random access memory, of a first data block comprising an array of distinct values of a database table column, and first header data comprising a first pointer to the first data block, determination of a memory size associated with the first header data and the first data block, allocation of a first memory block of the non-volatile random access memory based on the determined memory size, determination of an address of the non-volatile random access memory associated with the allocated first memory block, and writing of an indicator of the number of distinct values of the array and a binary copy of the first data block at the address of the non-volatile random access memory.