In-Memory Database Non-Volatile Primary Storage LRU Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In-memory databases face challenges in maintaining data consistency and processing speed due to the limitations of volatile memory, particularly when using non-volatile memory as primary storage, as existing methods like Everysec and Always flushing impact performance and capacity.
Innovation Solution
An in-memory database system that prioritizes non-volatile memory for primary storage and volatile memory for excess data, periodically storing log files in a block device, using a least recently used (LRU) policy to manage data distribution and ensure consistency, thereby overcoming capacity limitations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If non-volatile memory is used as primary storage in in-memory database, then data consistency is improved, but storage capacity is limited
Solution Approach 1:
The patent segments the storage system into two distinct parts: non-volatile memory for primary storage to ensure data consistency, and volatile memory for secondary storage to expand capacity. This segmentation allows each memory type to fulfill its strength - non-volatile memory provides reliability while volatile memory provides capacity, resolving the contradiction between data consistency and storage capacity.
2Reliability
If flushing is performed always to maintain data consistency, then data consistency is improved, but processing speed is lowered
Solution Approach 1:
The patent implements periodic flushing where data is written to non-volatile memory at regular intervals rather than on every transaction. This periodic action maintains data consistency over time while avoiding the performance penalty of synchronous flushing on every operation, thus resolving the contradiction between data consistency and processing speed.
Solution Approach 2:
The patent introduces volatile memory as an intermediary layer between the application and non-volatile memory. This intermediary allows fast writes to volatile memory while periodically syncing to non-volatile memory, maintaining consistency without blocking processing speed. The intermediary absorbs the conflict between immediate consistency requirements and performance demands.
3Productivity
If flushing is performed every second in background thread, then processing speed is maintained, but data consistency becomes unstable
Solution Approach 1:
The patent performs preliminary writing of data to non-volatile memory before it is truly needed, using periodic flushing intervals. This preliminary action ensures that data is already persisted when needed, providing stable consistency without requiring synchronous flushing that would slow down processing. The system prepares data in advance at regular intervals.
Data Source
AI summary
The exemplary embodiments provide an in-memory database which uses a non-volatile memory as a primary storage, uses a volatile memory for data which exceeds a predetermined capacity of the non-volatile memory, as a secondary storage, and periodically stores a log file for data stored in the volatile memory in a block device, thereby ensuring a data consistency while overcoming a capacity limit of the non-volatile memory.