Persistent Memory Index Views in NUMA Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for constructing persistent memory indexes in non-uniform memory access (NUMA) architectures fail to guarantee crash consistency and efficiently utilize persistent memory space and write bandwidth, leading to exponential consumption and performance issues.
Innovation Solution
The method involves maintaining partial persistent views in persistent memory and a global volatile view in DRAM, with foreground threads processing requests for cold data and background threads managing hot data, generating new views and recycling old ones, ensuring crash consistency and optimizing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If persistent memory index is constructed in NUMA architecture with multiple NUMA nodes, then the system can support larger-scale systems with more cores and DIMM slots, but the persistent memory space and write bandwidth are exponentially consumed due to the need to maintain index copies on each NUMA node
Solution Approach 1:
The patent divides the persistent memory index into multiple partial persistent views, with each NUMA node maintaining only its local partial view. This segmentation eliminates the need for each node to store complete index copies, reducing persistent memory consumption from exponential to linear growth with the number of NUMA nodes.
Solution Approach 2:
The patent introduces a global volatile view in DRAM as an intermediary that coordinates access to the distributed partial persistent views. This mediator enables crash consistency across NUMA nodes without requiring redundant persistent memory storage, as the volatile view temporarily holds coordination information during operations.
2Adaptability or versatility
If persistent memory index is constructed in NUMA architecture, then the system can support larger-scale systems, but the write bandwidth of persistent memory is exponentially occupied due to synchronous updates across all NUMA nodes
Solution Approach 1:
The patent segments the index update operations so that each NUMA node only updates its local partial persistent view. This eliminates the need for synchronous writes to all NUMA nodes, reducing persistent memory write bandwidth consumption from exponential to linear scaling with system size.
Solution Approach 2:
The global volatile view acts as a mediator that coordinates updates across NUMA nodes in memory before persistence is required. This allows asynchronous local updates without immediate remote persistence, dramatically reducing the write bandwidth burden on persistent memory while maintaining eventual consistency.
3Productivity
If existing methods are used to construct persistent memory index, then the index can be maintained, but crash consistency cannot be guaranteed due to the distributed nature of NUMA architecture
Solution Approach 1:
The global volatile view serves as a coordination intermediary that manages crash consistency across distributed NUMA nodes. It tracks the state of partial persistent views and enables recovery mechanisms that guarantee crash consistency without requiring redundant persistent storage at each node.
Solution Approach 2:
The patent performs preliminary actions by maintaining the global volatile view that pre-coordinates update states before crashes can occur. This preliminary coordination structure enables efficient recovery after crashes, as the volatile view preserves enough state information to restore consistency without requiring complex distributed recovery protocols.
4Speed
If partial persistent views are maintained in persistent memory and global volatile view in DRAM, then remote access to persistent memory is reduced, but additional memory resources are required
Solution Approach 1:
The patent applies local quality by having each NUMA node maintain its partial persistent view locally in persistent memory, enabling fast local access without remote memory operations. The global volatile view in DRAM provides fast coordination for hot data, optimizing the quality of service for different access patterns while using appropriate memory types for each function.
Data Source
AI summary
A method for constructing a persistent memory index in a non-uniform memory access architecture includes: maintaining partial persistent views in a persistent memory and maintaining a global volatile view in a DRAM; an underlying persistent memory index processing a request in a foreground thread when cold data is accessed; when hot data is accessed, reading a key-value pair for a piece of hot data in the global volatile view in response to a query operation carried in the request, and in response to an insert/update/delete operation carried in the request, updating a local partial persistent view and the global volatile view; and in response to a hotspot migration, a background thread generating new partial persistent views and a new global volatile view, and recycling the partial persistent views and the global volatile view for old hot data into the underlying persistent memory index.