Distributed Naming Service for Persistent Memory Object Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed computing solutions fail to provide low-latency access to data objects that span memory and storage across multiple nodes, lacking support for both memory and file storage semantics, which is essential for high-performance computing tasks exceeding DRAM capacity and requiring frequent data movements between DRAM and lower storage tiers.
Innovation Solution
The MemVerge DMO system implements a naming service in a distributed memory object architecture that allows persistent distributed memory objects to be accessed as either in-memory or file-storage mode, using RDMA for low-latency data access, with a master key value store replicated across nodes for directory pathnames and utilizing user space, kernel, or hypervisor approaches to manage larger memory spaces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional storage systems are used, then storage capacity and density are improved, but access latency increases significantly
Solution Approach 1:
The system segments storage into three distinct tiers: persistent memory (PMEM) for high-performance access, SSD for intermediate storage, and HDD for bulk capacity. This segmentation allows each tier to be optimized for its specific function, with PMEM providing memory-speed access for frequently accessed data while HDD provides economical bulk storage.
Solution Approach 2:
The patent introduces an intermediary memory hierarchy between traditional RAM and storage devices. The persistent memory layer acts as a mediator that provides storage density comparable to HDDs while delivering access speeds comparable to RAM, thereby bridging the performance gap between memory and storage.
2Speed
If memory-speed data access is implemented, then access latency is reduced, but storage persistence and reliability are compromised
Solution Approach 1:
The patent changes the fundamental parameters of memory by using persistent memory technology that combines the speed characteristics of volatile memory with the persistence characteristics of non-volatile storage. This parameter change enables data to be retained without power while maintaining fast access speeds.
Solution Approach 2:
The system creates a composite storage architecture that combines different storage technologies (PMEM, SSD, HDD) with complementary characteristics. Each layer contributes its strengths: PMEM provides speed and persistence, SSD provides capacity and speed, and HDD provides bulk capacity at low cost.
3Quantity of substance
If distributed memory architecture is used, then memory capacity is increased, but access latency and complexity increase
Solution Approach 1:
The patent implements distributed copying of data across multiple nodes and storage tiers. Each node maintains local copies of frequently accessed data in persistent memory, while replicas are distributed across the network. This copying strategy enables parallel access and reduces latency while distributing the complexity burden.
Solution Approach 2:
The system adds a new dimension to the memory hierarchy by introducing persistent memory as an intermediate layer between volatile memory and traditional storage. This dimensional addition creates a four-tier hierarchy that manages complexity through structured organization rather than simple expansion.
4Adaptability or versatility
If frequent data movements between DRAM and storage tiers are performed, then data access flexibility is improved, but performance degradation occurs
Solution Approach 1:
The patent implements preliminary action by pre-loading frequently accessed data into persistent memory and maintaining it in a readily accessible state. This preparation reduces the need for frequent data movements during computation, as data is already positioned in the optimal storage tier for the given access pattern.
Solution Approach 2:
The system uses feedback mechanisms to monitor data access patterns and dynamically adjust data placement decisions. By analyzing access frequencies and patterns, the system can optimize which data resides in which tier, reducing unnecessary data movements while maintaining flexibility for different access scenarios.
Data Source
AI summary
Disclosed herein is an apparatus and method for a naming service in a distributed memory object system. In one embodiment, a name service method includes electing a primary node for the master key value store from a plurality of name service nodes, the primary node to receive master key value requests, a master key value store containing an entry for each directory within the distributed memory object, wherein the master key value store is configured for associating a directory pathname to a uniform unique identifier, and replicating the master key value store across the plurality of name service nodes. In another embodiment, a name service computing device includes a primary node for the master key value store, wherein the primary node is selected from a plurality of name service nodes and is configured to receive master key value requests, a master key value store containing an entry for each directory within the distributed memory object, wherein the master key value store is configured to associate a directory pathname to a uniform unique identifier, and the system further being in communication with the plurality of name service nodes, wherein the master key value store is replicated across the plurality of name service nodes.