PMEM Distributed Memory Object System for Legacy Server Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed memory systems fail to provide low-latency access to persistent memory across multiple nodes while integrating with legacy devices that do not support persistent memory technology, and they do not effectively combine memory-speed data access with disk-like storage density, leading to high costs and inefficiencies in large-scale deployments.
Innovation Solution
A PMEM-based distributed memory object system that creates a cluster of PMEM servers and application servers connected via a high-speed network, allowing client applications to directly write and read data from remote PMEM servers using a DMO agent, which manages data replication and caching to optimize access and storage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If PMEM-based distributed memory object system is deployed to provide low-latency memory-speed data access across multiple nodes, then access latency is reduced and storage density is increased, but system complexity and deployment cost increase significantly
Solution Approach 1:
The system segments the distributed memory object into smaller chunks that can be stored across multiple PMEM nodes. Each node manages a portion of the total data space, allowing parallel access and reducing the complexity of managing a single large memory space across distributed nodes.
Solution Approach 2:
The patent introduces a coordinator process as an intermediary between client applications and the distributed PMEM storage. This coordinator manages the complexity of data placement, replication, and access coordination across nodes, abstracting the complexity from the client applications while enabling low-latency access.
2Speed
If all legacy servers are replaced with PMEM-compatible servers to enable direct access to persistent memory, then data access performance improves, but deployment cost increases significantly
Solution Approach 1:
The system design allows both PMEM-native servers and legacy servers to participate in the distributed memory object system. PMEM servers provide high-performance direct access, while legacy servers can access data through the distributed file system interface, making the system universally compatible across different hardware generations without requiring complete replacement.
Solution Approach 2:
Instead of requiring expensive PMEM hardware for all servers, the patent enables legacy servers to function as part of the distributed system through software abstraction. This allows the system to use a mix of expensive PMEM nodes for critical storage functions and cheaper legacy servers for general computation, optimizing the cost-performance ratio.
3Quantity of substance
If data is frequently moved between DRAM and storage tiers to handle working datasets exceeding memory capacity, then storage density is increased, but access latency increases and performance deteriorates
Solution Approach 1:
The system performs preliminary actions by pre-loading frequently accessed data into PMEM storage and maintaining it in a readily accessible state. The distributed memory object structure allows data to be kept in PMEM rather than requiring frequent transfers between DRAM and storage, reducing the time lost during data movement while still providing access to large datasets.
Data Source
AI summary
Disclosed herein is a persistent memory (PMEM)-based distributed memory object system, referred to as the PMEM DMO system, that provides affordable means of integrating low-latency PMEM spaces with other devices, including servers that do not directly support PMEM. One embodiment comprises providing a cluster of servers with PMEM storage (PMEM servers) and connecting the PMEM servers to a plurality of applications servers using a low-latency network, such as a remote direct memory access; background processes on each of the application servers are tasked to perform input/output operations for the application servers to locally materialize objects from and synchronize/persist objects to the remote PMEM spaces on the PMEM servers. Data materialized from the PMEM servers is stored to the local cache of the application server for use. Also disclosed are data eviction policies for clearing the local cache of the application servers to make space for new data read.