RDMA Data Durability Flag for Persistent Memory Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed systems in large data centers face challenges in maintaining data consistency and durability across multiple nodes, particularly with the use of persistent memory (PMEM) and remote direct memory access (RDMA), where data may not be guaranteed to be persisted in non-volatile memory during power loss, leading to potential data loss and performance degradation.
Innovation Solution
Implementing a system that uses RDMA with a data_durability flag to ensure data is written to both cache and persistent memory, and employing an RDMA_read with a data_sync flag to explicitly flush data from memory buffers to persistent memory, thereby ensuring data durability and consistency across nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is written only to volatile memory for fast access, then access speed is improved, but data durability deteriorates (data loss on power loss)
Solution Approach 1:
The system performs preliminary actions by writing data to both volatile memory (for speed) and persistent memory (for durability) simultaneously during the write operation. The durability flag triggers the receiver to copy data to persistent memory before acknowledging the write, ensuring data is preserved before power loss can occur.
Solution Approach 2:
The patent introduces an intermediary mechanism (durability flag and sync flag) that mediates between the fast volatile memory and the durable persistent memory. This intermediary allows the system to leverage both memory types' strengths without sacrificing performance, as the flag system coordinates the dual-write operation transparently.
2Reliability
If data is written to both volatile memory and persistent memory, then data durability is improved, but write performance deteriorates
Solution Approach 1:
The patent merges the write operations to volatile memory and persistent memory into a single coordinated action. By combining these operations and using the durability flag to trigger the persistent memory write, the system achieves durability without the performance penalty of sequential writes, as both memory locations are updated in parallel.
Solution Approach 2:
The receiver side automatically handles the data duplication to persistent memory when it receives data with the durability flag set. This self-service mechanism eliminates the need for the sender to manage the dual-write complexity, maintaining high write performance while ensuring durability through automated local copying.
3Reliability
If data is flushed from memory buffer to persistent memory explicitly, then data consistency is improved, but latency increases
Solution Approach 1:
Instead of continuous or explicit flushing, the system uses periodic action through the sync flag mechanism. Data is flushed to persistent memory only when the sync flag is triggered, allowing data to remain in the fast memory buffer longer and reducing unnecessary flush operations, thereby minimizing latency while maintaining consistency when needed.
Solution Approach 2:
The system performs preliminary copying of data to persistent memory using the durability flag before the data is potentially needed for consistency checks. This preliminary action ensures that when consistency is required, the data is already in persistent memory, eliminating the need for late-stage flushing that would increase latency.
Data Source
AI summary
Examples described herein relates to a network interface apparatus that includes packet processing circuitry and a bus interface. In some examples, the packet processing circuitry to: process a received packet that includes data, a request to perform a write operation to write the data to a cache, and an indicator that the data is to be durable and based at least on the received packet including the request and the indicator, cause the data to be written to the cache and non-volatile memory. In some examples, the packet processing circuitry is to issue a command to an input output (IO) controller to cause the IO controller to write the data to the cache and the non-volatile memory. In some examples, the cache comprises one or more of: a level-0 (L0), level-1 (L1), level-2 (L2), or last level cache (LLC) and the non-volatile memory comprises one or more of: volatile memory that is part of an Asynchronous DRAM Refresh (ADR) domain, persistent memory, battery-backed memory, or memory device whose state is determinate even if power is interrupted to the memory device. In some examples, based on receipt of a second received packet that includes a request to persist data, the packet processing circuitry is to request that data stored in a memory buffer be copied to the non-volatile memory.