RDMA Distributed Storage Controller Lock-Free Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing RDMA-based systems face challenges in providing a scalable, thread-safe, and lock-free storage solution across multiple networked computers, leading to high CPU utilization and latency due to the need for CPU intervention in memory access operations.
Innovation Solution
A Distributed Storage Controller utilizing RDMA-based messaging enables self-directed, lock-free operations across shared memory distributed across multiple servers, allowing concurrent allocations, de-allocations, reads, and writes without CPU intervention through RDMA messages, with a reservation bitmap mechanism to manage memory access and prevent collisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional DMA-based techniques are used for local memory access, then memory access speed is improved, but CPU utilization increases due to interrupt handling
Solution Approach 1:
The patent replaces CPU-based interrupt handling with hardware-based RDMA mechanisms. The RDMA-enabled network interface controller directly manages memory access operations between remote devices, eliminating the need for CPU intervention in DMA operations. This substitution of the mechanical (CPU interrupt) system with a hardware-autonomous system resolves the contradiction by maintaining fast memory access while reducing CPU utilization.
Solution Approach 2:
The patent implements self-service by enabling remote devices to autonomously perform memory access operations without CPU notification. The RDMA mechanism allows the network interface controller to independently manage memory reads and writes, allocation and deallocation, without requiring the host CPU to be interrupted or notified. This self-service capability resolves the contradiction by maintaining efficient memory access while freeing the CPU from memory management tasks.
2Reliability
If CPU locks are used to manage shared memory access, then memory access safety is improved, but system throughput decreases due to locking overhead
Solution Approach 1:
The patent replaces CPU-based locking mechanisms with hardware-based atomic operations. The RDMA interface controller performs atomic compare-and-swap operations directly at the hardware level, eliminating the need for software locks. This substitution maintains memory access safety through atomicity guarantees while eliminating the throughput penalty associated with lock acquisition and release overhead.
Solution Approach 2:
The patent introduces an intermediary mechanism - the RDMA-enabled network interface controller - that mediates memory access between remote devices. This intermediary handles the coordination and safety guarantees that would otherwise require CPU locks, performing atomic operations and memory management autonomously. This resolves the contradiction by maintaining access safety through the intermediary's atomic operations while avoiding the throughput penalty of CPU-based locking.
3Reliability
If RDMA operations require CPU notification, then memory access control is improved, but latency increases due to CPU intervention
Solution Approach 1:
The patent substitutes CPU-based notification and control with hardware-autonomous RDMA operations. The network interface controller directly executes memory access operations and manages memory allocation without requiring CPU intervention or notification. This substitution maintains reliable memory access control through hardware-level atomic operations while eliminating the latency introduced by CPU context switches and notification mechanisms.
4Stability of the object's composition
If a centralized storage controller is used, then memory management consistency is improved, but system scalability decreases due to single point of failure
Solution Approach 1:
The patent segments the centralized storage controller functionality into distributed components across multiple networked devices. Each device maintains local memory management consistency through RDMA operations while collectively providing scalable storage capacity. This segmentation eliminates the single point of failure limitation while maintaining consistency through distributed atomic operations and the reservation bitmap mechanism.
Solution Approach 2:
The patent transitions from a single-point (centralized) controller to a multi-point (distributed) controller architecture. By distributing memory management responsibilities across multiple networked devices, the system achieves both consistency through coordinated RDMA operations and scalability through parallel access paths. This dimensional change from 1D (single controller) to N-D (distributed controllers) resolves the contradiction between consistency and scalability.
Data Source
AI summary
A “Distributed Storage Controller” applies an RDMA-based memory allocator to implement a distributed thread-safe and lock-free storage system in shared memory distributed across multiple networked computers. The Distributed Storage Controller enables any number of client applications, processes, subroutines, etc., on any number of networked computers to concurrently perform self-directed lock-free allocations, de-allocations, reads, writes, etc., on the shared memory via sequences of one-way RDMA messages (e.g., RDMA reads, RDMA writes, and RDMA atomic compare and swap (CAS) operations) without requiring CPU locks. As such, the CPUs of computing devices hosting the shared memory do not need to be notified of RDMA-based reads, writes or CAS operations on that memory. Consequently, CPU utilization for computing devices hosting any portion of the shared memory is minimal, thereby enabling the Distributed Storage Controller to scale to very large numbers of concurrent accesses by very large numbers of networked computing devices.