RDMA Network Architecture Reducing Latency via Kernel Bypass
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network architectures and protocols for remote direct memory access (RDMA) suffer from increased latency due to the need for intermediate or kernel copies of data during remote read and write operations, which slows down data transfer between nodes.
Innovation Solution
A lightweight network architecture and protocol are implemented, using a constrained usage model that minimizes kernel copies by directly accessing and transferring data between nodes, employing a direct transport protocol (DTP) with features like static transmit windows and piggyback acknowledgments, and relying on native data integrity checks and OSI layer 2 routing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional network architecture with kernel copies is used, then data transfer reliability is maintained, but latency increases and transfer speed decreases
Solution Approach 1:
The patent extracts and removes the kernel copy mechanism from the data transfer path. By using RDMA technology, data transfers directly between application memory buffers without being copied through the kernel, eliminating an intermediate processing step that caused latency and reduced transfer speed.
Solution Approach 2:
The patent introduces a network interface card (NIC) with RDMA capabilities as an intermediary device that enables direct memory access between nodes. The NIC handles data transfer and integrity checks independently, allowing bypass of the traditional kernel copy path while maintaining reliability through hardware-based error checking.
2Reliability
If comprehensive data integrity checking is implemented, then transfer reliability improves, but processing complexity and overhead increase
Solution Approach 1:
The patent replaces software-based integrity checking mechanisms with hardware-based cyclic redundancy check (CRC) validation in the NIC. This substitution moves the complexity from the software layer to dedicated hardware circuits, providing reliable data validation without increasing processing overhead or system complexity.
Solution Approach 2:
The NIC performs self-validation of data integrity using hardware-based CRC checks automatically during data reception. This self-service mechanism validates data without requiring additional processing from the host system, maintaining reliability while minimizing overhead.
3Loss of time
If lightweight protocol with constrained usage model is used, then latency is reduced and speed is improved, but adaptability to different network conditions decreases
Solution Approach 1:
The patent uses fixed parameter values for data transfer operations, such as predetermined buffer sizes, queue pair configurations, and static transmit windows. These parameter changes from dynamic to fixed values reduce processing overhead and latency, optimizing performance for the target use case of enterprise clusters with stable network conditions.
Data Source
AI summary
Remote direct memory access over a network uses a constrained usage model. In one embodiment, a method of accessing memory on a network comprises a first node of the network generating a remote direct memory access request to remotely access application-level memory on a second node of the network. The first node sends the request in a packet over the network to the second node, the sending being performed in accordance with a constrained usage model. In a further embodiment, constrained usage model comprises sending the packet in an Ethernet protocol packet over an Ethernet network, the Ethernet protocol packet having a OSI layer two header, but not a layer three header.