Hardware RDMA Transport with Selective Recovery for Packet Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware-based RDMA transports suffer from reduced goodput in best-effort datacenter networks due to packet loss, network congestion, asymmetric goodput, and inefficient resource allocation, among other issues, leading to suboptimal performance and scalability.
Innovation Solution
A hardware-based RDMA transport system that minimizes on-chip complexity, supports all RDMA operations, and decouples from congestion-control solutions, utilizing packet-level interleaving and selective acknowledgment to optimize resource allocation and improve goodput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If GoBackN reliability protocol is used for packet loss recovery, then reliability is improved, but all correctly received packets posterior to a lost packet must be retransmitted, causing loss of time and reduced productivity
Solution Approach 1:
The patent implements selective acknowledgment (SACK) mechanism that allows the receiver to identify and acknowledge specifically which packets have been received correctly. When packet loss is detected, only the lost packets are retransmitted rather than forcing retransmission of all subsequent packets. This selective recovery approach recovers from packet loss while minimizing unnecessary retransmissions of already-received data.
2Reliability
If PFC is deployed to resolve packet loss in Ethernet LANs, then reliability is improved, but head-of-line blocking and congestion spreading occur, increasing device complexity and reducing productivity
Solution Approach 1:
The patent implements explicit feedback mechanisms through acknowledgment (ACK) and negative acknowledgment (NAK) packets. The receiver sends ACK packets to confirm receipt of packets and NAK packets to indicate packet loss. This feedback loop enables the sender to make informed retransmission decisions without requiring complex network-wide congestion control mechanisms like PFC, thereby reducing device complexity while maintaining reliability.
3Productivity
If out-of-order placement is used to improve throughput, then productivity is improved, but no signaling exists to notify the sender about receiver running out of tracking resources, causing loss of information
Solution Approach 1:
The patent extends the feedback mechanism to include resource status reporting. When the receiver's out-of-order tracking resources are depleted, the system can send appropriate feedback signals to the sender. This enables the sender to adjust its transmission strategy, such as reducing the rate of out-of-order packets or implementing flow control, thereby preventing information loss while maintaining high throughput through continued out-of-order placement.
4Reliability
If static retransmission timeout with maximum attempts is used, then reliability is improved, but goodput degrades when tail drops occur, reducing productivity
Solution Approach 1:
The patent implements dynamic retransmission timeout adjustment based on network conditions and packet characteristics. Rather than using a fixed static timeout, the system adapts the timeout values dynamically. When tail drops occur, the dynamic timeout mechanism can adjust to avoid premature retransmission attempts that would fail, thereby maintaining reliability guarantees while preventing goodput degradation through intelligent retry timing.
Data Source
AI summary
A best efforts (BE) hardware remote direct memory access (RDMA) transport being performed by a smart network interface controller (NIC). Elements from RoCEv2 and iWARP are utilized in combination with extensions to improve flexibility and packet error recovery. Flexibility is provided by allowing RDMA roles to be individually specified. Flexibility is also provided by additional packet numbering options to allow interleaving of request and response messages at a packet boundary. Error recovery is improved by utilized new acknowledgement responses, SNAK provided for each new hole detected and RACK for each received packet after a SNAK. SNAK allows the indication of resource exhaustion at the receiver, causing entry into a recovery mode where only packets in a hole are transmitted until resources are recovered.


