Network Interface Payload Computation with Header-Only Forwarding for RDMA
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing in-network computation solutions for distributed deep neural network (DNN) training face issues with connection breaks due to network interface devices processing packets, leading to undetectable packet losses and network congestion, which disrupt RDMA reliable connections and hinder efficient data aggregation and multicast operations.
Innovation Solution
Implementing programmable match-action units in network interface devices to perform computations on data packets, managing RDMA connections, and maintaining data processing and transport protocol separation between the network interface device and the endpoint host, allowing for in-network aggregation and multicast operations while ensuring reliable data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If network interface devices process packets to reduce network traffic through in-network computation, then network efficiency is improved, but connection reliability deteriorates due to undetectable packet losses
Solution Approach 1:
The patent segments packet processing functions between network interface devices and endpoint hosts. The network interface device processes packet payloads for computation while forwarding headers to endpoint hosts for connection management. This segmentation allows in-network computation to improve efficiency while preserving endpoint control over connection reliability through header-only forwarding and retransmission protocols.
Solution Approach 2:
The patent introduces an intermediary mechanism where the network interface device processes packet payloads but forwards packet headers to endpoint hosts. The headers contain metadata that enables endpoints to detect packet losses and manage retransmissions. This intermediary approach allows the network interface device to perform computation while maintaining endpoint awareness for reliability management.
2Quantity of substance
If network interface devices perform in-network aggregation to reduce traffic, then network bandwidth utilization is improved, but packet loss detection capability deteriorates
Solution Approach 1:
The patent extracts the packet header from the payload and processes them separately. The network interface device performs aggregation on extracted payload data while forwarding headers to endpoint hosts. This extraction allows the system to reduce traffic volume through payload aggregation while maintaining packet loss detection capability through header forwarding and endpoint-based sequence number tracking.
Solution Approach 2:
The patent implements feedback mechanisms where packet headers contain metadata (such as sequence numbers) that enable endpoint hosts to detect packet losses. The endpoint hosts receive feedback about which packets were successfully delivered and can request retransmissions. This feedback loop maintains detection capability even when the network interface device performs aggressive payload aggregation.
3Productivity
If network interface devices process full packets including headers, then computation capability is improved, but device complexity increases
Solution Approach 1:
The patent segments packet processing into two parts: the network interface device processes only packet payloads for computation, while packet headers are forwarded to endpoint hosts for transport protocol processing. This segmentation reduces device complexity at the network interface level while maintaining computation capability on payload data, avoiding the need for the network interface device to implement complex transport protocols.
4Reliability
If packet retransmission is implemented at endpoint hosts, then connection reliability is improved, but network latency increases
Solution Approach 1:
The patent implements preliminary actions by having the network interface device perform payload processing and aggregation before packets reach endpoint hosts. This reduces the amount of data that needs to be retransmitted if packet losses occur, thereby reducing the time penalty associated with retransmissions while maintaining connection reliability through endpoint-based retransmission protocols.
Data Source
AI summary
Examples described herein relate to a switch comprising: circuitry, when operational, to receive a packet comprising a header and a payload and in conjunction with performance of computation on the packet payload, forward the packet header, but not the payload, to a destination endpoint. In some examples, the destination endpoint of the packet is to perform management of reliable transport. In some examples, the circuitry includes programmable data plane circuitry comprising ingress pipeline or egress pipeline and one or more match action units (MAUs) to perform processing of the payload, wherein the programmable data plane circuitry is to perform computation on the packet payload.