Relaxed Reliable Datagram Transport Service for High-Performance Computing Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-performance computing clusters face challenges in achieving high bandwidth and low latency due to the limitations of existing networking protocols, which often require significant processor involvement and are not scalable, especially in environments with frequent packet drops.
Innovation Solution
The implementation of a Relaxed Reliable Datagram transport service that separates packet reordering and reliability operations, where the network adapter device handles reliable delivery and the host device handles packet reordering, allowing for efficient data transfer across computing clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional networking protocols are used to ensure reliable packet delivery, then packet delivery reliability is improved, but processor involvement increases and latency increases
Solution Approach 1:
The patent segments the reliable packet delivery function into two parts: the network adapter device handles packet transmission and retransmission, while the host processor handles higher-level protocol operations. This segmentation allows the host processor to be bypassed for routine packet delivery, reducing latency while maintaining reliability through the network adapter's autonomous retransmission capabilities.
Solution Approach 2:
The network adapter device is empowered to autonomously handle packet retransmission without requiring host processor intervention. The adapter monitors packet delivery status and automatically retransmits dropped packets, enabling self-service reliable delivery that reduces processor involvement and latency while maintaining delivery reliability.
2Device complexity
If traditional networking protocols with kernel bypass framework are used, then processor involvement is reduced, but scalability deteriorates in environments with frequent packet drops
Solution Approach 1:
The patent implements dynamic path selection and adaptive retransmission strategies where the network adapter can switch between different network paths and adjust retransmission behavior based on observed network conditions. This dynamic adaptation enables the system to scale effectively in environments with frequent packet drops while maintaining low processor involvement.
Solution Approach 2:
The system incorporates feedback mechanisms where the network adapter monitors packet delivery success rates and network conditions, then adjusts its retransmission behavior and path selection accordingly. This feedback-driven adaptation allows the system to scale effectively in challenging network environments without increasing processor involvement.
3Reliability
If packet reordering is handled by the network adapter device, then reliable delivery is improved, but processor load increases
Solution Approach 1:
The patent segments the packet processing functions by having the network adapter device handle only packet transmission and basic reliability operations, while packet reordering is deferred to the host processor or application layer. This segmentation reduces the complexity and processor load at the network adapter while maintaining reliable delivery through coordinated reordering at the host level.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are systems and methods for connection establishment over a network that does not require a user application to establish an explicit connection with a target application. In some implementations, provided is an apparatus configured to communicate with a network and a host device. The apparatus may receive a message and destination information associated with the message from the host device. The apparatus may further determine, using the destination information, a transport context from a plurality of transport contexts. The transport context may include a state of a connection with a destination on the network. The destination on the network may be associated with the destination information.