Inter-SmartNIC Virtual Link for Direct Data Plane Connectivity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing complexity of network traffic management in multi-smartNIC servers, particularly in supporting various features that require communication between smartNICs, leads to challenges such as packet loss and reduced traffic throughput due to the need for host device support and network entity involvement.
Innovation Solution
Establishing a virtual link between programmable IO devices, such as smartNICs, using PCIe peer-to-peer communication or a shared PCIe bus, to create a data plane that allows direct communication and reduces reliance on host devices and network entities, enabling features like link aggregation, state synchronization, and ERSPAN traffic redirection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple smartNICs are deployed to support various networking features, then the networking capability and feature support are improved, but the device complexity and communication overhead increase
Solution Approach 1:
A virtual link is introduced as an intermediary communication channel between smartNICs, enabling direct peer-to-peer communication without requiring host device intervention. This virtual link acts as a dedicated pathway that simplifies the complex interactions between multiple smartNICs while maintaining full networking feature support.
Solution Approach 2:
The communication path is segmented into direct smartNIC-to-smartNIC channels through the virtual link, separating the control plane communication from the data plane traffic. This segmentation reduces the overall system complexity by creating independent communication pathways that don't require host device mediation.
2Ease of operation
If host device support is used for smartNIC communication, then the system is easier to manage, but packet loss increases and traffic throughput decreases
Solution Approach 1:
The virtual link serves as a dedicated intermediary channel that bypasses the host device for data plane communication between smartNICs. This eliminates the bottleneck effect of host device processing while maintaining manageability through control plane oversight, thereby increasing traffic throughput without sacrificing ease of operation.
Solution Approach 2:
A new communication dimension is created through the virtual link that operates independently from the traditional host-mediated pathway. This additional communication dimension allows parallel operation of control and data planes, enabling high throughput data transfer while the host device continues to provide management functions through the control plane.
3Reliability
If traditional network entity involvement is used for packet routing, then the network control is more centralized, but packet loss increases and latency increases
Solution Approach 1:
The data plane packet routing function is extracted from traditional network entities and moved directly to the smartNICs through the virtual link. This extraction allows packets to be routed directly between smartNICs without the intermediate delays of traditional network entity processing, reducing latency while maintaining reliability through preserved control plane oversight.
Solution Approach 2:
The virtual link is pre-established between smartNICs before data plane communication begins. This preliminary setup creates ready-to-use direct pathways that eliminate the need for real-time routing decisions through traditional network entities, thereby reducing packet latency while maintaining network control stability through pre-configured reliable channels.
Data Source
AI summary
Described are programmable IO devices installed on a host device and configured to execute instructions that cause the programmable IO device to perform operations to establish a virtual link between another programmable IO device installed on the host device and provide a data plane using the virtual link. These operations comprise: establishing the virtual link with the other programmable IO device installed on the host device, wherein the virtual link provides a communication channel between the programmable IO devices; providing the data plane by establishing, with the other programmable IO device via the virtual link, a data path associated with the data plane; receiving a packet in the data plane destined for the other programmable IO device; and forwarding the packet to the other programmable IO device via the virtual link.