InfiniBand Network Virtualization via Private Virtual Interconnects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
InfiniBand networks face limitations in isolating communications and efficiently supporting virtualization, particularly due to the need for all inter-server communications to pass through an I/O director, which introduces bandwidth, latency, and throughput limitations, and is not scalable for systems requiring thousands of virtual networks.
Innovation Solution
Implementing private virtual interconnects (PVIs) using virtual network interface cards (VNICs) that encapsulate Ethernet packets within InfiniBand messages, allowing for logically isolated communications and eliminating the need for a centralized controller, with mechanisms for failover and autonomous discovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If all inter-server communications pass through an I/O director, then centralized control and management is simplified, but bandwidth, latency, and throughput are limited and scalability is reduced
Solution Approach 1:
The patent segments the communication paths by creating multiple virtual interconnects (PVIs) that allow servers to communicate directly without all traffic passing through the I/O director. This segmentation enables parallel communication channels, increasing overall bandwidth and throughput while maintaining centralized management capabilities through virtual network interface cards (VNICs).
Solution Approach 2:
The patent introduces a virtualization dimension by implementing PVIs and VNICs that create logical communication layers above the physical InfiniBand fabric. This additional dimension allows multiple isolated virtual networks to coexist on the same physical infrastructure, enabling direct server-to-server communication while preserving centralized control through the virtualization layer.
2Device complexity
If a centralized I/O director is used for all communications, then management is simplified, but the system is not scalable for thousands of virtual networks
Solution Approach 1:
The patent implements a universal PVI layer that can accommodate thousands of virtual networks through VNICs. Each PVI provides multi-functional capabilities including direct server communication, isolation, and integration with the InfiniBand fabric, allowing the system to scale to thousands of virtual networks without proportionally increasing management complexity.
Solution Approach 2:
The patent enables self-service through autonomous discovery mechanisms where servers can independently discover and join PVIs without requiring centralized provisioning for each virtual network. This self-service capability allows the system to scale to thousands of virtual networks while maintaining manageable complexity through automated resource allocation and discovery.
3Speed
If direct server-to-server communication is enabled, then bandwidth and latency are improved, but communication isolation between virtual networks becomes difficult
Solution Approach 1:
The patent applies local quality by creating distinct PVIs with unique properties for each virtual network. Each PVI provides localized communication paths with specific isolation characteristics, allowing direct high-speed communication within each virtual network while maintaining strict isolation between different virtual networks through the PVI boundary.
Solution Approach 2:
The patent introduces PVIs and VNICs as intermediary layers that mediate between direct InfiniBand communication and virtual network isolation requirements. These intermediaries enable high-speed direct server-to-server communication while enforcing communication isolation policies, acting as mediators that preserve both speed and reliability.
4Adaptability or versatility
If Ethernet packets are encapsulated within InfiniBand messages, then Ethernet compatibility is maintained, but protocol overhead increases
Solution Approach 1:
The patent implements nested doll by encapsulating Ethernet packets within InfiniBand messages. This nesting allows Ethernet protocols to operate transparently over InfiniBand infrastructure, maintaining Ethernet compatibility while utilizing the high-performance InfiniBand transport layer. The encapsulation overhead is minimized through efficient PVI and VNIC implementations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Mechanisms are provided to allow servers connected over an InfiniBand fabric to communicate using multiple private virtual interconnects (PVIs). In particular embodiments, the PVIs appear as virtual Ethernet networks to users on individual servers and virtual machines running on the individual servers. Each PVI is represented on the server by a virtual network interface card (VNIC) and each PVI is mapped to its own InfiniBand multicast group. Data can be transmitted on PVIs as Ethernet packets fully encapsulated, including the layer 2 header, within InfiniBand messages. Broadcast and multicast frames are propagated using InfiniBand.