Global Virtual Planes for GPU Network Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud infrastructure faces challenges in achieving high throughput for graphical processing unit (GPU) workloads due to lack of flow entropy in equal cost multi-path (ECMP) routing, leading to bandwidth contention and congestion issues.
Innovation Solution
The implementation of global virtual planes in a network infrastructure, where specific subsets of ports and switches are allocated to virtual planes, allowing for exclusive resource usage and eliminating the need for ECMP traffic load balancing decisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If ECMP routing is used for network traffic distribution, then network resource utilization is improved, but flow hash collisions cause bandwidth contention and reduce throughput
Solution Approach 1:
The network fabric is segmented into multiple virtual planes, where each virtual plane is assigned a dedicated subset of ports and switches. This segmentation isolates traffic flows within specific virtual planes, eliminating ECMP hash collisions between different flows while maintaining efficient resource utilization within each plane.
Solution Approach 2:
The patent introduces a new dimension of network organization by adding virtual plane abstraction layer over the physical network fabric. This dimensional change allows traffic to be routed through dedicated virtual plane paths rather than relying on ECMP hashing across shared physical ports, thereby avoiding flow collisions.
2Adaptability or versatility
If multiple flows are hashed to traverse the same outgoing link in ECMP, then network flexibility is improved, but bandwidth contention increases and performance decreases
Solution Approach 1:
The network infrastructure is divided into multiple virtual planes with dedicated port subsets. Each virtual plane provides independent routing paths, maintaining routing flexibility while preventing multiple flows from being hashed to the same physical link, thus eliminating bandwidth contention.
3Ease of operation
If host machines exchange traffic without considering local network neighborhood, then network simplicity is maintained, but throughput is reduced due to ignoring locality information
Solution Approach 1:
Virtual planes are pre-configured with specific port subsets and locality information before workload execution. This preliminary organization of network resources by locality allows host machines to automatically benefit from optimized paths without requiring complex runtime decisions or configuration changes.
Data Source
AI summary
A network environment comprises a plurality of host machines that are communicatively coupled to each other via a network fabric comprising a plurality of switches that in turn include a plurality of ports. Each host machine comprises one or more GPUs. A first subset of ports from is associated with a first virtual plane, wherein the first virtual plane identifies a first collection of resources to be used for communicating packets from and to host machines associated with the first virtual plane. A second subset of ports is associated with a second virtual plane that is different from the first virtual plane. A first host machine and a second host machine are associated with the first virtual plane. A packet originating at the first host machine and destined for the second host machine is communicated using only ports from the first subset of ports.