GPU Supercluster Fabric With Hierarchical Switching for Scale-Out
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional GPU clusters face limitations in scaling beyond 1K to 4K GPUs due to network topology constraints, oversubscription, and lack of support for diverse transmission speeds and custom routing protocols, leading to suboptimal workload throughput.
Innovation Solution
A supercluster architecture that enables hybrid clusters of GPUs with different generations and speeds to coexist in a network fabric, utilizing a hierarchical switch structure and dynamic workload allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional GPU cluster network topology is used, then GPU clusters can be supported, but scaling beyond 1K to 4K GPUs is limited due to significant oversubscription
Solution Approach 1:
The patent segments the network into fabric nodes, each containing multiple switches connected in a hierarchical structure. This segmentation allows the system to scale beyond traditional 1K-4K GPU limits by distributing the network load across multiple fabric nodes, reducing oversubscription at any single level while supporting larger-scale clusters.
Solution Approach 2:
The patent introduces a hierarchical switch structure with multiple layers (e.g., top-of-rack switches, aggregation switches) that adds a vertical dimension to the network topology. This multi-level hierarchy enables more efficient routing and reduces oversubscription by distributing traffic across multiple hierarchical levels, facilitating scaling beyond traditional planar topologies.
2Adaptability or versatility
If traditional GPU cluster network topology is used, then GPU clusters can be supported, but strict limitations are imposed on routing policy and transmission speed
Solution Approach 1:
The patent implements a universal network fabric that can support multiple transmission speeds (e.g., 25 Gbps, 100 Gbps) and various routing policies simultaneously. The hierarchical switch structure and fabric node design enable the network to adapt to different GPU generations and speed requirements without requiring separate dedicated networks, allowing one infrastructure to serve multiple purposes.
Solution Approach 2:
The patent enables dynamic adjustment of transmission speed parameters and routing policies based on workload requirements. By changing network parameters such as bandwidth allocation and routing paths, the system can optimize performance for different GPU generations and workload types without reconfiguring the entire network architecture.
3Productivity
If host machines exchange traffic without regard for local network neighborhood, then network simplicity is maintained, but throughput is reduced due to lack of flow entropy
Solution Approach 1:
The patent implements flow entropy mechanisms that provide feedback about traffic patterns and network conditions. By monitoring and analyzing traffic flows, the system can dynamically adjust routing decisions and load distribution to optimize throughput. This feedback mechanism enables the network to adapt to changing conditions and improve productivity without requiring complex manual configuration.
Data Source
AI summary
Described herein is a network fabric including a plurality of graphical processing unit (GPU) clusters. The plurality of GPU clusters includes at least a first GPU cluster operating at a first speed and a second GPU cluster operating at a second speed that is different than the first speed. The network fabric includes a plurality of blocks, wherein each block includes: (a) one or more racks that host a GPU cluster, and (b) a plurality of switches arranged in a hierarchical structure that communicatively couple the block to other blocks included in the network fabric. Responsive to receiving a request to execute a workload, allocating one or more GPUs from the plurality of GPU clusters to execute the workload.