Supercluster GPU Network Fabric for Mixed-Speed Cluster Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional GPU clusters face limitations in scaling beyond 1K to 4K GPUs due to network topology constraints, oversubscription, and lack of support for diverse transmission speeds and custom routing protocols, leading to suboptimal throughput in cloud environments.
Innovation Solution
A supercluster architecture that enables hybrid clusters of GPUs with different generations and speeds to coexist within a network fabric, utilizing a hierarchical switch structure and dynamic workload allocation across multiple GPU clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional network topology is used to support GPU clusters, then cluster scalability is limited to 1K-4K GPUs, but network performance and throughput deteriorate due to oversubscription
Solution Approach 1:
The network fabric is segmented into multiple blocks, each containing racks that host GPU clusters. This segmentation allows independent scaling of each block without affecting the entire network, enabling expansion beyond traditional 1K-4K GPU limits while maintaining network performance through distributed architecture
Solution Approach 2:
The patent transitions from traditional flat network topology to a hierarchical block-based structure with multiple dimensions. Blocks are organized in a scalable manner with switches arranged in hierarchical structures, adding spatial organization that enables linear scaling while avoiding oversubscription bottlenecks
2Adaptability or versatility
If hybrid clusters with different GPU speeds are implemented, then system versatility improves, but network complexity increases due to multiple transmission speeds
Solution Approach 1:
The network fabric is designed with universal interfaces and protocols that can handle multiple GPU transmission speeds within the same infrastructure. The block-based architecture with hierarchical switches provides a unified framework that accommodates different GPU generations and speeds without requiring separate network infrastructures
Solution Approach 2:
The system dynamically adjusts network parameters such as transmission speed and routing policies based on the specific GPU cluster being accessed. This allows hybrid clusters with different GPU speeds to coexist efficiently, with the network automatically optimizing performance for each GPU type rather than requiring manual configuration
3Productivity
If custom routing protocols are supported, then routing efficiency improves, but system complexity increases
Solution Approach 1:
The patent introduces a control plane as an intermediary between the data plane and routing logic. This control plane manages routing decisions and protocol implementations centrally, allowing custom routing protocols to be supported without increasing the complexity of individual network devices. The control plane handles the sophisticated routing logic while the data plane maintains simple, efficient forwarding
Data Source
AI summary
Described herein is a network fabric including a plurality of graphical processing unit (GPU) clusters. The plurality of GPU clusters includes at least a first GPU cluster operating at a first speed and a second GPU cluster operating at a second speed that is different than the first speed. The network fabric includes a plurality of blocks, wherein each block includes: (a) one or more racks that host a GPU cluster, and (b) a plurality of switches arranged in a hierarchical structure that communicatively couple the block to other blocks included in the network fabric. Responsive to receiving a request to execute a workload, allocating one or more GPUs from the plurality of GPU clusters to execute the workload.