Switching Fabric Topology for Full GPU Interconnection Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI large model all-in-one machines face low interconnection rates and communication bandwidth due to GPUs being grouped into separate interconnection groups, preventing full interconnection and optimal communication performance.
Innovation Solution
An interconnection apparatus with flexible topology architecture, utilizing switching devices equipped with embedded controllers to enable full interconnection among all devices, enhancing interconnection rates and communication bandwidth through selective interface connections and global identifier management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If GPUs are divided into multiple groups with limited interconnection, then device complexity is reduced and easier to manage, but interconnection rate and communication bandwidth deteriorate
Solution Approach 1:
Switching devices are introduced as intermediaries between GPU groups to enable indirect communication paths. The switching devices with embedded controllers manage interconnection interfaces selectively, allowing GPUs in different groups to communicate through the switching fabric without direct peer-to-peer connections, thus maintaining manageability while achieving full interconnection capability.
Solution Approach 2:
The interconnection architecture transitions from a flat, direct connection model to a hierarchical, multi-dimensional topology. GPUs are organized in groups that connect to switching devices, which in turn connect to other GPU groups. This dimensional restructuring enables full interconnection bandwidth while keeping individual device complexity manageable through the hierarchical organization.
2Adaptability or versatility
If selective interface connections are implemented, then adaptability of interconnection topology is improved, but device complexity increases due to additional control mechanisms
Solution Approach 1:
The embedded controller merges multiple control functions into a single integrated component within the switching device. It handles topology configuration information, enumerates external devices, establishes routing tables, and manages interface parameters all through one unified control mechanism, reducing the perceived complexity while maintaining high adaptability.
Solution Approach 2:
The embedded controller performs preliminary actions by pre-configuring routing tables and interface parameters based on topology configuration information before actual data transmission occurs. Device enumeration and global identifier assignment are done in advance, allowing the system to adapt to different topologies without real-time complex decision-making during data flow.
3Productivity
If full interconnection among all devices is achieved, then communication performance is improved, but device complexity and system scale increase
Solution Approach 1:
The system segments the interconnection fabric into multiple switching devices, each managing a subset of interconnection interfaces and external devices. This segmentation allows full interconnection capability to be distributed across multiple manageable units, where each switching device handles a portion of the overall complexity while contributing to the collective full interconnection performance.
Solution Approach 2:
Each switching device with an embedded controller is designed as a universal node that can connect to multiple types of external devices and configure various interconnection topologies. The embedded controller provides multi-functional capabilities including device enumeration, routing table establishment, interface parameter adjustment, and topology adaptation, allowing a single device design to handle diverse configuration scenarios without increasing overall system complexity.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
This application discloses an interconnection apparatus, a high-performance switching apparatus, and a large model all-in-one machine, and relates to the communication field, to solve the problems of low interconnection speed and low communication bandwidth among a plurality of cards within a large model all-in-one machine. The interconnection apparatus includes a plurality of switching devices. Each of the plurality of switching devices includes a plurality of external interfaces and a plurality of interconnection interfaces. The interconnection interfaces of a first switching device are selectively connected to the interconnection interfaces of a second switching device. The first switching device further includes an embedded controller, configured to receive topology configuration information, and when determining based on the topology configuration information that a target interfaces exists among the plurality of interconnection interfaces of the first switching device in which the embedded controller is located, control the one or more target interfaces to establish connections with corresponding interconnection interfaces of the second switching device in a one-to-one correspondence, to interconnect any two of of external devices connected to the plurality of switching devices. This application can flexibly adjust an interconnection topology architecture to achieve full interconnection among all devices on the interconnection topology architecture and enhance interconnection rate and communication bandwidth.