Multi-Accelerator Server Link Reconfiguration for AI Communication Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence (AI) servers face challenges in optimizing computing power efficiency and system performance due to communication bottlenecks and limited performance of single accelerator cards, especially when scaling up the number of accelerator cards.
Innovation Solution
A heterogeneous server system with multiple accelerator cards and a resource link reconfiguration method, utilizing a switching unit, management controller, and switching chips to dynamically allocate and reconfigure resource links based on task requirements, ensuring high bandwidth and low latency communication between processors and accelerator cards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of accelerator cards is increased to improve computing power, then system computing power and training speed are improved, but communication bottlenecks become more obvious and network communication efficiency deteriorates
Solution Approach 1:
The system segments the network communication function into multiple switching chips, where each switching chip independently manages a subset of accelerator cards. This segmentation allows parallel communication paths to be established, reducing the communication bottleneck that occurs when all cards compete for a single network switch's bandwidth.
Solution Approach 2:
The patent transitions from a single-dimension communication architecture (one switch connecting all cards) to a multi-dimensional architecture using multiple switching chips arranged in a network topology. This adds spatial dimensions to the communication paths, enabling simultaneous data flow through multiple independent channels and effectively increasing overall communication bandwidth.
2Productivity
If the number of accelerator cards is increased to improve training speed, then computing resources are expanded, but resource allocation efficiency deteriorates due to fixed link configurations
Solution Approach 1:
The system implements dynamic resource link reconfiguration where the management controller can programmatically change the connection topology between switching chips and accelerator cards based on task requirements. This dynamic adaptability allows the same hardware infrastructure to be optimized for different training workloads, improving resource allocation efficiency without requiring fixed dedicated links for each card.
Solution Approach 2:
Each switching chip is designed with multiple configurable ports that can be dynamically assigned to different accelerator cards based on task needs. This universal interface design allows any switching chip to serve multiple functions and connect to various cards, enabling flexible resource allocation and preventing waste when the number of active cards varies.
3Speed
If single card performance is optimized through precision training and operator fusion, then single card training speed is improved, but overall system throughput is limited by communication bandwidth
Solution Approach 1:
The patent merges multiple switching chips into a unified network fabric that provides aggregate bandwidth greater than any single switch. By combining the communication capacity of multiple chips and establishing parallel data flow paths, the system achieves overall throughput that scales with the number of accelerator cards, preventing communication bandwidth from becoming the limiting factor for multi-card training speed.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
Provided in the present application are a multi-accelerator-card heterogeneous server and a resource link reconstruction method. The server comprises a switch unit, a heterogeneous accelerated-computing pooling unit and a general-purpose computing resource pooling unit, wherein the general-purpose computing resource pooling unit comprises at least one processor, the heterogeneous accelerated-computing pooling unit comprises at least two accelerator cards, and the processor and the accelerator cards are heterogeneous relative to each other; the switch unit comprises a management controller and at least two switch chips, any switch chip is connected to at least one of the other switch chips in the switch unit by means of an internal port, and the processor and the accelerator cards are respectively connected to the switch chips; and the management controller is connected to the switch chips, and is used for reconstructing a resource link on the basis of preset task resource requirements and chip information of the current switch chips.