Multi-Branch Inter-Chip Network Routing for AI Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing interconnected chip networks face challenges in rapid and reliable information communication due to bandwidth limitations of traditional PCIe buses, which are insufficient for applications like neural networks and AI workloads that require high-speed memory access and data transfer.
Innovation Solution
The implementation of a high-bandwidth inter-chip network (ICN) that allows parallel processing units (PPUs) to communicate directly or indirectly through multiple paths without relying on PCIe, using static pre-determined routing tables and interconnects to facilitate communication between PPUs on the same or different compute nodes, enabling direct memory access and instruction-level communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional PCIe bus is used for communication between processing chips, then system compatibility and ease of implementation are maintained, but communication bandwidth and data transfer speed are insufficient
Solution Approach 1:
The system segments communication paths into multiple independent routes between processing chips. Each chip is assigned multiple network interfaces, and routing tables divide traffic into different branches, creating parallel communication channels that increase overall bandwidth and reduce transfer time.
Solution Approach 2:
The patent transitions from a single-dimension PCIe bus architecture to a multi-dimensional network topology. By introducing multiple network interfaces and establishing communication paths through different dimensions (multiple routes between same endpoints), the system achieves higher bandwidth without proportionally increasing complexity.
2Reliability
If multiple network interfaces and multi-path routing are implemented, then communication reliability and bandwidth are improved, but routing table complexity and configuration difficulty increase
Solution Approach 1:
The system pre-configures routing tables with multiple predetermined paths between processing chips before communication begins. These static routing tables are established in advance, eliminating the need for dynamic route discovery and reducing configuration complexity while ensuring reliable communication paths are available.
Solution Approach 2:
The patent introduces an intermediary routing table mechanism that mediates between multiple network interfaces and destination chips. The routing table acts as a lookup intermediary that simplifies the complexity of multi-path routing by providing deterministic, pre-calculated routes, making the system easier to operate while maintaining high reliability.
3Productivity
If PCIe bus is used for memory access requests, then implementation simplicity is maintained, but access speed and bandwidth are limited
Solution Approach 1:
The system creates a universal interconnect network that handles multiple functions: memory access requests, data transfer between chips, and communication coordination. This multi-functional network infrastructure replaces the specialized PCIe bus, providing higher bandwidth for memory access while consolidating multiple communication needs into a single flexible system.
Data Source
AI summary
A system includes a high-bandwidth inter-chip network (ICN) that allows communication between parallel processing units (PPUs) in the system. For example, the ICN allows a PPU to communicate with other PPUs on the same compute node or server and also with PPUs on other compute nodes or servers. In embodiments, communication may be at the command level (e.g., at the direct memory access level) and at the instruction level (e.g., the finer-grained load/store instruction level). The ICN allows PPUs in the system to communicate without using a PCIe bus, thereby avoiding its bandwidth limitations and relative lack of speed. The respective routing tables comprise information of multiple paths to any given other PPU.