AI Accelerator Chip With Integrated Cell-Based Fabric Adapter for RDMA
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing number of parameters in neural networks necessitates high-speed communication between AI chips, which conventional systems struggle to facilitate efficiently.
Innovation Solution
An AI accelerator integrated circuit chip with an integrated cell-based fabric adapter that enables remote direct memory access (RDMA) and datagram communication over a cell-based switch fabric, facilitating high-speed data transfer between AI chips through a fabric adapter and data interconnect.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional communication systems are used between AI chips, then device complexity is reduced, but communication speed and efficiency deteriorate
Solution Approach 1:
The fabric adapter merges multiple communication functions (RDMA read, RDMA write, datagram communication) and memory access operations into a single integrated component that directly interfaces with the cell-based switch fabric. This consolidation eliminates the need for separate communication controllers and memory interfaces, achieving high-speed communication while managing complexity through functional integration.
Solution Approach 2:
The fabric adapter acts as an intermediary component between the inference engine clusters and the cell-based switch fabric. It mediates data transfer by providing a specialized interface that translates internal memory access requests into fabric-compatible communication protocols, enabling efficient RDMA operations without requiring complex end-to-end communication logic throughout the system.
2Productivity
If high-speed communication is implemented between AI chips, then communication efficiency is improved, but latency requirements become more stringent
Solution Approach 1:
The fabric adapter implements preliminary action by pre-configuring virtual output queues (VOQs) and maintaining ready-to-transmit data buffers before actual communication needs arise. RDMA operations are prepared in advance with pre-allocated memory regions and pre-configured fabric paths, enabling immediate data transfer when communication is initiated, thus minimizing latency while maintaining high throughput.
3Adaptability or versatility
If neural network parameters are increased, then model complexity is improved, but communication overhead between chips increases
Solution Approach 1:
The fabric adapter segments large parameter datasets into manageable communication units that can be efficiently transferred across the cell-based switch fabric. By dividing parameter data into smaller chunks and managing their transfer through multiple virtual channels, the system can handle increased model complexity without being overwhelmed by communication overhead, as each segment can be processed and transmitted independently through the fabric adapter's segmented architecture.
Data Source
AI summary
An integrated circuit formed on (i) a single semiconductor die or (ii) a plurality semiconductor dies that are integrated into a single package. The integrated circuit may include a communication interface including a serializer/deserializer (SerDes) interface; a fabric adapter communicatively coupled to the communication interface; a plurality of inference engine clusters, each inference engine cluster including a respective memory element and/or memory interface; and a data interconnect communicatively coupling each respective memory element and/or memory interfaces of the plurality of inference engine clusters to the fabric adapter. The fabric adapter may be configured to facilitate remote direct memory access (RDMA) read and write services and/or datagram communication over a cell-based switch fabric to and from the respective memory elements and/or memory interfaces of the plurality of inference engine clusters via the data interconnect.