Integrated Cell-Based Fabric Adapter for AI Accelerator RDMA
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI chips face challenges in high-speed communication due to the increasing number of parameters in neural networks, requiring efficient data transfer between multiple AI chips.
Innovation Solution
An integrated circuit with a fabric adapter configured for remote direct memory access (RDMA) and datagram communication over a cell-based switch fabric, facilitating high-speed data transfer between AI chips through a fabric adapter and data interconnect.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If AI chips are used to handle increasing neural network parameters, then computational capability is improved, but communication speed between multiple AI chips becomes insufficient
Solution Approach 1:
A fabric adapter is introduced as an intermediary component between the inference engine cluster and the cell-based switch fabric. The fabric adapter includes a virtual output queue (VOQ) subsystem that manages data packets, performs cellification, and handles routing to multiple AI chips, thereby enabling high-speed communication without bottlenecking the computational capability of the AI chips themselves.
Solution Approach 2:
The communication interface is segmented into multiple virtual output queues (VOQs) that can independently manage data flows to different destinations. This segmentation allows parallel processing of multiple data streams simultaneously, increasing the overall communication throughput between AI chips while maintaining the computational power of each chip.
2Productivity
If data is transferred between multiple AI chips, then neural network processing capability is improved, but communication latency increases
Solution Approach 1:
The fabric adapter performs preliminary actions by pre-processing data into fixed-size cells and maintaining virtual output queues ready for transmission. This preliminary organization of data allows for immediate transmission when needed, reducing the latency that would otherwise occur during data preparation and routing operations between AI chips.
Solution Approach 2:
The VOQ subsystem maintains continuous readiness for data transmission by keeping multiple virtual queues populated and organized. This continuity ensures that data transfer operations can proceed without interruption or waiting periods, minimizing communication latency while supporting enhanced neural network processing across multiple AI chips.
3Speed
If high-speed communication is implemented between AI chips, then data transfer efficiency is improved, but system complexity increases
Solution Approach 1:
The fabric adapter is designed as a universal interface that handles multiple functions including packetization, cellification, routing, and queue management within a single component. This multi-functionality reduces the need for separate dedicated components for each function, thereby achieving high-speed communication without proportionally increasing system complexity.
Data Source
AI summary
An integrated circuit formed on (i) a single semiconductor die or (ii) a plurality semiconductor dies that are integrated into a single package. The integrated circuit may include a communication interface including a serializer/deserializer (SerDes) interface; a fabric adapter communicatively coupled to the communication interface; a plurality of inference engine clusters, each inference engine cluster including a respective memory element and/or memory interface; and a data interconnect communicatively coupling each respective memory element and/or memory interfaces of the plurality of inference engine clusters to the fabric adapter. The fabric adapter may be configured to facilitate remote direct memory access (RDMA) read and write services and/or datagram communication over a cell-based switch fabric to and from the respective memory elements and/or memory interfaces of the plurality of inference engine clusters via the data interconnect.