AI Accelerator Topology Using Broadcast Buffer for Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI core accelerator topologies, such as Crossbar fully interconnected and Mesh grid structures, face issues like reduced bandwidth, timing and wiring congestion, poor extensibility, and unbalanced synchronous running times, which hinder performance and power consumption efficiency.
Innovation Solution
A point-to-point topology where each AI core corresponds to a dedicated SRAM, utilizing a broadcast buffer and arbiter to manage data access and prioritize write operations, ensuring each core can directly access its memory without arbitration, and enabling broadcast of weight data for efficient reuse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If Crossbar fully interconnected structure or Mesh grid structure is used for multiple AI cores, then interconnection capability is provided, but bandwidth is reduced to 1/n when multiple AI cores access data in the same address range in parallel due to arbitration
Solution Approach 1:
The system segments the memory access resources by providing each AI core with a dedicated SRAM block. This segmentation eliminates the need for arbitration when multiple cores access data in parallel, as each core has its own dedicated memory resource, thereby maintaining full bandwidth for each core without the 1/n bandwidth reduction.
2Speed
If Crossbar structure or Mesh structure has large bus bandwidth (such as 1024 bit), then data transfer capability is improved, but back-end timing and wiring congestion problems occur
Solution Approach 1:
The system divides the large bus into multiple smaller dedicated point-to-point connections between each AI core and its corresponding SRAM. This segmentation maintains high data transfer speed for each connection while avoiding the wiring congestion and timing issues associated with a single large bus, as each connection is independently optimized.
3Adaptability or versatility
If Crossbar structure or Mesh structure is used for multiple AI cores, then interconnection is enabled, but extensibility of core quantity is poor due to performance and back-end implementation limitations
Solution Approach 1:
The system adopts a modular point-to-point topology where each AI core is independently connected to its own SRAM block. This modular segmentation allows for easy extension of the system by simply adding more core-SRAM pairs without increasing back-end implementation complexity, as each module operates independently with the same simple connection pattern.
4Stability of the object's composition
If multiple AI cores are synchronized in traditional topologies, then coordination is achieved, but synchronous running time is unbalanced which is not conducive to performance and power consumption
Solution Approach 1:
By providing each AI core with a dedicated SRAM block through point-to-point connections, the system segments the memory access paths to eliminate variability and arbitration delays. This ensures that each core experiences identical and balanced access times to its local memory, achieving balanced synchronous running time across all cores and optimizing both performance and power consumption.
Data Source
AI summary
A topology of accelerators is provided, including a plurality of accelerators and a broadcast buffer. Each of the plurality of accelerators corresponds to a first memory and obtain input data from an external second memory respectively, wherein the accelerator can only directly access its corresponding first memory, and the broadcast buffer is coupled between one of the plurality of accelerators and the corresponding first memory. When receiving a write command and the input data from the accelerator to which it is coupled, the broadcast buffer is configured to write the input data into the corresponding first memory according to the write command, and when broadcast is enabled, the broadcast buffer is configured to broadcast the write command and the weight data in the input data. This application can improve the access performance of the accelerators and reduce the access delay.