Cascaded Chiplet AI Accelerators With On-Chip Model Parameters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale artificial intelligence models are difficult to fit entirely on a processing chip due to size limitations, leading to substantial data communication bandwidth consumption and processing latency.
Innovation Solution
A processing unit is configured with a plurality of chiplets that can be cascaded and mapped to execute layers or blocks of artificial intelligence models, with parameters stored on-chip to reduce data movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If parameters for each layer or block of layers are brought onto the chip, then the artificial intelligence model can be executed, but substantial data communication bandwidth is consumed and processing latency increases
Solution Approach 1:
The patent divides the AI accelerator into multiple chiplets, each capable of independently executing layers or blocks of layers of an AI model. Each chiplet contains its own parameter storage, allowing parameters to be locally cached rather than continuously communicated across a monolithic chip. This segmentation reduces the communication bandwidth required while maintaining full model execution capability.
2Productivity
If parameters for each layer or block of layers are brought onto the chip, then the artificial intelligence model can be executed, but processing latency increases
Solution Approach 1:
The patent implements preliminary action by pre-loading and caching parameters for multiple layers or blocks of layers into the chiplet's local memory before execution begins. This allows the chiplet to process layers sequentially without repeated communication cycles, significantly reducing processing latency while maintaining the ability to execute complete AI models.
3Device complexity
If a monolithic integrated circuit chip is used, then data communication is simplified, but the chip size becomes too large to fit all AI model parameters
Solution Approach 1:
The patent transitions from a two-dimensional monolithic chip architecture to a three-dimensional stacked chiplet architecture. Multiple chiplets are vertically stacked and interconnected, allowing the system to accommodate large AI model parameters without requiring a single large chip area. This dimensional change enables scalable AI acceleration while maintaining manageable chip sizes.
Data Source
AI summary
A processing unit can include a plurality of chiplets coupled in a cascade topology by a plurality of interfaces. A set of the plurality of cascade coupled chiplets can be configured to execute a plurality of layers or blocks of layers of an artificial intelligence model. The set of cascade coupled chiplets can also be configured with parameter data of corresponding ones of the plurality of layers or blocks of layers of the artificial intelligence model.