3D Split GPU Logic Die Stacking for Bandwidth and Thermal Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI processor systems face challenges in reducing latency and power consumption during model training and inference, while also managing thermal issues due to heat generation by processors in multi-dimensional packaging setups.
Innovation Solution
The proposed solution involves a packaging technology that includes a substrate with a first die and a second die stacked over the first die, where the second die is a compute chiplet with ferroelectric or paraelectric logic, and the first die can be a high bandwidth memory (HBM) or a compute die. This configuration allows for ultra-high bandwidth AI processing and improved thermal management by placing the memory die below or adjacent to the compute die, reducing thermal issues and increasing bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If compute die and memory die are stacked in multi-dimensional packaging, then bandwidth is improved, but thermal management becomes challenging
Solution Approach 1:
The patent segments the compute die into multiple functional units (FUBs) that can be independently controlled and powered. This segmentation allows selective activation of compute units, reducing overall power consumption and heat generation while maintaining high bandwidth through efficient data flow between stacked memory and compute units.
Solution Approach 2:
The patent implements dynamic power management where individual compute units can be activated or deactivated based on workload requirements. This dynamic control allows the system to optimize between performance and thermal characteristics, maintaining high bandwidth when needed while reducing heat generation during lower intensity operations.
2Productivity
If AI processor systems increase processing power for model training, then productivity is improved, but power consumption increases
Solution Approach 1:
The compute die is divided into multiple independently controllable functional units. This segmentation enables the system to activate only the necessary processing units for each task, achieving high productivity when needed while minimizing power consumption during lighter workloads or model training phases.
Solution Approach 2:
The patent employs dynamic adjustment of operational parameters including clock frequencies and power states of individual compute units. This allows the system to optimize processing power for AI model training while controlling overall power consumption through selective scaling of operational intensity.
3Speed
If die-to-die I/O density is increased, then bandwidth is improved, but TSV density requirements increase complexity
Solution Approach 1:
The patent implements different I/O interface densities at different locations on the die. Memory interface units are positioned with higher density near the memory stack to maximize bandwidth, while other regions use standard density interfaces. This local optimization achieves high bandwidth without requiring uniformly high TSV density throughout the entire die.
Data Source
AI summary
A packaging technology to improve performance of an AI processing system resulting in an ultra-high bandwidth system. An IC package is provided which comprises: a substrate; a first die on the substrate, and a second die stacked over the first die. The first die can be a first logic die (e.g., a compute chip, CPU, GPU, etc.) while the second die can be a compute chiplet comprising ferroelectric or paraelectric logic. Both dies can include ferroelectric or paraelectric logic. The ferroelectric/paraelectric logic may include AND gates, OR gates, complex gates, majority, minority, and/or threshold gates, sequential logic, etc. The IC package can be in a 3D or 2.5D configuration that implements logic-on-logic stacking configuration. The 3D or 2.5D packaging configurations have chips or chiplets designed to have time distributed or spatially distributed processing. The logic of chips or chiplets is segregated so that one chip in a 3D or 2.5D stacking arrangement is hot at a time.