3D Logic-on-Logic Packaging for AI Heat and Bandwidth Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI processor systems face challenges in reducing latency and power consumption, and managing thermal issues due to hardware-intensive training and inference processes, particularly in multi-dimensional packaging setups where the compute die is stacked below the memory die, leading to limited I/O bandwidth and thermal inefficiencies.
Innovation Solution
A logic-on-logic stacking configuration is implemented, where the compute die is positioned on top of a memory die or memory dies are placed alongside the compute die, allowing for direct heat sink placement over the compute die and reducing thermal issues, while also achieving ultra-high bandwidth through tight micro-bump spacing and using ferroelectric or paraelectric logic to lower power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the compute die is stacked below the memory die in multi-dimensional packaging, then integration density is improved, but thermal management becomes difficult and I/O bandwidth is limited
Solution Approach 1:
The patent inverts the conventional stacking order by placing the compute die above the memory die, allowing the heat sink to be positioned adjacent to the compute die for effective thermal management, while still achieving high integration density through 3D stacking
Solution Approach 2:
The patent transitions from 2D planar integration to 3D vertical stacking, enabling multiple memory dies to be stacked above the substrate while the compute die is positioned above them, achieving ultra-high integration density without compromising thermal management through strategic heat sink placement
2Productivity
If compute-intensive AI processing is performed, then processing capability is improved, but power consumption and heat generation increase
Solution Approach 1:
The patent introduces a specialized AI accelerator die positioned between the compute die and memory dies, serving as an intermediary that performs AI-specific computations more efficiently, reducing the power consumption and heat generation of the main compute die while maintaining high processing capability
3Speed
If the compute die is positioned away from the heat sink, then I/O bandwidth is improved, but thermal issues worsen
Solution Approach 1:
The patent segments the system into functionally optimized layers: memory dies stacked above the substrate for high-speed I/O access, compute die positioned above the memory stack with direct heat sink access for thermal management, and AI accelerator die for specialized processing, allowing each component to operate at optimal performance without compromising the others
Data Source
AI summary
A packaging technology to improve performance of an AI processing system resulting in an ultra-high bandwidth system. An IC package is provided which comprises: a substrate; a first die on the substrate, and a second die stacked over the first die. The first die can be a first logic die (e.g., a compute chip, CPU, GPU, etc.) while the second die can be a compute chiplet comprising ferroelectric or paraelectric logic. Both dies can include ferroelectric or paraelectric logic. The ferroelectric/paraelectric logic may include AND gates, OR gates, complex gates, majority, minority, and/or threshold gates, sequential logic, etc. The IC package can be in a 3D or 2.5D configuration that implements logic-on-logic stacking configuration. The 3D or 2.5D packaging configurations have chips or chiplets designed to have time distributed or spatially distributed processing. The logic of chips or chiplets is segregated so that one chip in a 3D or 2.5D stacking arrangement is hot at a time.