3D Logic-on-Logic AI Packaging for Thermal and Bandwidth Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI processor systems face challenges in reducing latency and power consumption, and managing thermal issues due to hardware-intensive training and inference processes, particularly in multi-dimensional packaging setups where the compute die is stacked below the memory die, leading to limited I/O bandwidth and thermal management difficulties.
Innovation Solution
A logic-on-logic stacking configuration is implemented, where the compute die is positioned on top of a memory die, allowing for direct heat sink placement over the compute die and reducing thermal issues, while also achieving ultra-high bandwidth through tight micro-bump spacing and using ferroelectric or paraelectric logic to minimize power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the compute die is stacked below the memory die in multi-dimensional packaging, then integration density is improved, but thermal management becomes difficult and I/O bandwidth is limited
Solution Approach 1:
The patent inverts the conventional stacking order by placing the compute die above the memory die instead of below it. This inversion allows the heat sink to be positioned directly over the compute die (the primary heat-generating component), enabling effective thermal management while maintaining high integration density in 3D packaging.
2Productivity
If compute-heavy processing is performed to achieve high AI processing power, then productivity is improved, but heat generation increases making thermal management challenging
Solution Approach 1:
The patent transitions from 2D packaging to 3D stacking configuration, allowing the heat sink to be positioned in the vertical dimension directly over the compute die. This dimensional change enables efficient heat removal path while maintaining high processing power through close integration of compute and memory components.
3Ease of manufacture
If the compute die is positioned away from the heat sink in conventional packaging, then manufacturing is simplified, but thermal issues arise due to inadequate heat dissipation
Solution Approach 1:
The patent segments the packaging structure into distinct functional layers: memory die at the bottom, interconnect layer in the middle, and compute die at the top with direct heat sink attachment. This segmentation allows each component to be optimized independently while achieving effective thermal management through the dedicated heat dissipation path.
Data Source
AI summary
A packaging technology to improve performance of an AI processing system resulting in an ultra-high bandwidth system. An IC package is provided which comprises: a substrate; a first die on the substrate, and a second die stacked over the first die. The first die can be a first logic die (e.g., a compute chip, CPU, GPU, etc.) while the second die can be a compute chiplet comprising ferroelectric or paraelectric logic. Both dies can include ferroelectric or paraelectric logic. The ferroelectric/paraelectric logic may include AND gates, OR gates, complex gates, majority, minority, and/or threshold gates, sequential logic, etc. The IC package can be in a 3D or 2.5D configuration that implements logic-on-logic stacking configuration. The 3D or 2.5D packaging configurations have chips or chiplets designed to have time distributed or spatially distributed processing. The logic of chips or chiplets is segregated so that one chip in a 3D or 2.5D stacking arrangement is hot at a time.