3D Logic-on-Logic Die Stacking for AI Thermal and Bandwidth Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI processor systems face challenges in reducing latency and power consumption, and managing thermal issues due to hardware-intensive training and inference processes, particularly in multi-dimensional packaging setups where the compute die is stacked below the memory die, leading to limited I/O bandwidth and thermal inefficiencies.
Innovation Solution
A logic-on-logic stacking configuration is implemented, where the compute die is positioned on top of a memory die or memory dies are placed on the sides of the compute die, allowing for direct heat sink attachment to the compute die and reducing thermal issues, while also achieving ultra-high bandwidth through tight micro-bump spacing and using ferroelectric or paraelectric logic to lower power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the compute die is stacked below the memory die in multi-dimensional packaging, then integration density is improved, but thermal management becomes difficult and I/O bandwidth is limited
Solution Approach 1:
The patent inverts the conventional stacking order by placing the compute die above the memory die instead of below it. This inversion allows the compute die to be positioned closer to the heat sink, improving thermal management while maintaining high integration density through the 3D stacked architecture.
Solution Approach 2:
The patent transitions from traditional 2D packaging to 3D multi-dimensional packaging, stacking multiple dies vertically. This dimensional change increases integration density by utilizing the vertical space, while the inverted compute-die-up configuration optimizes thermal pathways in this new dimensional arrangement.
2Quantity of substance
If the compute die is stacked below the memory die, then packaging integration is improved, but I/O bandwidth is reduced
Solution Approach 1:
By inverting the stack configuration to compute-die-up, the patent positions the compute die with direct access to the heat sink and optimizes the I/O pathway. This inversion enables higher I/O bandwidth while maintaining the benefits of 3D stacked packaging integration.
3Ease of manufacture
If traditional stacking configuration is used, then manufacturing is simplified, but thermal efficiency deteriorates
Solution Approach 1:
The patent applies the inversion principle to the stacking configuration, placing the compute die above the memory die. This maintains compatibility with existing manufacturing processes while dramatically improving thermal efficiency by positioning the heat-generating compute die closer to the heat sink.
Data Source
AI summary
A packaging technology to improve performance of an AI processing system resulting in an ultra-high bandwidth system. An IC package is provided which comprises: a substrate; a first die on the substrate, and a second die stacked over the first die. The first die can be a first logic die (e.g., a compute chip, CPU, GPU, etc.) while the second die can be a compute chiplet comprising ferroelectric or paraelectric logic. Both dies can include ferroelectric or paraelectric logic. The ferroelectric/paraelectric logic may include AND gates, OR gates, complex gates, majority, minority, and/or threshold gates, sequential logic, etc. The IC package can be in a 3D or 2.5D configuration that implements logic-on-logic stacking configuration. The 3D or 2.5D packaging configurations have chips or chiplets designed to have time distributed or spatially distributed processing. The logic of chips or chiplets is segregated so that one chip in a 3D or 2.5D stacking arrangement is hot at a time.