Logic-on-Logic Die Stacking for AI Bandwidth and Thermal Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI processor systems face challenges in reducing latency and power consumption, and managing thermal issues due to hardware-intensive training and inference processes, particularly in multi-dimensional packaging setups where the compute die is stacked below the memory die, leading to limited I/O bandwidth and thermal management difficulties.
Innovation Solution
A logic-on-logic stacking configuration is implemented, where the compute die is positioned on top of a memory die, with ferroelectric or paraelectric logic chiplets, allowing for ultra-high bandwidth and efficient thermal management by decoupling TSV density from micro-bump spacing and placing the heat sink directly over the compute die.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the compute die is stacked below the memory die in multi-dimensional packaging, then the I/O bandwidth is limited and thermal management becomes difficult, but this configuration was traditionally used to save space
Solution Approach 1:
The patent inverts the traditional stacking configuration by placing the compute die on top of the memory die instead of below it. This inversion allows the compute die to be directly over the memory die, enabling shorter interconnect paths and improved bandwidth while also allowing the heat sink to be positioned directly over the compute die for effective thermal management.
Solution Approach 2:
The patent transitions from a two-dimensional packaging approach to a three-dimensional stacked configuration with the compute die positioned vertically above the memory die. This dimensional change enables simultaneous optimization of bandwidth (through vertical stacking) and thermal management (through direct heat sink placement over the compute die).
2Productivity
If TSV density is increased to improve bandwidth between stacked dies, then manufacturing complexity and cost increase, but lower TSV density limits performance
Solution Approach 1:
The patent performs preliminary routing of interconnect signals through the substrate before die stacking, allowing signals to be pre-positioned to minimize the number of TSVs required. This preliminary action reduces the complexity of TSV formation while maintaining high bandwidth between the compute die and memory die.
3Area of stationary object
If the compute die is placed below the memory die to save space, then thermal management becomes challenging, but placing it above increases the footprint
Solution Approach 1:
The patent inverts the traditional stacking order, placing the compute die on top of the memory die rather than below it. This inversion enables the heat sink to be positioned directly over the compute die, achieving effective thermal management while maintaining a compact footprint through vertical stacking.
Data Source
AI summary
A packaging technology to improve performance of an AI processing system resulting in an ultra-high bandwidth system. An IC package is provided which comprises: a substrate; a first die on the substrate, and a second die stacked over the first die. The first die can be a first logic die (e.g., a compute chip, CPU, GPU, etc.) while the second die can be a compute chiplet comprising ferroelectric or paraelectric logic. Both dies can include ferroelectric or paraelectric logic. The ferroelectric/paraelectric logic may include AND gates, OR gates, complex gates, majority, minority, and/or threshold gates, sequential logic, etc. The IC package can be in a 3D or 2.5D configuration that implements logic-on-logic stacking configuration. The 3D or 2.5D packaging configurations have chips or chiplets designed to have time distributed or spatially distributed processing. The logic of chips or chiplets is segregated so that one chip in a 3D or 2.5D stacking arrangement is hot at a time.