Ferroelectric Memory Chiplet Layout for High-Bandwidth AI Thermal Relief
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI processing systems face limitations in I/O bandwidth and thermal management due to the stacking of dynamic random-access memory (DRAM) on top of compute dies, leading to periphery constraints and thermal issues, which are not adequately addressed by current packaging technologies.
Innovation Solution
The proposed solution involves an integrated circuit package design where the memory die is positioned below the compute die, or on its sides, with tight micro-bump spacing for ultra-high bandwidth and reduced thermal issues, decoupling the number of micro-bumps from through-silicon vias (TSVs) to minimize perforation requirements and enhance thermal management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If DRAM is stacked on top of compute die, then memory bandwidth is improved, but thermal management deteriorates and periphery constraints occur
Solution Approach 1:
The patent inverts the conventional stacking order by placing the memory die below the compute die instead of above it. This inversion allows the compute die to be positioned closer to the package substrate and heat sink, improving thermal management while maintaining high memory bandwidth through tight micro-bump spacing.
Solution Approach 2:
The patent transitions from a single vertical stacking dimension to a multi-dimensional arrangement where memory dies can be positioned below or on the sides of the compute die. This multi-dimensional configuration provides flexibility in thermal management and I/O routing while maintaining high bandwidth.
2Speed
If DRAM is stacked on top of compute die, then memory bandwidth is improved, but device complexity increases due to periphery constraints
Solution Approach 1:
By inverting the stack order with memory below compute die, the patent enables I/O interfaces to be routed through the package substrate without requiring complex periphery routing. This simplifies the device structure while maintaining high bandwidth through direct micro-bump connections.
Solution Approach 2:
The package substrate acts as an intermediary that facilitates I/O routing between the inverted stack configuration and external interfaces. This mediator approach simplifies the connection architecture by providing a centralized routing layer that avoids complex periphery constraints.
3Speed
If micro-bump spacing is reduced for ultra-high bandwidth, then bandwidth is improved, but manufacturing precision requirements increase
Solution Approach 1:
The patent combines multiple micro-bumps into dense arrays with standardized spacing patterns that leverage existing semiconductor manufacturing capabilities. By using regular, repeating patterns rather than irregular tight spacing, the design achieves ultra-high bandwidth while remaining compatible with standard fabrication processes.
4Speed
If through-silicon vias are used for I/O routing, then I/O bandwidth is improved, but device complexity increases due to perforation requirements
Solution Approach 1:
The patent extracts the I/O routing function from the compute die itself and relocates it to the package substrate level. This separation eliminates the need for through-silicon vias in the compute die, reducing perforation requirements and simplifying the compute die structure while maintaining high I/O bandwidth through the substrate.
Data Source
AI summary
A ferroelectric memory chiplet in a multi-dimensional packaging. The multi-dimensional packaging includes a first die comprising a switch and a first plurality of input-output transceivers. The multi-dimensional packaging includes a second die comprising a processor, wherein the second die includes a second plurality of input-output transceivers coupled to the first plurality of input-output transceivers. The multi-dimensional packaging includes a third die comprising a coherent cache or memory-side buffer, wherein the coherent cache or memory-side buffer comprises ferroelectric memory cells, wherein the coherent cache or memory-side buffer is coupled to the second die via I/Os. The dies are wafer-to-wafer bonded or coupled via micro-bumps, copper-to-copper hybrid bond, or wire bond, Flip-chip ball grid array routing, chip-on-wafer substrate, or embedded multi-die interconnect bridge.