3D Stacked FeRAM and Compute Dies for Low-Latency AI Packaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI processor systems face challenges in reducing latency and power consumption during the training and inference processes, which are hardware-intensive activities.
Innovation Solution
The implementation of a packaging technology that stacks a compute die on top of a memory die, utilizing ferroelectric random access memory (FeRAM) for improved performance. This configuration enhances data locality, reduces memory access latency, and lowers power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If compute die and memory die are separated and connected through interconnects, then device complexity and interconnect energy are reduced, but memory access latency increases and data locality deteriorates
Solution Approach 1:
The patent merges the compute die and memory die into a single integrated 3D stacked structure, placing memory directly underneath compute elements. This integration eliminates the need for complex external interconnects while simultaneously reducing memory access latency through direct vertical coupling, thus resolving the contradiction between device complexity and memory access speed.
Solution Approach 2:
The patent transitions from a planar 2D arrangement to a vertical 3D stacked architecture. By stacking memory die directly beneath compute die and utilizing vertical through-silicon vias (TSVs) for interconnection, the design achieves short interconnect paths in the vertical dimension, reducing both interconnect complexity and access latency compared to traditional horizontal interconnect arrangements.
2Speed
If compute die and memory die are integrated in 3D stacked configuration, then data locality and memory access speed are improved, but device complexity and manufacturing difficulty increase
Solution Approach 1:
The patent segments the integrated system into distinct functional layers: memory die at the bottom, compute die in the middle, and I/O die at the top. Each layer is independently designed and manufactured using standard CMOS processes, then vertically integrated through TSVs. This segmentation allows complex 3D functionality to be achieved while maintaining manufacturing simplicity through modular assembly.
Solution Approach 2:
The patent employs universal TSV structures that serve multiple functions: electrical interconnection between layers, mechanical alignment features, and thermal management pathways. The same vertical stacking architecture supports different die configurations and I/O arrangements, reducing overall device complexity despite the 3D integration.
3Ease of manufacture
If traditional 2D processor architecture is used, then manufacturing is simpler, but power consumption is high and latency is increased
Solution Approach 1:
The patent moves from 2D planar architecture to 3D vertical stacking, placing memory directly beneath compute elements. This vertical arrangement dramatically shortens the distance data must travel between memory and compute units, reducing dynamic power consumption. The same approach enables better thermal management by directing heat vertically, further improving energy efficiency.
4Quantity of substance
If memory capacity is increased in traditional architecture, then data storage is improved, but memory access latency and power consumption increase
Solution Approach 1:
The patent achieves high memory capacity by stacking multiple memory die vertically beneath the compute die, utilizing the third dimension for expansion. This vertical scaling provides abundant on-chip memory capacity while maintaining short access paths through direct vertical coupling, avoiding the latency penalties associated with expanding memory capacity in traditional 2D architectures.
Data Source
AI summary
Described is a packaging technology to improve performance of an AI processing system. An IC package is provided which comprises: a substrate; a first die on the substrate, and a second die stacked over the first die. The first die includes memory and the second die includes computational logic. The first die comprises a ferroelectric RAM (FeRAM) having bit-cells. Each bit-cell comprises an access transistor and a capacitor including ferroelectric material. The access transistor is coupled to the ferroelectric material. The FeRAM can be FeDRAM or FeSRAM. The memory of the first die may store input data and weight factors. The computational logic of the second die is coupled to the memory of the first die. The second die is an inference die that applies fixed weights for a trained model to an input data to generate an output. In one example, the second die is a training die that enables learning of the weights.