3D Stacked FeRAM and Compute Dies for Low-Latency AI Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI processor systems are hardware intensive and consume high power, leading to increased latency and energy consumption during the training and inference processes, necessitating a reduction in these aspects to enhance performance and efficiency.
Innovation Solution
The integration of a ferroelectric random access memory (FeRAM) as a memory die stacked below or adjacent to a compute die in an AI processing system, which includes computational logic for efficient matrix multiplication and weight storage, reducing power consumption and latency by utilizing FeRAM for storing input data and weight factors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by stationary object
If conventional memory and compute architectures are used in AI processor systems, then processing capability is achieved, but power consumption is high and latency is increased
Solution Approach 1:
The patent merges memory and compute functions into a unified 3D stacked architecture where FeRAM memory dies are directly coupled to compute dies through copper pillars. This integration eliminates separate memory access cycles, allowing compute units to directly access weight factors stored in FeRAM, thereby reducing power consumption while maintaining high processing efficiency.
Solution Approach 2:
The patent changes the memory technology parameter by using ferroelectric RAM (FeRAM) instead of conventional DRAM or SRAM. FeRAM provides non-volatile storage with lower power consumption and faster write speeds, directly addressing the power consumption issue while maintaining the processing capability required for AI workloads.
2Loss of time
If conventional memory and compute architectures are used in AI processor systems, then processing capability is achieved, but latency is increased
Solution Approach 1:
The 3D stacked architecture merges memory and compute into a single integrated unit, eliminating the physical distance and access latency associated with conventional separate memory and processor components. The direct copper pillar connections enable sub-nanosecond access times, dramatically reducing latency while maintaining high processing throughput.
Solution Approach 2:
The patent transitions from a 2D planar architecture to a 3D vertical stacking architecture. This dimensional change allows memory and compute units to be vertically integrated, reducing signal path lengths and access latency while maintaining high processing efficiency through multiple stacked layers.
3Device complexity
If memory die is stacked below compute die using conventional interconnects, then integration is achieved, but interconnect energy and circuit complexity increase
Solution Approach 1:
The patent changes the interconnect material parameter from conventional solder or copper traces to vertical copper pillars with electroplated construction. This parameter change reduces resistive losses and inductance, lowering interconnect energy consumption while simplifying the overall circuit design through standardized vertical connections.
Data Source
AI summary
Described is a packaging technology to improve performance of an AI processing system. An IC package is provided which comprises: a substrate; a first die on the substrate, and a second die stacked over the first die. The first die includes memory and the second die includes computational logic. The first die comprises a ferroelectric RAM (FeRAM) having bit-cells. Each bit-cell comprises an access transistor and a capacitor including ferroelectric material. The access transistor is coupled to the ferroelectric material. The FeRAM can be FeDRAM or FeSRAM. The memory of the first die may store input data and weight factors. The computational logic of the second die is coupled to the memory of the first die. The second die is an inference die that applies fixed weights for a trained model to an input data to generate an output. In one example, the second die is a training die that enables learning of the weights.