Ferroelectric Memory Chiplet Layout for AI Bandwidth and Thermal Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI processing systems face limitations in I/O bandwidth and thermal management due to the stacking of dynamic random-access memory (DRAM) on top of a compute die, leading to periphery constraints and thermal issues, which affect the performance and efficiency of artificial intelligence (AI) processing systems.
Innovation Solution
The proposed solution involves an integrated circuit package design where the memory die is positioned below the compute die or on its sides, utilizing high-bandwidth memory (HBM) and incorporating ferroelectric RAM (FeRAM) or other fast access memories, with a substrate acting as an active interposer to enhance communication and thermal management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If DRAM is stacked on top of compute die, then memory capacity is increased, but I/O bandwidth is limited due to periphery constraints
Solution Approach 1:
The patent transitions from 2D periphery-based I/O to 3D vertical stacking architecture. Memory dies are stacked vertically above the compute die, with I/O connections established through the stacking interface rather than peripheral edges. This dimensional change allows multiple I/O channels to be formed through the vertical stack, dramatically increasing bandwidth while maintaining compact footprint.
Solution Approach 2:
The memory system is segmented into multiple separate memory dies stacked vertically, each die providing independent I/O channels to the compute die. This segmentation allows parallel data transmission through multiple stacked layers, increasing overall bandwidth while each individual die remains manageable in size and complexity.
2Quantity of substance
If DRAM is stacked on top of compute die, then memory capacity is increased, but thermal issues arise affecting performance
Solution Approach 1:
The patent introduces an interposer substrate as an intermediary layer between the compute die and stacked memory dies. This interposer serves as a thermal management interface, providing pathways for heat dissipation and enabling separate thermal management strategies for compute and memory components while maintaining electrical connectivity.
Solution Approach 2:
Thermal management is segmented by separating compute and memory components into distinct stacked layers. This spatial segmentation allows independent thermal management approaches for each component type, with heat sinks and cooling solutions optimized for their specific thermal characteristics and operational requirements.
3Speed
If compute die is perforated with TSVs to couple with memory bumps, then I/O bandwidth is increased, but device complexity increases
Solution Approach 1:
The patent merges the TSV formation process with the memory die stacking process. TSVs are formed in the compute die to align with bump patterns on memory dies, and the stacking process simultaneously establishes both mechanical support and electrical connectivity. This merging reduces overall manufacturing steps and complexity compared to separate processes.
Solution Approach 2:
The memory die bump pattern serves a dual function: providing mechanical attachment points and defining the TSV location map for the compute die. The bump arrangement self-determines the TSV configuration, eliminating the need for separate alignment processes and reducing manufacturing complexity while enabling high-bandwidth connections.
Data Source
AI summary
A ferroelectric memory chiplet in a multi-dimensional packaging. The multi-dimensional packaging includes a first die comprising a switch and a first plurality of input-output transceivers. The multi-dimensional packaging includes a second die comprising a processor, wherein the second die includes a second plurality of input-output transceivers coupled to the first plurality of input-output transceivers. The multi-dimensional packaging includes a third die comprising a coherent cache or memory-side buffer, wherein the coherent cache or memory-side buffer comprises ferroelectric memory cells, wherein the coherent cache or memory-side buffer is coupled to the second die via I/Os. The dies are wafer-to-wafer bonded or coupled via micro-bumps, copper-to-copper hybrid bond, or wire bond, Flip-chip ball grid array routing, chip-on-wafer substrate, or embedded multi-die interconnect bridge.