Ferroelectric Logic Stacking for AI Thermal and I/O Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI processor systems face challenges in reducing latency and power consumption, and managing thermal issues due to hardware-intensive training and inference processes, particularly in multi-dimensional packaging setups where compute dies are stacked below memory dies, leading to limited I/O bandwidth and thermal inefficiencies.
Innovation Solution
A logic-on-logic stacking configuration is implemented, where a compute die is positioned on top of a memory die or memory dies are placed alongside the compute die, allowing for direct heat sink placement over the compute die and reducing thermal issues, while also achieving ultra-high bandwidth through tight micro-bump spacing and using ferroelectric or paraelectric logic to minimize power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If compute dies are stacked below memory dies in multi-dimensional packaging, then integration density is improved, but thermal management becomes difficult and I/O bandwidth is limited
Solution Approach 1:
The patent inverts the conventional stacking order by placing memory dies below compute dies instead of above them. This inversion allows heat sinks to be positioned adjacent to the compute dies for effective thermal management, while maintaining high integration density through the stacked configuration.
Solution Approach 2:
The patent transitions from two-dimensional planar integration to three-dimensional stacked integration, enabling multiple memory dies to be stacked below a single compute die. This dimensional change increases integration density while allowing thermal management solutions to be implemented in the horizontal plane adjacent to the compute die.
2Quantity of substance
If compute dies are stacked below memory dies, then integration density is improved, but I/O bandwidth is limited
Solution Approach 1:
By inverting the stack order with compute dies on top and memory dies below, the patent enables direct access pathways for I/O operations. The compute die can communicate with multiple memory dies through vertical interconnects, increasing bandwidth while maintaining compact integration.
3Quantity of substance
If conventional logic is used in stacked configuration, then integration is achieved, but power consumption increases
Solution Approach 1:
The patent changes the fundamental parameter of logic technology from conventional CMOS to ferroelectric or paraelectric logic. This parameter change enables non-volatile data storage in the logic elements themselves, reducing the need for continuous refresh operations and thereby lowering power consumption in the stacked integrated system.
Solution Approach 2:
The patent employs composite logic structures combining ferroelectric or paraelectric materials with conventional semiconductor materials. This composite approach enables the integration of non-volatile memory capabilities directly within the logic fabric, reducing power consumption while maintaining computational functionality in the stacked configuration.
Data Source
AI summary
A packaging technology to improve performance of an AI processing system resulting in an ultra-high bandwidth system. An IC package is provided which comprises: a substrate; a first die on the substrate, and a second die stacked over the first die. The first die can be a first logic die (e.g., a compute chip, CPU, GPU, etc.) while the second die can be a compute chiplet comprising ferroelectric or paraelectric logic. Both dies can include ferroelectric or paraelectric logic. The ferroelectric/paraelectric logic may include AND gates, OR gates, complex gates, majority, minority, and/or threshold gates, sequential logic, etc. The IC package can be in a 3D or 2.5D configuration that implements logic-on-logic stacking configuration. The 3D or 2.5D packaging configurations have chips or chiplets designed to have time distributed or spatially distributed processing. The logic of chips or chiplets is segregated so that one chip in a 3D or 2.5D stacking arrangement is hot at a time.