Logic-on-Logic AI Packaging for Heat and Bandwidth Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI processor systems face challenges in reducing latency and power consumption, and managing thermal issues due to hardware-intensive training and inference processes, particularly in multi-dimensional packaging setups where compute dies are stacked below memory dies, leading to limited I/O bandwidth and thermal inefficiencies.
Innovation Solution
A logic-on-logic stacking configuration is implemented, where a compute die is positioned on top of a memory die or memory dies are placed alongside the compute die, allowing for direct heat sink placement over the compute die and reducing thermal issues, while also achieving ultra-high bandwidth through tight micro-bump spacing and decoupling of TSV density from micro-bump density, utilizing ferroelectric or paraelectric logic for low power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If compute dies are stacked below memory dies in multi-dimensional packaging, then integration density is improved, but thermal management becomes more difficult and I/O bandwidth is limited
Solution Approach 1:
The patent inverts the conventional stacking order by placing memory dies below compute dies instead of above them. This inversion allows heat sinks to be positioned directly over the heat-generating compute dies, dramatically improving thermal management while maintaining high integration density through the stacked configuration.
Solution Approach 2:
The patent transitions from two-dimensional planar packaging to three-dimensional stacked packaging, enabling vertical integration of multiple compute and memory dies. This dimensional change increases integration density while the strategic placement of heat sinks in the vertical dimension addresses thermal management challenges.
2Quantity of substance
If compute dies are stacked below memory dies, then integration density is improved, but I/O bandwidth is limited
Solution Approach 1:
By inverting the stack order to place memory dies below compute dies, the patent enables more direct and numerous I/O connections between compute elements and memory elements. This configuration increases I/O bandwidth while preserving the high integration density benefits of 3D stacking.
3Productivity
If hardware-intensive training and inference processes are used, then AI processing capability is improved, but power consumption increases
Solution Approach 1:
The patent introduces ferroelectric or paraelectric materials as intermediaries in the logic circuits. These materials enable non-volatile storage and logic operations that consume significantly less power during inference operations, reducing the energy required for AI processing while maintaining high computational capability.
Solution Approach 2:
The patent changes the physical parameter of memory materials from conventional volatile memory to ferroelectric or paraelectric non-volatile memory. This parameter change enables the system to maintain AI processing capability with reduced power consumption, particularly during inference operations where weights need to be retained without continuous power.
4Use of energy by moving object
If ferroelectric or paraelectric logic is used, then power consumption is reduced, but device complexity increases
Solution Approach 1:
The patent employs ferroelectric or paraelectric materials that serve multiple functions: they provide non-volatile memory storage and enable logic operations simultaneously. This multi-functionality reduces the need for separate memory and logic components, thereby reducing overall device complexity despite the advanced materials used.
Solution Approach 2:
The patent utilizes composite material structures combining ferroelectric or paraelectric layers with conventional semiconductor materials. These composite structures integrate the low-power benefits of ferroelectric/paraelectric materials with the成熟 manufacturing processes of conventional semiconductors, managing device complexity through compatible material integration.
Data Source
AI summary
A packaging technology to improve performance of an AI processing system resulting in an ultra-high bandwidth system. An IC package is provided which comprises: a substrate; a first die on the substrate, and a second die stacked over the first die. The first die can be a first logic die (e.g., a compute chip, CPU, GPU, etc.) while the second die can be a compute chiplet comprising ferroelectric or paraelectric logic. Both dies can include ferroelectric or paraelectric logic. The ferroelectric/paraelectric logic may include AND gates, OR gates, complex gates, majority, minority, and/or threshold gates, sequential logic, etc. The IC package can be in a 3D or 2.5D configuration that implements logic-on-logic stacking configuration. The 3D or 2.5D packaging configurations have chips or chiplets designed to have time distributed or spatially distributed processing. The logic of chips or chiplets is segregated so that one chip in a 3D or 2.5D stacking arrangement is hot at a time.