HBM Twin-Tower Package for Memory Wall and Thermal Density
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing core count in modern processing units leads to a 'memory wall' bottleneck due to insufficient memory latency and bandwidth, exacerbated by the distance between processor cores and high bandwidth memory (HBM), which requires new computing and memory architectures to meet the demands of high performance computing and artificial intelligence.
Innovation Solution
A semiconductor device with a base die and stacked semiconductor chips encapsulated by a dielectric material with low coefficient of thermal expansion (CTE), forming an HBM multi-stack package that integrates computing and memory dies closely, reducing latency and increasing bandwidth through advanced silicon node fabrication and dielectric encapsulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If core count of processing units is increased to meet high performance computing demand, then computing capability is improved, but memory wall bottleneck worsens due to insufficient memory bandwidth and increased distance between cores and memory
Solution Approach 1:
The patent transitions from traditional 2D memory arrangement to 3D stacked architecture, where memory dies are stacked vertically above the processing die. This vertical stacking in the third dimension dramatically increases memory capacity and bandwidth while reducing the physical distance between computing cores and memory, directly addressing the memory wall bottleneck caused by increased core counts.
2Quantity of substance
If distance between processor cores and HBM is reduced to improve memory bandwidth, then memory latency is reduced, but manufacturing complexity increases due to advanced packaging requirements
Solution Approach 1:
The patent divides the memory system into separate stacked modules, with processing dies and memory dies organized into distinct stacks. Each stack is independently manufactured and then interconnected through redistribution layers, allowing complex 3D integration to be achieved through modular assembly rather than monolithic fabrication, thereby managing manufacturing complexity while maintaining high bandwidth performance.
Solution Approach 2:
The patent introduces redistribution layers (RDLs) and interconnect structures as intermediary elements between the processing die and stacked memory dies. These intermediaries facilitate electrical connections across multiple layers and stacks, enabling reduced latency and increased bandwidth while providing a manageable interface for the complex packaging process.
3Loss of time
If HBM is integrated closer to processor cores to reduce latency, then memory access time is reduced, but thermal management becomes more challenging due to increased heat density
Solution Approach 1:
The patent implements different thermal management approaches for different regions of the stacked package. Heat dissipation structures, thermal vias, and cooling solutions are strategically positioned in high-density heat generation areas, while lower-density regions use simpler thermal management. This localized approach effectively manages heat density while maintaining close integration between cores and memory for reduced access time.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A semiconductor device (1) and a method of manufacturing the semiconductor device (1) are provided. The semiconductor device (1) may include: a first semiconductor chip (10); at least one semiconductor chip stack (20) on a surface of the first semiconductor chip (10) in a first direction of the semiconductor device (1), the at least one semiconductor chip stack (20) configured to be electrically connected to the first semiconductor chip (10); and a dielectric (30) on the first semiconductor chip (10) in the first direction, and surrounding the at least one semiconductor chip stack (20) in at least one second direction of the semiconductor device (1), perpendicular to the first direction, wherein the at least one semiconductor chip stack (20) includes at least one second semiconductor chip (22L-22U).