Vertical Logic-PNM Package Layout for Low-Energy High Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current semiconductor devices face challenges in achieving high-capacity, high-bandwidth memory for large artificial intelligence workloads like ChatGPT, due to high energy per bit and increased form factor in conventional HBM solutions.
Innovation Solution
The proposed solution involves a package configuration with vertically aligned first and second chips, where the first chip is a logic die, the second chip is a processing near memory (PNM) die, and the package includes molds for encapsulation, allowing for direct vertical connections and reducing energy per bit.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional HBM solutions are used, then high-capacity high bandwidth memory is achieved, but energy per bit increases and form factor increases
Solution Approach 1:
The patent transitions from lateral routing in conventional HBM to vertical stacking architecture. Memory chips are stacked vertically above the compute chiplet using TSVs, changing the routing dimension from 2D lateral to 3D vertical. This dimensional change reduces the physical distance for data transfer, thereby reducing energy per bit while maintaining high bandwidth capability.
Solution Approach 2:
The patent implements a nested structure where memory chips are placed directly on top of the compute chiplet, forming a stacked configuration. The memory stack is nested within the vertical space above the compute chiplet, eliminating the need for separate lateral placement and reducing overall form factor while maintaining high capacity and bandwidth.
2Productivity
If conventional HBM solutions are used, then high-capacity high bandwidth memory is achieved, but form factor increases
Solution Approach 1:
The patent moves from 2D lateral placement to 3D vertical stacking. By utilizing the vertical dimension for memory placement above the compute chiplet, the solution reduces the lateral footprint (form factor) while maintaining high bandwidth through vertical TSV routing.
Solution Approach 2:
The memory chips are nested vertically above the compute chiplet in a stacked configuration. This nesting approach consolidates the memory and compute components into a compact vertical structure, significantly reducing the lateral form factor compared to side-by-side placement.
3Reliability
If lateral routing on interposer is used, then memory connectivity is achieved, but energy per bit increases
Solution Approach 1:
The patent replaces lateral routing through an interposer with vertical routing using TSVs. Data transfers occur vertically through the stacked layers rather than laterally across the interposer, shortening the transmission path and reducing energy consumption per bit while maintaining reliable memory connectivity.
4Reliability
If side-by-side placement with compute chiplet is used, then memory connectivity is achieved, but form factor increases
Solution Approach 1:
The patent transitions from side-by-side lateral placement to vertical stacking. Memory chips are positioned in the vertical dimension above the compute chiplet rather than in the lateral plane, reducing the overall form factor while maintaining connectivity through vertical TSV routing.
Data Source
AI summary
Disclosed are packages that may include first and second substrates with first and second chips therebetween. The first chip may be a logic chip and the second chip may be a processing near memory (PNM) chip. The active side of the first chip may face the first substrate and the active side of the second chip may face the second substrate. The first chip may be encapsulated by a first mold, and the second chip may be encapsulated by a second mold. The first and/or the second molds may be thermally conductive. A third chip (e.g., a memory) may be on the second substrate opposite the second chip. The second substrate may include very short vertical connections that connect the active sides of the second and third chips.