3D Memory Package Layout for Bandwidth and Power Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory solutions face bottlenecks in memory bandwidth and power consumption due to the physical distance between compute nodes and memory, leading to sub-optimal communication and increased costs, as computational advances outpace memory bandwidth.
Innovation Solution
A device configuration with a logic die and multiple types of memory dies, including high-bandwidth memory and other memory types, stacked on top of each other with direct electrical connections, optimizing memory hierarchy for power, latency, and capacity by utilizing 3D stacking technology.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If memory is placed farther from compute node to increase capacity, then memory capacity increases, but power consumption increases and bandwidth decreases
Solution Approach 1:
The patent transitions from traditional 2D memory placement to 3D stacking architecture, where HBM2 memory is vertically stacked above the compute node die. This dimensional change allows memory to be positioned much closer to the compute node in the vertical dimension, reducing physical distance and improving bandwidth while maintaining scalability for large capacity requirements.
Solution Approach 2:
The patent implements nested memory structures where HBM2 memory stacks are positioned above and near the compute node die, creating a hierarchical nesting arrangement. Multiple HBM2 stacks can be nested around the compute node, maximizing memory capacity while keeping all memory close to the compute node, thus reducing power consumption and increasing bandwidth simultaneously.
2Quantity of substance
If memory is placed farther from compute node to increase capacity, then memory capacity increases, but bandwidth decreases
Solution Approach 1:
The patent transitions from traditional 2D memory placement to 3D stacking architecture, where HBM2 memory is vertically stacked above the compute node die. This dimensional change allows memory to be positioned much closer to the compute node in the vertical dimension, reducing physical distance and improving bandwidth while maintaining scalability for large capacity requirements.
Solution Approach 2:
The patent merges multiple HBM2 memory stacks with the compute node die in a tightly integrated 3D package. By combining multiple high-bandwidth memory stacks around the compute node, the system achieves both large total capacity and high aggregate bandwidth, as all memory resides in close proximity to the compute node.
3Productivity
If pin bandwidth is increased to improve memory communication, then bandwidth increases, but cost increases
Solution Approach 1:
The patent transitions from traditional 2D memory placement to 3D stacking architecture, where HBM2 memory is vertically stacked above the compute node die. This dimensional change allows memory to be positioned much closer to the compute node in the vertical dimension, reducing physical distance and improving bandwidth while maintaining scalability for large capacity requirements.
Data Source
AI summary
A memory solution device may include a logic die, a high-bandwidth memory, and a first memory die. The logic die may be a central processing unit or an accelerator, and may include a first surface. The high-bandwidth memory die may be located on the first surface at a first predetermined location. The first memory die may be located on the first surface at a second predetermined location that is different from the first predetermined location. The first memory die may be a read-only memory, a random access memory, a non-volatile memory, or a combination thereof.