3D Stacked Memory on Processor for Local High-Bandwidth Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current high-performance computing and graphics processing systems face significant memory limitations due to a high byte to floating-point operation (B:F) ratio, leading to increased energy consumption and latency in memory accesses.
Innovation Solution
The solution involves stacking memory dies on top of a processor die, creating a one-level memory system with vertically aligned memory tiles directly coupled to processing tiles, using conductive paths for efficient data transfer and reducing routing lengths, thereby improving memory bandwidth and reducing energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory is stacked vertically on processor die, then memory bandwidth is improved, but device complexity increases
Solution Approach 1:
The patent transitions from conventional planar memory architecture to a three-dimensional vertically stacked architecture. Memory dies are stacked above the processor die along the vertical dimension, with through-silicon vias providing conductive paths through multiple die layers. This dimensional change enables significantly increased memory bandwidth by providing direct vertical access paths from processing tiles to memory tiles, eliminating the need for lengthy lateral routing through the processor substrate.
Solution Approach 2:
The patent implements a nested hierarchical structure where memory dies are physically stacked above the processor die, forming a compact three-dimensional integration. Each die layer is nested within the overall device package, with the processor die at the base, through-silicon via layers in the middle, and memory dies stacked above. This nesting approach maximizes memory capacity and bandwidth within a small footprint while organizing complex interconnections in a structured manner.
2Use of energy by moving object
If memory access is localized, then energy consumption is reduced, but memory capacity is limited
Solution Approach 1:
The patent divides the memory system into multiple independent memory dies, each stacked above specific processing tiles. Each memory die can be accessed independently through dedicated through-silicon via paths, allowing localized memory access for energy efficiency. The segmentation enables the system to provide large total memory capacity across multiple dies while allowing processing tiles to access only the memory portions they need, minimizing energy consumption for memory operations.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances memory bandwidth by 50× and reduces energy per bit transfer by 10×, significantly improving performance and efficiency in memory access operations.
Implementation Method 1
conductive paths between each processing tile and a corresponding memory tile in each memory die
Data Source
AI summary
Embodiments of the present disclosure relate to memory stacked on processor for high bandwidth. Systems and methods are disclosed for providing a one-level memory for a processing system by stacking bulk memory on a processor die. In an embodiment, one or more memory dies are stacked on the processor die. The processor die includes multiple processing tiles, where each tile includes a processing unit, mapper, and tile network. Each memory die includes multiple memory tiles. The processing tile is coupled to each memory tile that is above or below the processing tile. The vertically aligned memory tiles comprise the local memory block for the processing tile. The ratio of memory bandwidth (byte) to floating-point operation (B:F) may improve 50× for accessing the local memory block compared with conventional memory. Additionally, the energy consumed to transfer each bit may be reduced by 10×.