Standalone Processing-in-Memory Architecture for Scalable HBM Stacks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Centralized compute architectures in high-performance computing systems limit the number of memory dies and memory capacity, resulting in suboptimal performance for memory-intensive workloads due to restricted memory bandwidth and connectivity.
Innovation Solution
A standalone architecture with disaggregated compute and memory, incorporating stacked memory modules with integrated computing capability, such as High-Bandwidth Memory (HBM) cubes, and a programming model that allows for efficient execution and communication between processor elements via die-to-die connections and network-on-chip interconnects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If centralized compute architectures are used, then system integration is simplified, but memory capacity and memory bandwidth are limited
Solution Approach 1:
The system is divided into multiple independent stacked memory modules, each capable of autonomous processing. Each module contains its own base die with processor elements and stacked memory dies, creating modular units that can be independently configured and scaled without increasing overall system integration complexity.
Solution Approach 2:
The architecture transitions from traditional 2D centralized compute to 3D stacked memory modules with vertical die-to-die connections. This dimensional change enables significantly higher memory capacity and bandwidth within the same footprint by utilizing the vertical stacking dimension.
2Quantity of substance
If more memory dies are added to increase memory capacity, then memory bandwidth is improved, but connectivity and integration become more difficult
Solution Approach 1:
Multiple memory dies are merged into a single stacked memory module with integrated compute capability. The base die combines processor elements, interconnect, and memory controller functions, while memory dies are vertically stacked and interconnected through die-to-die bonds, creating a unified module that simplifies external connectivity.
Solution Approach 2:
The base die acts as an intermediary between the stacked memory dies and the external system. It provides memory controller functionality and interconnect interfaces that manage data flow between multiple memory dies and external processors, simplifying the connectivity complexity.
3Productivity
If stacked memory modules with integrated compute are used, then memory bandwidth and capacity are improved, but architecture complexity increases
Solution Approach 1:
The base die is designed as a universal platform that integrates multiple functions: processor elements for computation, interconnect for communication, memory controller for memory management, and support for multiple stacked memory dies. This multi-functionality reduces the need for separate components and simplifies the overall architecture despite the 3D stacking complexity.
4Quantity of substance
If die-to-die connections are implemented for stacked memory modules, then memory capacity and bandwidth are enhanced, but manufacturing complexity increases
Solution Approach 1:
Die-to-die connections are established during the module assembly process before the module is integrated into the final system. The base die and memory dies are pre-aligned and bonded with precise interconnect patterns, ensuring reliable connections are formed in advance. This preliminary action allows for quality control and testing at the module level, reducing the impact of manufacturing precision requirements on the final system integration.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are systems, methods, and apparatuses for a standalone architecture for processing in memory. In one or more examples, the systems, devices, and methods include assigning, via application code of a host of a memory system in package, a kernel sub grid of a kernel grid to a stack of memory dies; assigning, via a microcontroller of a base die of the stack, execution of a first thread block of the kernel sub grid to a first processor element of the stack; assigning, via the microcontroller, execution of a second thread block of the kernel sub grid to a second processor element of the stack; executing threads of the first thread block on the first processor element; and executing threads of the second thread block on the second processor element.