Standalone Processing-in-Memory Architecture for Scalable Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Centralized compute architectures in high-performance computing limit the scalability and performance of memory capacity and bandwidth, particularly in memory-intensive workloads.
Innovation Solution
A standalone architecture with disaggregated compute and memory, featuring stacked memory modules interconnected via die-to-die connections, includes a base die with a microcontroller, processor elements, shared memory, and an interconnect, supporting a programming model with kernel grids and thread blocks for efficient execution across multiple processor elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If centralized compute architectures are used, then system integration is simplified, but memory capacity and bandwidth scalability is limited
Solution Approach 1:
The system is divided into multiple independent stacked memory modules, each capable of autonomous operation with its own microcontroller and processor elements. This segmentation allows each module to be scaled independently while maintaining overall system functionality, resolving the contradiction between integration simplicity and memory scalability.
Solution Approach 2:
The architecture transitions from traditional 2D memory organization to 3D stacked memory modules connected via die-to-die interfaces. This dimensional change enables significant increases in memory capacity and bandwidth without proportionally increasing system complexity, as multiple memory layers are integrated vertically within each module.
2Productivity
If memory capacity and bandwidth are increased, then performance for memory-intensive workloads improves, but system complexity increases
Solution Approach 1:
By dividing the system into multiple identical stacked memory modules, high memory capacity and bandwidth are achieved through parallelism rather than complexity. Each module maintains a standardized, manageable complexity level while the aggregate system delivers high performance for memory-intensive workloads.
Solution Approach 2:
Each stacked memory module includes its own microcontroller and processor elements that can autonomously manage memory operations and processing tasks. This self-service capability reduces the burden on external controllers, allowing the system to scale memory capacity without proportionally increasing control complexity.
3Quantity of substance
If standalone architecture with disaggregated compute and memory is implemented, then memory capacity and bandwidth scalability improves, but interconnection complexity increases
Solution Approach 1:
The architecture merges compute elements (processor elements and microcontrollers) directly within each stacked memory module, eliminating the need for complex external interconnections between separate memory and compute subsystems. This integration maintains memory capacity and bandwidth scalability while reducing interconnection complexity through localized processing.
Solution Approach 2:
Die-to-die connections establish vertical interconnections between stacked memory layers within each module, creating a compact 3D interconnect structure. This approach achieves high bandwidth and capacity scaling without requiring complex 2D routing networks, as data can be accessed directly across stacked layers through short vertical paths.
4Productivity
If multiple processor elements are added, then compute resource scaling improves, but coordination and control complexity increases
Solution Approach 1:
Each stacked memory module contains its own microcontroller that independently manages the processor elements within that module. This segmentation of control authority allows compute resources to scale across multiple modules without requiring a single complex centralized controller, as each microcontroller handles coordination locally within its module.
Solution Approach 2:
The microcontroller and processor element architecture is designed as a universal, standardized unit that can be replicated across multiple stacked memory modules. This multi-functionality allows the same control logic to manage multiple processor elements through consistent interfaces, reducing coordination complexity despite increased compute resource scaling.
Data Source
AI summary
Provided are systems, methods, and apparatuses for a standalone architecture for processing in memory. In one or more examples, the systems, devices, and methods include assigning, via application code of a host of a memory system in package, a kernel sub grid of a kernel grid to a stacked memory module; assigning, via a microcontroller of a base die of the stacked memory module, execution of a first thread block of the kernel sub grid to a first processor element of the stacked memory module; assigning, via the microcontroller, execution of a second thread block of the kernel sub grid to a second processor element of the stacked memory module; executing threads of the first thread block on the first processor element; and executing threads of the second thread block on the second processor element.