PIM Memory Controller for Concurrent GPU and DRAM Die Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory management systems in high-performance computing and AI systems lack efficient concurrent execution of processing-in-memory operations, leading to reduced throughput and increased latency due to serialized access and mode changes between GPU and DRAM die processing units.
Innovation Solution
A hybrid architecture with a memory processing unit (MPU) agent enables concurrent execution of GPU and DRAM die processing units, allowing simultaneous access and processing-in-memory operations through a custom memory controller and region snoop filter, facilitating GPU-PIM execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If processing units on memory dies are activated for PIM operations, then processing capability is improved, but power consumption increases
Solution Approach 1:
The system dynamically activates processing units on memory dies only when PIM operations are required, and deactivates them when not needed. The memory controller monitors operation types and selectively powers up processing units based on workload characteristics, transitioning between active and inactive states to optimize the trade-off between processing capability and power consumption.
Solution Approach 2:
The system changes the operational state parameter of processing units from inactive to active based on the type of memory operation being performed. When PIM operations are detected, the processing units are activated; otherwise, they remain inactive. This parameter change allows the system to adapt power consumption levels to actual processing needs.
2Reliability
If memory access is serialized between GPU and DRAM processing units, then resource conflicts are reduced, but throughput decreases
Solution Approach 1:
The memory system is segmented into multiple independent memory channels, with each channel having its own processing units on the memory die. This segmentation allows different channels to operate simultaneously and independently, enabling the GPU and DRAM processing units to access different channels concurrently without resource conflicts, thereby maintaining high throughput while ensuring reliable operation.
Solution Approach 2:
The memory controller acts as an intermediary that manages and coordinates access between the GPU and DRAM processing units. It allocates memory channels and processing units to different requests, enabling concurrent access without direct conflicts. The intermediary resolves potential resource conflicts while maintaining high system throughput through intelligent resource management.
3Adaptability or versatility
If mode changes between GPU and DRAM die processing units are implemented, then operational flexibility is improved, but latency increases
Solution Approach 1:
The system maintains continuous useful action by keeping processing units on memory dies in an active state when PIM operations are anticipated or ongoing. Instead of deactivating and reactivating processing units for each operation, the system sustains their active state, eliminating mode change transitions and associated latency while maintaining operational flexibility through software-controlled task allocation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are systems, methods, and apparatuses for systems and methods of execution of processing-in-memory, PIM, operations. In one aspect, the systems, devices, and methods include receiving a memory request for processing on a memory die; modifying the memory request based on the memory request being configured for processing on the memory die; and sending the modified memory request to a processing unit on the memory die. In another aspect, the systems, devices, and methods include enabling the memory die for processing of PIM on-die processing and disabling the memory die for off-die processing.