High-Bandwidth Extended Memory for Parallel GPU Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing parallel processing systems face challenges in accessing extended memory due to limited high-bandwidth memory availability, leading to reduced performance when executing threads that require large memory spaces, and existing solutions incur bandwidth penalties or reduce memory availability on other GPUs.
Innovation Solution
A method for accessing extended memory in a parallel processing system by directing memory accesses through a memory controller associated with the central processing unit, bypassing system memory management units, and utilizing a high-speed chip-to-chip interface to access a portion of system memory as extended GPU memory (EGM), which is managed by the operating system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is paged in and out from system memory to GPU local memory, then threads can access large memory spaces, but GPU performance is significantly reduced due to low bandwidth
Solution Approach 1:
The patent merges system memory with GPU local memory to create a unified high-bandwidth memory space. The GPU can access both its local high-bandwidth memory and the CPU's system memory through the same high-speed interconnect, eliminating the need to page data through low-bandwidth system memory interfaces. This allows threads to access large memory capacities while maintaining high bandwidth performance.
Solution Approach 2:
The patent makes system memory accessible to the GPU through the high-speed interconnect, allowing the same memory resource to serve both CPU and GPU needs. The system memory can be used as extended GPU memory without requiring separate memory resources, enabling the GPU to access large datasets while the CPU continues to use the same memory space for its operations.
2Quantity of substance
If a GPU is allocated memory from another GPU via high-speed bus, then high-bandwidth memory capacity increases, but the allocating GPU's available memory is reduced
Solution Approach 1:
The patent makes system memory universally accessible to multiple GPUs through the high-speed interconnect. Instead of allocating memory from one GPU to another (which reduces the source GPU's availability), the system allows each GPU to access the shared system memory space independently, maintaining full availability for each GPU while providing extended memory capacity.
3Ease of operation
If IOMMU translation is used for direct system memory access, then memory management is simplified, but access bandwidth is reduced
Solution Approach 1:
The patent extracts the IOMMU translation step from the memory access path by allowing the GPU to directly access system memory through the high-speed interconnect without requiring address translation. The GPU's address space is directly mapped to the system memory physical addresses, eliminating the translation overhead and associated bandwidth reduction while maintaining simplified memory management through the unified address space.
Data Source
AI summary
Various embodiments include techniques for accessing extended memory in a parallel processing system via a high-bandwidth path to extended memory residing on a central processing unit. The disclosed extended memory system extends the directly addressable high-bandwidth memory local to a parallel processing system and avoids the performance penalties associated with low-bandwidth system memory. As a result, execution threads that are highly parallelizable and access a large memory space execute with increased performance on a parallel processing system relative to prior approaches.