GPU Demand Paging for Large Random Access Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graphics processing unit (GPU) applications face challenges when dealing with large input data that exceeds GPU device memory, particularly in scenarios requiring random access, such as recursive ray tracing, where existing solutions like splitting computations into multiple kernel runs are inadequate.
Innovation Solution
Implementing GPU demand paging by organizing input data into pages, managing them in a virtual address space, and temporarily storing them in a physical cache, with mechanisms for accessing and updating pages based on usage, including time stamps and page table lookups to facilitate efficient access and eviction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If input data is split into multiple kernel runs to process large datasets, then memory capacity limitation is addressed, but processing efficiency deteriorates for random access operations
Solution Approach 1:
The patent divides the large input data into fixed-size pages that can be independently managed and loaded into GPU memory. This segmentation allows the system to handle datasets larger than available memory by loading only the required pages, thus resolving the contradiction between processing large quantities of data and maintaining processing efficiency.
Solution Approach 2:
The patent introduces a page management system with virtual address space and page tables as an intermediary between the application and physical GPU memory. This intermediary layer handles page loading, eviction, and address translation, enabling efficient random access to large datasets without requiring all data to be simultaneously present in GPU memory.
2Speed
If all input data is loaded into GPU device memory, then random access efficiency is improved, but memory capacity is exceeded
Solution Approach 1:
The patent implements dynamic page management where pages are loaded into and evicted from GPU memory based on actual access patterns during computation. The page replacement policy dynamically decides which pages to evict when memory is full, allowing the system to adapt to varying data access requirements while maintaining fast access speeds for actively used data.
Solution Approach 2:
The patent changes the state of data from requiring complete presence in memory to being accessible through virtual addresses with on-demand loading. By introducing page presence/absence states and virtual-to-physical address translation, the system enables random access efficiency without loading all data simultaneously into GPU memory.
3Quantity of substance
If sequential processing is used to fit data in memory, then memory usage is reduced, but processing capability for random access algorithms deteriorates
Solution Approach 1:
The patent creates a universal memory management system that supports both sequential and random access patterns through page-based virtual memory. The page management mechanism provides a unified interface that works with various algorithms regardless of their access patterns, making the GPU capable of handling diverse workloads including ray tracing, finite element analysis, and other random-access-intensive applications.
Data Source
AI summary
A system, method, and computer program product are provided for GPU demand paging. In operation, input data is addressed in terms of a virtual address space. Additionally, the input data is organized into one or more pages of data. Further, the input data organized as the one or more pages of data is at least temporarily stored in a physical cache. In addition, access to the input data in the physical cache is facilitated.