GPU Memory Virtualization via Address Space Identifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional GPU execution models in high-performance computing (HPC) applications lead to poor GPU utilization and overall performance due to the inability to concurrently execute tasks with different virtual address spaces, resulting in inefficient execution of applications that require independent virtual address spaces.
Innovation Solution
A method for concurrently executing multiple thread programs with different virtual address spaces on a parallel processing subsystem by retrieving task metadata, scheduling thread programs across processing cores, and using address space identifiers for virtual address requests, enabling simultaneous execution of tasks with address space isolation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional GPU execution model is used with one thread program per virtual address space, then address space isolation is maintained, but GPU utilization deteriorates due to sequential execution of multiple MPI tasks
Solution Approach 1:
The patent segments the virtual address space management by introducing address space identifiers (ASIDs) that allow multiple independent virtual address spaces to coexist within the same GPU context. Each thread program is associated with a unique ASID, enabling the GPU to maintain multiple page tables concurrently and execute multiple thread programs with different virtual address spaces simultaneously, thereby improving GPU utilization while preserving address space isolation.
Solution Approach 2:
The patent makes the GPU execution model universal by enabling it to handle multiple virtual address spaces through a single context. The introduction of ASIDs allows the GPU to function as both a single-threaded processor with isolation and a multi-threaded processor with high utilization, depending on how contexts and ASIDs are configured. This multi-functionality resolves the contradiction between isolation and utilization.
2Reliability
If multiple thread programs are executed sequentially on GPU, then each thread program has dedicated virtual address space, but execution time increases due to context switching overhead
Solution Approach 1:
The patent performs preliminary action by pre-configuring multiple page tables and associating them with different ASIDs before execution. The context structure is prepared in advance to include references to multiple page tables, allowing the GPU to switch between different virtual address spaces without the overhead of traditional context switching. This eliminates the need for time-consuming context switches during execution while maintaining virtual address space independence.
Solution Approach 2:
The patent introduces ASIDs as an intermediary mechanism that mediates between multiple virtual address spaces and the GPU's memory management unit. The ASID acts as a key that quickly identifies which page table to use for address translation, eliminating the need for full context switches. This intermediary approach maintains virtual address space independence while dramatically reducing the time loss associated with switching between thread programs.
3Device complexity
If single thread program executes on GPU at a time, then memory management is simplified, but overall application performance deteriorates due to underutilization of GPU resources
Solution Approach 1:
The patent changes the parameter of address space identification from a context-based model to an ASID-based model. By assigning unique ASIDs to different thread programs and configuring the memory management unit to use ASIDs for page table selection, the system can manage multiple virtual address spaces with the same level of simplicity as a single address space. This parameter change enables concurrent execution of multiple thread programs, improving application performance without significantly increasing memory management complexity.
Data Source
AI summary
A technique for simultaneously executing multiple tasks, each having an independent virtual address space, involves assigning an address space identifier (ASID) to each task and constructing each virtual memory access request to include both a virtual address and the ASID. During virtual to physical address translation, the ASID selects a corresponding page table, which includes virtual to physical address mappings for the ASID and associated task. Entries for a translation look-aside buffer (TLB) include both the virtual address and ASID to complete each mapping to a physical address. Deep scheduling of tasks sharing a virtual address space may be implemented to improve cache affinity for both TLB and data caches.