GPU Memory Virtualization via Address Space Identifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional GPU execution models in high-performance computing (HPC) applications lead to poor GPU utilization and performance due to the inability to concurrently execute tasks with different virtual address spaces, resulting in inefficient execution of applications that require independent virtual address spaces.
Innovation Solution
A method for concurrently executing multiple thread programs with different virtual address spaces on a parallel processing subsystem by retrieving task metadata, scheduling thread programs across processing cores, and using address space identifiers for virtual address requests, allowing simultaneous execution and address space isolation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the GPU executes one thread program at a time within a corresponding virtual address space, then each thread program has its own private virtual address space ensuring address space isolation, but only one thread program may execute on the GPU at any one time resulting in poor GPU utilization
Solution Approach 1:
The patent segments the virtual address space management by introducing address space identifiers (ASIDs) that allow multiple virtual address spaces to coexist within the GPU. Each thread program is associated with a unique ASID, enabling the GPU to maintain multiple page tables concurrently, each corresponding to a different virtual address space. This segmentation allows the GPU to switch between different virtual address spaces without requiring exclusive execution of single thread programs.
Solution Approach 2:
The patent adds a new dimension to address space management by introducing ASIDs as an additional identifier layer. Instead of relying solely on sequential execution contexts, the system now operates in a multi-dimensional address space where each dimension is represented by an ASID. This allows the GPU to concurrently manage multiple virtual address spaces (e.g., first virtual address space with ASID 1, second virtual address space with ASID 2) within the same execution pipeline, effectively transforming the single-thread execution model into a multi-address-space concurrent execution model.
2Reliability
If the GPU sequentially executes tasks comprising a small number of thread instances that each requires an independent virtual address space, then each task has its own virtual address space ensuring proper isolation, but the sequential execution results in poor GPU utilization and poor overall application performance
Solution Approach 1:
The patent prepares multiple virtual address spaces in advance by pre-loading multiple page tables into the GPU's memory management unit before execution begins. Each page table is associated with a specific ASID, allowing the GPU to have multiple virtual address spaces ready for concurrent access. This preliminary preparation eliminates the need for time-consuming context switches during execution, as all necessary address space mappings are already in place.
Solution Approach 2:
The patent changes the execution model parameter from sequential single-address-space execution to concurrent multi-address-space execution. By modifying how the GPU handles address space identifiers and page table selections, the system enables multiple thread programs with different ASIDs to execute simultaneously on the same GPU hardware, fundamentally altering the execution paradigm from sequential to concurrent while maintaining address space independence.
3Productivity
If multiple thread programs are executed concurrently on the GPU, then GPU utilization improves, but conventional execution models only allow one thread program at a time within a single virtual address space
Solution Approach 1:
The patent makes the GPU execution engine universal by enabling it to handle multiple types of thread programs concurrently, each with its own virtual address space. The execution engine is modified to accept thread programs from different address spaces simultaneously, allowing it to function as both a single-address-space sequential executor and a multi-address-space concurrent executor. This multi-functionality is achieved through the ASID mechanism that routes memory access requests to the appropriate page table based on the thread program's associated address space.
Data Source
AI summary
A technique for simultaneously executing multiple tasks, each having an independent virtual address space, involves assigning an address space identifier (ASID) to each task and constructing each virtual memory access request to include both a virtual address and the ASID. During virtual to physical address translation, the ASID selects a corresponding page table, which includes virtual to physical address mappings for the ASID and associated task. Entries for a translation look-aside buffer (TLB) include both the virtual address and ASID to complete each mapping to a physical address. Deep scheduling of tasks sharing a virtual address space may be implemented to improve cache affinity for both TLB and data caches.