GPU Universal Processing Context for Multi-Application Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for sharing hardware resources on a GPU between multiple compute applications require central resource management, leading to thread unresponsiveness and increased GPU idle time due to context switching and synchronization issues.
Innovation Solution
Establishing a universal processing context shared by multiple applications, allowing them to be executed simultaneously by the GPU with reduced context switches through a server-managed processing system within the GPU driver.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a central resource manager is used to receive workloads from compute applications and transmit them to the GPU for execution, then resource sharing between applications is enabled, but thread unresponsiveness occurs during synchronization operations and system responsiveness deteriorates
Solution Approach 1:
The patent introduces a queue mechanism as an intermediary between compute applications and the GPU. Each application has its own dedicated queue where workloads are submitted and stored independently. The GPU can process workloads from multiple queues simultaneously without requiring synchronization between application threads, eliminating the unresponsiveness issue while maintaining resource sharing capability.
2Stability of the object's composition
If the GPU controls synchronization between compute applications and permits only threads of a single application to be executed at a time, then resource contention is managed, but GPU idle time increases due to context switches
Solution Approach 1:
The patent segments the workload management by creating separate queues for each compute application. Instead of executing threads from a single application at a time, the GPU can select and execute workloads from any ready queue independently. This segmentation eliminates the need for traditional context switches between applications, reducing GPU idle time while maintaining controlled execution through the queue mechanism.
3Reliability
If context switches are performed for every transition to threads of a different compute application, then application isolation is maintained, but overhead increases and performance decreases
Solution Approach 1:
The queue structure serves as an intermediary that maintains application isolation without requiring context switches. Each application's workloads are submitted to its dedicated queue with metadata identifying the source application. The GPU can process workloads from different queues in any order while the queue mechanism ensures proper attribution and isolation, eliminating performance-degrading context switches while maintaining application boundaries.
4Reliability
If large sets of state associated with compute applications are stored and reloaded during context switches, then application state integrity is preserved, but memory overhead and execution time increase
Solution Approach 1:
The patent extracts the state management burden from the GPU execution flow by using queues that store workload descriptors and metadata in host memory. The actual application state remains in the application's address space, and only lightweight workload pointers and control data need to be transferred to the GPU. This eliminates the need to save and restore large application states during context transitions, reducing memory bandwidth consumption while preserving state integrity through the queue mechanism.
Data Source
AI summary
A technique for executing a plurality of applications on a GPU. The technique involves establishing a first connection to a first application and a second connection to a second application, establishing a universal processing context that is shared by the first application and the second application, transmitting a first workload pointer to a first queue allocated to the first application, the first workload pointer pointing to a first workload generated by the first application, transmitting a second workload pointer to a second queue allocated to the second application, the second workload pointer pointing to a second workload generated by the second application, transmitting the first workload pointer to a first GPU queue in the GPU, and transmitting the second workload pointer to a second GPU queue in the GPU, wherein the GPU is configured to execute the first workload and the second workload in accordance with the universal processing context.