Urgency-Based Process Preemption on Accelerated Processing Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems face challenges in efficiently utilizing GPUs for general computation due to constrained programming environments, separate memory systems, scheduling inefficiencies, and high latency, which limits the effective use of GPUs and prevents fine-grain offloading and context switching.
Innovation Solution
The implementation of a system that allows for preempting processes running on an Accelerated Processing Device (APD) based on urgency, using an interface that exposes the urgency of the request and provides a grace period for completing issued work, enabling efficient context switching and resource management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If processes are allowed to run on GPU without preemption, then computational tasks can complete fully, but system resource utilization deteriorates due to rogue processes occupying GPU for arbitrary amounts of time
Solution Approach 1:
The patent applies preliminary action by establishing preemption thresholds and urgency-based scheduling mechanisms before process execution begins. The system pre-configures time quantum limits, priority levels, and grace period parameters that will be enforced during execution, preventing rogue processes from indefinitely occupying GPU resources while allowing legitimate processes to complete their computational tasks.
Solution Approach 2:
The patent implements dynamics by making the scheduling policy adaptive and responsive. The system dynamically adjusts process priorities, preemption decisions, and grace period enforcement based on real-time urgency assessments. This allows the GPU scheduling mechanism to flexibly respond to changing workload conditions, balancing between allowing process completion and preventing resource hoarding by rogue processes.
2Device complexity
If context switching is constrained to coarse granularity and limited points, then hardware complexity is reduced, but processing efficiency deteriorates due to inability to switch between processes
Solution Approach 1:
The patent applies segmentation by dividing the continuous execution timeline into discrete time quantum segments. Instead of allowing unlimited context switching or no context switching, the system segments execution into manageable intervals where preemption can occur. This segmentation enables fine-grain control over when context switches are permitted, improving processing efficiency without overwhelming hardware complexity.
Solution Approach 2:
The patent introduces an intermediary scheduling layer between the CPU and GPU that mediates context switching decisions. This intermediary component handles the complexity of monitoring process urgency, managing time quantum expiration, and coordinating context switches, thereby enabling efficient processing without requiring complex hardware modifications.
3Ease of operation
If commands are dispatched to GPU without urgency-based preemption, then programming simplicity is maintained, but latency increases due to commands waiting in line for previously issued commands
Solution Approach 1:
The patent implements feedback mechanisms that monitor GPU queue status and process urgency levels in real-time. The scheduling system receives feedback about command queue depth, process priorities, and time quantum expiration, and uses this information to make dynamic preemption decisions. This feedback loop enables the system to reduce command dispatch latency by preempting lower-priority processes when queues are congested, while maintaining programming simplicity through automated management.
Data Source
AI summary
The desire to use an Accelerated Processing Device (APD) for general computation has increased due to the APD's exemplary performance characteristics. However, current systems incur high overhead when dispatching work to the APD because a process cannot be efficiently identified or preempted. The occupying of the APD by a rogue process for arbitrary amounts of time can prevent the effective utilization of the available system capacity and can reduce the processing progress of the system. Embodiments described herein can overcome this deficiency by enabling the system software to pre-empt a process executing on the APD for any reason. The APD provides an interface for initiating such a pre-emption. This interface exposes an urgency of the request which determines whether the process being preempted is allowed a grace period to complete its issued work before being forced off the hardware.