Cooperative GPU Scheduling for Frame Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scheduling schemes for multiple frame-based applications on a processing unit, such as GPUs, are not accurately aware of the status or urgency of each application's frame workload, leading to sub-optimal scheduling decisions and increased frame completion latency, even when the processing unit is underutilized.
Innovation Solution
A cooperative scheduling approach where applications communicate their frame job status to the scheduler, allowing it to adjust priorities and schedule jobs to minimize latency, by sending signals when all jobs for a frame are queued or completed, enabling high-priority applications to preempt lower-priority ones and ensuring fair execution time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If existing scheduling schemes are used for multiple frame-based applications on a GPU, then the processing unit can execute multiple applications, but the scheduling decisions are sub-optimal because the scheduler is not accurately aware of the status or urgency of each application's frame workload, resulting in increased frame completion latency
Solution Approach 1:
The patent implements a feedback mechanism where applications send signals to the scheduler indicating their frame workload status (e.g., when all jobs for a frame are queued or completed). This feedback enables the scheduler to accurately track the urgency and status of each application's frame workload, allowing it to make informed scheduling decisions that minimize frame completion latency.
Solution Approach 2:
The patent introduces an intermediary signaling mechanism between applications and the scheduler. Applications communicate their frame job status through standardized signals, which act as intermediaries carrying information about workload urgency. This intermediary layer enables the scheduler to gain accurate awareness of application states without requiring direct complex communication protocols.
2Loss of time
If the scheduler increases priority for applications with urgent frame workloads, then frame completion latency is reduced, but the scheduling system becomes more complex due to dynamic priority adjustment and signal handling
Solution Approach 1:
The patent implements dynamic priority adjustment where the scheduler's treatment of different applications changes based on real-time feedback signals. When an application sends a signal indicating urgent frame workload status, the scheduler dynamically increases its priority. This dynamic behavior allows the system to optimize for frame completion latency while maintaining a relatively simple base scheduling structure that only becomes complex when needed.
Solution Approach 2:
The patent changes the priority parameter of applications dynamically based on received signals. When the scheduler receives a signal from an application indicating that all jobs for a frame are queued or completed, it changes the priority parameter of that application to ensure faster processing. This parameter change approach allows complex scheduling behavior to be achieved through simple parameter adjustments rather than complex scheduling logic.
3Productivity
If the scheduler processes all applications equally without priority adjustment, then the scheduling system remains simple, but processing hardware utilization is sub-optimal even when underutilized
Solution Approach 1:
The patent uses feedback signals from applications to the scheduler to enable productivity optimization. When applications signal their frame workload status, the scheduler can identify opportunities to better utilize processing hardware by prioritizing applications that are ready to execute. This feedback loop allows the system to improve hardware utilization without requiring complex proactive scheduling decisions.
Solution Approach 2:
The patent implements a self-service scheduling approach where applications themselves provide the scheduler with the information needed for optimal scheduling by sending signals about their frame workload status. This self-service mechanism allows the scheduler to improve hardware utilization and reduce frame completion latency without requiring complex monitoring or control of application internals, as the applications voluntarily provide the necessary scheduling information.
Data Source
AI summary
Systems, apparatuses, and methods for scheduling jobs for multiple frame-based applications are disclosed. A computing system executes a plurality of frame-based applications for generating pixels for display. The applications convey signals to a scheduler to notify the scheduler of various events within a given frame being rendered. The scheduler adjusts the priorities of applications based on the signals received from the applications. The scheduler attempts to adjust priorities of applications and schedule jobs from these applications so as to minimize the perceived latency of each application. When an application has enqueued the last job for the current frame, the scheduler raises the priority of the application to high. This results in the scheduler attempting to schedule all remaining jobs for the application back-to-back. Once all jobs of the application have been completed, the priority of the application is reduced, permitting jobs of other applications to be executed.