GPU Command Scheduler With Idle Periods For Power Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional GPU command scheduling approaches are application-centric, leading to inefficient use of GPU resources and increased power consumption, as they do not account for the specific needs and capabilities of individual GPUs, potentially causing the GPU to be monopolized or blocked by individual applications.
Innovation Solution
A method that decouples command batch dispatching from GPU processing requests, allowing for the creation of idle periods by analyzing and excluding buffered commands using a graph-based approach, enabling the GPU to enter power-saving modes and optimizing resource allocation through an operating system-controlled scheduling process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional application-centric GPU command scheduling is used, then individual applications can be allocated GPU time based on their demand, but the GPU may be monopolized or blocked by individual applications and resource allocation efficiency deteriorates
Solution Approach 1:
The patent segments the command queue into multiple per-application queues and introduces a scheduler that divides GPU time into fixed time slices allocated to different applications. This segmentation prevents any single application from monopolizing the GPU while maintaining fair resource distribution based on time slices rather than demand-based allocation.
Solution Approach 2:
The patent implements periodic action by using fixed time slices for each application in a round-robin fashion. The scheduler repeatedly cycles through applications in a predetermined order, allocating GPU time in periodic intervals. This ensures that even applications with low demand receive periodic service while preventing continuous monopolization by high-demand applications.
2Productivity
If the GPU processes commands continuously without idle periods, then processing throughput is maximized, but power consumption increases and the GPU cannot enter power-saving modes
Solution Approach 1:
The patent creates idle periods between consecutive GPU transactions by introducing periodic gaps in the command stream. The scheduler allows the GPU to remain idle for a predetermined period between processing different applications or command batches, enabling the GPU to enter low-power or sleep modes during these idle intervals while maintaining continuous processing capability when needed.
Solution Approach 2:
The patent performs preliminary actions by buffering commands in advance and pre-calculating when idle periods should occur. The scheduler anticipates when the GPU can safely enter idle mode by analyzing the command queue and scheduling idle periods before they are needed, allowing the GPU to transition to power-saving states proactively rather than reactively.
3Loss of time
If command batches are dispatched immediately upon GPU readiness, then response time is minimized, but the GPU cannot be properly controlled for power management and idle period creation
Solution Approach 1:
The patent introduces a scheduler as an intermediary component between the command queue and the GPU driver. This mediator receives commands from the queue, analyzes them according to power management policies, and dispatches appropriate command batches to the GPU at optimized intervals. The scheduler acts as a buffer that decouples immediate command availability from actual GPU execution timing, enabling both fast response and power management control.
Solution Approach 2:
The patent implements dynamic control by adjusting the dispatching behavior based on real-time conditions. The scheduler dynamically determines when to dispatch commands and when to create idle periods, adapting the GPU's operational state based on system power management requirements. This dynamic approach allows the system to switch between immediate response mode and power-saving mode as needed.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A technique for scheduling the dispatching of Graphics Processing Unit (GPU) commands is described. A method implementation of this technique comprises receiving commands from a plurality of applications, buffering the received commands, and dispatching the buffered commands as a command batch towards a GPU. The dispatching is scheduled such that a GPU idle period is created between two successive command batches dispatched towards the GPU.