GPU Stream Scheduler Dynamic Kernel Dispatch
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing GPUs face challenges in efficiently scheduling kernel codes in kernel streams, leading to low hardware utilization rates and poor computation performance due to fixed sequencing and competition for hardware resources.
Innovation Solution
A GPU architecture that includes stream multiprocessors, an available hardware resource table, a resource comparator, a stream scheduler, and a global dispatcher, which dynamically selects and dispatches kernel codes based on available hardware resources and priority, optimizing execution and resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If kernel codes are executed in a fixed sequence in each kernel stream, then the execution order is predictable and simple to manage, but the hardware utilization rate decreases due to resource competition and inability to optimize scheduling
Solution Approach 1:
The patent applies dynamics by transforming the static fixed-sequence execution model into a dynamic scheduling system. The stream scheduler dynamically selects which kernel code to execute next based on real-time hardware resource availability and kernel priorities, rather than following a predetermined sequence. This allows the system to adapt to changing resource conditions and optimize hardware utilization while maintaining execution predictability through priority-based selection.
2Adaptability or versatility
If multiple kernel codes compete for the same hardware resources, then resource allocation becomes complex and scheduling difficulty increases, but parallel execution capability is reduced
Solution Approach 1:
The patent introduces an intermediary scheduling layer (stream scheduler and global dispatcher) that mediates between multiple kernel codes and shared hardware resources. This intermediary manages resource allocation by selecting appropriate kernels based on resource availability and priorities, preventing direct competition between kernels. The intermediary abstracts the complexity of resource management, allowing multiple kernels to coexist without direct conflict while maintaining efficient parallel execution.
3Productivity
If kernel streams are executed in parallel to enhance computation performance, then overall processing speed increases, but coordination and scheduling between streams becomes more difficult
Solution Approach 1:
The patent applies universality by designing a unified scheduling architecture (global dispatcher and stream scheduler) that handles multiple kernel streams simultaneously through a common set of rules and resource management mechanisms. This universal scheduler can manage any combination of kernel streams regardless of their specific requirements, coordinating their parallel execution while maintaining system-wide resource optimization. The same scheduling logic applies across all streams, simplifying coordination complexity despite increased parallel processing capability.
Data Source
AI summary
A GPU and a GPU method are provided. The GPU includes stream multiprocessors, an available hardware resource table, resource comparator, stream scheduler, and global dispatcher. Each stream multiprocessor executes at least one thread block. The available hardware resource table records hardware resources available to the stream multiprocessors. The resource comparator selects from first-priority kernel codes in the kernel streams at least one first dispatchable kernel code whose required hardware resources are less than hardware resources available to the stream multiprocessors according to the available hardware resource table. The stream scheduler selects from the at least one first dispatchable kernel code a kernel code as a selected kernel code. The global dispatcher dispatches thread blocks in the selected kernel code to the stream multiprocessors for execution and updates the available hardware resource table according to usage conditions of hardware resources of the stream multiprocessors.