GPU Resource Allocation via Global Logic Controller
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In GPU resource allocation systems, high-priority kernel programs face delays due to long-running kernel programs occupying SM resources, leading to inadequate response times and quality of service issues, as they can only access the GPU when an idle SM is available, which may not always be the case.
Innovation Solution
A method and system where a global logic controller determines a high-priority kernel program and allocates it to either an SM that can run a complete block or a warp, ensuring timely access by searching for available resources, even if an idle SM is not immediately available, thereby reducing wait times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If kernel programs access GPU one by one in chronological order, then resource allocation is simple, but high-priority kernel programs cannot obtain timely response
Solution Approach 1:
The patent introduces priority levels as a new parameter for kernel program scheduling, changing the scheduling criterion from chronological order to priority-based selection. The global logic controller maintains a kernel status register table that tracks priority levels, allowing high-priority kernels to preempt lower-priority ones and access the GPU immediately, thus reducing response time while maintaining manageable complexity through structured priority management
Solution Approach 2:
The patent implements dynamic resource allocation where the GPU assignment is not fixed but changes based on real-time priority conditions. When a high-priority kernel program needs access, the system dynamically reassigns SM resources from lower-priority kernels, creating a flexible scheduling mechanism that adapts to changing priorities rather than following static chronological ordering
2Productivity
If an idle SM is searched for high-priority kernel, then resource utilization improves, but wait time increases when no idle SM is available
Solution Approach 1:
The patent segments the kernel program execution into blocks that can be distributed across multiple SMs independently. Instead of treating the entire kernel as a single unit that requires one idle SM, the global logic controller divides the kernel into smaller block units that can be allocated to different SMs, including those currently executing other kernels. This segmentation allows high-priority kernels to utilize partially available resources rather than waiting for complete idle SMs
Solution Approach 2:
The patent allows high-priority kernel programs to execute with partial resource allocation rather than requiring complete idle SMs. The system can allocate blocks of the high-priority kernel to SMs that have partial availability, enabling the kernel to start execution immediately with the resources currently available, and continue acquiring additional resources as they become available, thus eliminating the need to wait for fully idle SMs
3Stability of the object's composition
If long-delayed kernel program occupies SM resource, then sequential processing is maintained, but quality of service deteriorates
Solution Approach 1:
The patent implements preliminary scheduling decisions based on priority levels before actual GPU execution begins. The global logic controller pre-identifies high-priority kernel programs and prepares their block allocations in advance, so when these kernels need execution, they can immediately access pre-identified SM resources without being blocked by sequentially executing lower-priority kernels. This preliminary prioritization maintains system stability while improving service quality for critical tasks
Solution Approach 2:
The patent introduces the global logic controller as an intermediary layer between kernel programs and SM resources. This intermediary maintains a kernel status register table that tracks priority levels and makes intelligent routing decisions, allowing high-priority kernels to bypass lower-priority kernels in the execution queue. The intermediary manages the reassignment of SM resources dynamically, enabling quality of service improvements while maintaining overall system coherence through centralized control
Data Source
AI summary
The present invention provides a GPU resource allocation method. A to-be-distributed kernel program in a kernel status register table is identified by using a global logic controller (201). The global logic controller searches in an SM status register table for an SM that can run at least one complete block, and the SM status register table is configured to store a quantity of available resources in each SM (202). When the global logic controller does not find the SM that can run at least one complete block, the SM status register table is searched for a first SM, and the first SM is an SM that can run at least one warp (203). When the global logic controller finds the first SM, a block in the to-be-distributed kernel program is distributed to the first SM (204).