GPU Resource Scheduling for Priority Isolation and Elastic Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The scarcity of GPU computational resources and inefficient utilization due to waste in current usage, particularly in AI scenarios, where tasks with different service levels compete for shared resources, leading to interference and unsatisfied service levels.
Innovation Solution
A scheduler that dynamically allocates tasks with varying service levels to appropriate GPU hardware based on resource availability, ensuring high-priority tasks exclusively use resources and lower-priority tasks are rescheduled when high-priority tasks reach a threshold, allowing for elastic sharing and reduced interference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If GPU hardware is shared among multiple computing tasks to improve resource utilization, then resource utilization is improved, but service level of high-priority tasks deteriorates due to resource interference
Solution Approach 1:
The patent segments GPU computational resources into different pools based on task priority. High-priority tasks are allocated dedicated GPU resource pools while low-priority tasks use separate pools or idle resources, preventing resource interference and ensuring service levels are met while improving overall utilization
Solution Approach 2:
The patent implements dynamic resource allocation where the scheduler continuously monitors resource usage and task requirements, adjusting resource distribution in real-time. When high-priority tasks need resources, the system dynamically allocates them, and when resources are idle, low-priority tasks can utilize them, making the system adaptive to changing conditions
2Reliability
If GPU hardware is allocated exclusively to high-priority tasks to ensure service level, then service level is improved, but resource utilization deteriorates due to idle resources
Solution Approach 1:
The patent applies different resource allocation strategies to different parts of the system. Critical resources needed by high-priority tasks are protected and allocated exclusively, while non-critical resources are made available for low-priority tasks or idle utilization, creating local quality differences in resource access rights
Solution Approach 2:
The patent makes GPU resources universal by enabling them to serve multiple purposes and task types. The same GPU hardware can be allocated to high-priority tasks when needed and to low-priority tasks when idle, making the resource pool multi-functional and adaptable to different workload requirements
3Loss of energy
If multiple types of computing tasks are allowed on the same GPU hardware to improve utilization, then resource utilization is improved, but task execution speed deteriorates due to resource contention
Solution Approach 1:
The patent segments task execution into isolated environments where high-priority and low-priority tasks run separately on different GPU resource pools. This segmentation prevents resource contention and ensures that tasks execute at full speed without interference from other tasks competing for the same resources
Solution Approach 2:
The patent introduces a scheduler as an intermediary between tasks and GPU resources. The scheduler manages resource allocation, task scheduling, and priority enforcement, acting as a mediator that coordinates task execution to minimize contention while maximizing utilization through intelligent resource management
Data Source
AI summary
This disclosure provides GPU computational resource scheduling methods and apparatuses. In an implementation, a method includes: in response to a target computing task created in a computing cluster, determining a task type of the target computing task. If the target computing task is a first-type computing task, scheduling, for running, the target computing task to a first GPU hardware that has remaining computational resources satisfying a computational demand of the target computing task in the computing cluster. In response to a first indication indicating that is reported by a first computing node integrated with the first GPU hardware and that indicates that the first-type computing task exclusively occupies computational resources of the first GPU hardware, rescheduling, for running to a second GPU hardware that has remaining computational resources satisfying a computational demand of the second-type computing task in the computing cluster.