Processor Resource Queue Scheduling for Multi-Task GPU Contention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face frequent task request failures due to insufficient resources when multiple tasks compete for processor resources, leading to increased complexity, resource overhead, and reduced performance, particularly in environments using Kubernetes (K8S) with customized GPU scheduling algorithms.
Innovation Solution
A method and system that involves a client, proxy node, and server nodes with containers, where task requests are forwarded to a server node based on user IDs, and managed through task request queues, ensuring efficient allocation and release of processor resources based on user correspondence and queue management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If Kubernetes and customized GPU scheduling algorithms are introduced to manage multiple tasks, then task scheduling capability is improved, but system complexity increases and performance decreases
Solution Approach 1:
The patent extracts the scheduling logic from the complex Kubernetes ecosystem and implements it directly at the processor level through simple queue management. The server node maintains a task request queue for each processor, eliminating the need for K8S scheduling overhead while achieving similar task management capabilities.
Solution Approach 2:
The patent introduces a proxy node as an intermediary between client users and server nodes. The proxy node forwards task requests to appropriate server nodes based on user ID correspondence, simplifying the architecture by removing the need for complex K8S scheduling infrastructure while maintaining efficient task distribution.
2Productivity
If customized scheduling algorithms are added to achieve task scheduling, then task allocation is improved, but resource overhead increases
Solution Approach 1:
The server node autonomously manages task requests by maintaining its own task request queue and making scheduling decisions locally. The processor automatically selects tasks from the queue without requiring external K8S coordination, reducing resource overhead while maintaining efficient task allocation.
3Adaptability or versatility
If PODs are frequently applied and destroyed to enable scheduling, then scheduling flexibility is improved, but effective resource utilization decreases
Solution Approach 1:
The patent segments the scheduling unit from the POD concept and implements it through processor-level task request queues. Each processor maintains an independent queue that persists across task executions, eliminating the need for frequent POD creation and destruction while maintaining scheduling flexibility.
4Adaptability or versatility
If PODs are introduced for scheduling purposes, then scheduling capability is improved, but debugging difficulty increases
Solution Approach 1:
The patent uses simple queue data structures that are easy to implement and debug, replacing the complex POD abstraction. The task request queue is a straightforward first-in-first-out structure that is simpler to understand and troubleshoot compared to K8S PODs, reducing debugging difficulty while maintaining scheduling capability.
Data Source
Figure 1
Figure 2
Figure 3~5
AI summary
The present application provides a method, device and system of scheduling a resource of a processor, and relates to the field of computer technologies. The method includes: receiving a current task request forwarded by a proxy node from a client user corresponding to any container; adding the request to a queue: adding the current task request to a task request queue of a processor corresponding to the client user; allocating a resource: in a case where a resource of the processor is idle, fetching a task request at a head of the task request queue and allocating the resource of the processor to the currently fetched task request, so that the processor executes a task corresponding to the currently fetched task request; and returning a result: returning a task execution result to the proxy node after the task is completed and releasing the resource of the processor; and in a case where the task request queue is not empty, skipping to the step of allocating a resource. The present application may significantly reduce a probability of frequent failure of task request caused by insufficient resources when a plurality of tasks compete for a resource of the processor.