GPU-Shared Dispatching via Resource Markers in Kubernetes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU dispatching methods in Kubernetes clusters support only single-machine single-card dispatching, leading to resource waste and utilization inefficiencies, especially during GPU development, as they fail to accurately manage and share GPU resources among multiple users.
Innovation Solution
A GPU-shared dispatching method that determines resource usage situations by querying and updating GPU information in environment variables using resource occupation and update markers, screens for optimal controlled hosts and GPUs, and allocates pods to the most suitable targets, ensuring consistent resource allocation and avoiding dispatching failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If single-machine single-card dispatching is used, then dispatching simplicity is maintained, but GPU resource utilization is low and resource waste occurs
Solution Approach 1:
The patent segments GPU resources at the card level rather than allocating entire GPUs to single pods. It divides GPU cards into multiple segments that can be shared among multiple pods on the same machine, enabling fine-grained resource allocation and improving utilization without requiring complex multi-machine coordination
Solution Approach 2:
The patent makes a single GPU card serve multiple functions by allowing multiple pods to share it simultaneously. The GPU resource can be divided and allocated to different pods based on their needs, making one GPU card universally serve multiple computational tasks rather than being dedicated to a single pod
2Quantity of substance
If fine-grained GPU segmentation is not performed, then dispatching management is simple, but at least one entire GPU must be allocated causing resource waste
Solution Approach 1:
The system segments GPU cards into allocatable units that can be divided among multiple pods. Each GPU card can be split into multiple segments, allowing precise allocation matching pod requirements and eliminating the waste of allocating entire GPUs when only partial resources are needed
Solution Approach 2:
The patent changes the allocation parameter from entire GPU cards to fractional GPU segments. This parameter change enables flexible allocation where pods can receive exactly the amount of GPU resources they need, improving allocation efficiency while maintaining manageable complexity through standardized segmentation
3Adaptability or versatility
If UUID of only one GPU is written to environment variables, then single-card dispatching works, but multi-card dispatching fails causing dispatching errors
Solution Approach 1:
The patent adds another dimension to GPU identification by using both pod-level UUIDs and GPU card-level UUIDs. This dimensional expansion allows the system to track and manage multiple GPU cards assigned to different pods, enabling multi-card dispatching while maintaining accuracy through hierarchical identification
Solution Approach 2:
The patent introduces an intermediary mapping mechanism that connects pod UUIDs with GPU card UUIDs. This intermediary layer resolves the conflict between single-UUID simplicity and multi-card complexity by providing a many-to-many mapping relationship that enables accurate multi-card dispatching
4Reliability
If resource usage situation is not determined before dispatching, then dispatching speed is fast, but resource allocation consistency cannot be ensured
Solution Approach 1:
The patent performs preliminary actions by determining resource usage situations and updating GPU information before actual dispatching occurs. This pre-check mechanism ensures allocation consistency by verifying resource availability and correctness beforehand, preventing dispatching failures without significantly impacting overall dispatching speed
Data Source
AI summary
The present application discloses Graphics Processing Unit (GPU)-shared dispatching and single-machine multi-card methods, systems, and devices, applied to a dispatcher of a central control host. In the present application, a resource occupation marker and an update marker are added to a pod, the resource occupation marker is used to prove that the pod is running, and the update marker is used to indicate that resource usage information in an annotation of the pod is consistent with actual resource usage information, whereby during dispatching, a resource usage situation of each pod may be determined. The resource usage information, i.e., GPU information of an actually used GPU, in the annotation of the pod is updated before dispatching, to ensure that the GPU information in the annotation is consistent with GPU information actually recorded in an environment variable of the pod.