Kubernetes GPU Resource Sharing via Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing resource scheduling mechanisms in Kubernetes, such as those used for container orchestration, often allocate entire devices (e.g., GPUs) exclusively to a single application, leading to resource wastage when not all processing capabilities are utilized.
Innovation Solution
A resource scheduling method and system that allows for the sharing of devices like GPUs across multiple applications, enabling flexible allocation of shareable resources based on the requirements of each container, thereby optimizing resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If an exclusive device is allocated to a single application, then the application can use the full processing capabilities of the device, but resource waste occurs when the application cannot utilize all processing capabilities
Solution Approach 1:
The patent segments the GPU device into multiple independent shareable resources (e.g., GPU 0, GPU 1, GPU 2) that can be independently allocated to different containers. This allows the device to be divided into allocatable units that can be shared across multiple applications simultaneously, resolving the contradiction between guaranteeing processing capabilities and avoiding resource waste.
Solution Approach 2:
The patent implements dynamic resource allocation where the GPU device can transition between exclusive and shareable modes based on application requirements. The system dynamically determines whether to allocate the entire device exclusively or to share it among multiple containers, allowing flexible adaptation to different workload scenarios and optimizing resource utilization while maintaining reliability.
2Productivity
If a device is shared among multiple applications, then resource utilization increases, but resource allocation complexity increases
Solution Approach 1:
The patent introduces a device plugin as an intermediary layer between the GPU device and the Kubernetes scheduler. The device plugin translates complex device sharing requirements into standard Kubernetes resource allocation mechanisms, managing the shareable resource information and allocation logic while presenting a simplified interface to the scheduler. This intermediary handles the complexity of resource sharing, allowing the scheduler to operate with standard resource allocation procedures.
Solution Approach 2:
The patent creates a universal resource allocation mechanism that handles both exclusive and shareable device allocation through a unified interface. The shareable resource information structure and allocation logic can manage multiple types of resources (GPU, FPGA, TPU) and multiple allocation modes (exclusive, shareable) using the same framework, reducing overall system complexity while enabling flexible resource utilization.
3Ease of manufacture
If the Kubernetes scheduling mechanism uses device-level allocation, then implementation is simple, but resource flexibility is reduced when applications need only partial device capabilities
Solution Approach 1:
The patent segments the device-level allocation into finer-grained shareable resource units. Instead of allocating entire devices as atomic units, the system divides devices into allocatable shareable resources that can be independently scheduled. This segmentation maintains implementation simplicity by using standard Kubernetes scheduling mechanisms while enabling flexible partial device allocation to meet diverse application requirements.
Data Source
AI summary
The embodiments of the present disclosure disclose a resource scheduling method, apparatus and system. The method comprises: receiving, by a management node, a Pod creation request from a user, wherein the Pod creation request comprises: requirements of each container for each type of shareable resources; selecting, by the management node, a node for a Pod object to be created, and allocating each type of shareable resources of each container to the Pod object according to shareable resource information of a shareable device in the selected node; and binding the Pod object, the selected node and allocated resources, and storing the Pod object bound to the selected node and the allocated resources.