GPU virtualization system based on kubernetes
Through the GPU virtualization system based on kubernetes, GPU resources are dynamically scheduled and managed, and the problems of low utilization rate, high cost and poor flexibility in the existing technology are solved, and efficient, secure and flexible GPU resource management and use are achieved.
Patent Information
- Application Number
- CN202411607118.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-04-25
AI Technical Summary
The existing GPU resource utilization rate is low, resource idle, fixed allocation, lack of flexibility, difficulty in scaling, high hardware costs, high operation and maintenance costs, poor security and isolation, and complex resource competition and development and testing environments.
The GPU virtualization system based on kubernetes is adopted, including the Kubernets cluster module, GPUManager module, Device Plugin module, Pod module, kube-scheduler module and Kubernets module. Through the collaborative work of these modules, dynamic resource scheduling, multi-user parallel processing, resource isolation and management are realized.
It improves GPU resource utilization, reduces hardware and operation and maintenance costs, enhances system flexibility and scalability, simplifies management and operation and maintenance, and ensures the security and isolation of resources.
Smart Images
Figure BDA0005129885840000041 
Figure BDA0005129885840000051 
Figure HDA0005337888840000011
Abstract
Description
Technical Field
[0001] The present invention relates to a virtualization system, and more specifically to a GPU virtualization system based on Kubernetes. Background Art
[0002] GPU is a hardware unit specially designed for processing graphics computing and parallel computing tasks. Initially, GPU was mainly used for graphics rendering, but as computing needs increased, GPU gradually expanded to the field of general computing. The main uses include graphics rendering, parallel computing, deep learning and artificial intelligence, image and video processing, data processing and analysis, hardware acceleration and many other aspects. GPU is expensive and has powerful computing power, but there are some disadvantages in actual use:
[0003] Low resource utilization and idle resources: The physical GPU may be idle at some time, especially when the tasks are not continuous or intensive, resulting in resource waste. Fixed allocation: The resource allocation is fixed and cannot be dynamically adjusted according to real-time needs, resulting in some GPU resources not being fully utilized.
[0004] Lack of flexibility, single-user use: A physical GPU can usually only be used exclusively by one user or one application, and cannot achieve parallel processing of multiple users or tasks.
[0005] Difficulty in expansion: As workloads increase, expanding GPU resources requires physically adding more GPUs, which is a complex and costly process.
[0006] Hardware costs: A physical GPU needs to be configured separately for each user or each application, which increases hardware procurement costs.
[0007] Operation and maintenance costs: Physical GPUs have high maintenance and management costs, including power consumption, cooling requirements, and hardware replacement.
[0008] Poor security and isolation: When multiple users or applications share the same physical GPU, data security and privacy issues may arise due to the lack of isolation and security brought by virtualization.
[0009] Resource competition,When multiple tasks are executed in parallel, resource contention may occur,,leading to unstable or degraded performance.
[0010] Development and test environments. Creating and managing multiple physical GPU environments for development and testing is complex and expensive, and is not as flexible and efficient as virtualized environments. Summary of the invention
[0011] In view of the shortcomings of the prior art, the object of the present invention is to provide a GPU virtualization system based on Kubernetes that can effectively overcome the above-mentioned shortcomings.
[0012] To achieve the above object, the present invention provides the following technical solution: a GPU virtualization system based on Kubernetes, characterized in that it includes:
[0013] The Kubernetes cluster module communicates with the GPU virtualization cluster and is used to schedule the resources of the GPU virtualization cluster.
[0014] The GPUManager module communicates with the Kubernetes cluster module and the GPU virtualization cluster to limit the use of GPU and video memory by the Kubernetes cluster module.
[0015] As a further improvement of the present invention, the Kubernetes cluster module includes:
[0016] Device Plugin module, which communicates with the GPU virtualization cluster and is used to call the GPU;
[0017] Pod module, used as a container for mapped devices;
[0018] The kube-scheduler module is used to bind the appropriate node to the Pod module;
[0019] The Kubernets module is used to monitor the Pod module bound to the current node and allocate devices to the Pod module. As a further improvement of the present invention, the specific steps of the Device Plugin module calling the GPU are as follows: Step 1, the Deviceplugin module reports the current node resource status to the Kubernets module, and the user creates a load through a yaml file to define the GPU resource request;
[0020] Step 2: The kube-scheduler module binds the appropriate node to the Pod based on the resource information synchronized from the kubelet and the resource request of the Pod module;
[0021] Step 3: The Kubernetes module listens to the Pod module bound to the current node and calls the allocate interface of the Device plugin module to allocate devices for the Pod module.
[0022] The Kubernetes module starts the container in the Pod module and maps the device to the container.
[0023] As a further improvement of the present invention, the GPUManager module includes a cuda encapsulation library vcuda, a deviceplugin plug-in gpu-manager-daemonset and a Kubernetes scheduling plug-in gpu-quota-admission, wherein the vcuda library is an encapsulation library of the nvidia-ml and libcuda libraries, which limits the use of the GPU and video memory by the process in the current container by hijacking the cuda call of the user program in the container; gpu-manager-daemonset is a standard Kubernetesdevice plugin, which realizes the functions of GPU topology perception, device and driver mapping;
[0024] gpu-quota-admission is a Kubernetes Scheduler extender that implements the Scheduler's predicates interface. When kube-scheduler schedules a Pod that requests resources, the predicates phase calls the gpu-quota-admission's predicates interface to filter and bind nodes. At the same time, gpu-quota-admission provides GPU resource pool scheduling capabilities.
[0025] Compared with the traditional GPU usage scheme, the present invention has the following beneficial effects:
[0026] 1. Improve resource utilization, avoid idle GPU resources, and improve overall system efficiency;
[0027] 2. Reduce costs. Shared GPU solutions can reduce hardware requirements.
[0028] 3. Enhance flexibility and scalability;
[0029] 4. Simplify management and operation and maintenance;
[0030] 5. GPU resources can be managed and scheduled uniformly through centralized management and monitoring tools, thereby simplifying operation and maintenance work and improving management efficiency;
[0031] 6. You can select and deploy computing resources more flexibly to meet different business needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 A module block diagram of a GPU virtualization system based on Kubernetes of the present invention;
[0033] Figure 2 Schematic diagram of the process of calling GPU for user program. DETAILED DESCRIPTION
[0034] The present invention will be further described below in conjunction with the embodiments given in the accompanying drawings, wherein the terms used in the embodiments are explained as follows:
[0035]
[0036]
[0037] Reference Figures 1 to 2 As shown, a GPU virtualization system based on Kubernetes in this embodiment is characterized by comprising:
[0038] The Kubernetes cluster module communicates with the GPU virtualization cluster and is used to schedule the resources of the GPU virtualization cluster.
[0039] The GPUManager module is connected to the Kubernetes cluster module and the GPU virtualization cluster, and is used to limit the use of GPU and video memory by the Kubernetes cluster module. By setting the GPUManager module, the use of GPU and video memory can be effectively restricted, thereby achieving a better resource allocation effect.
[0040] Specifically, the Kubernetes cluster module of this embodiment includes:
[0041] Device Plugin module, which communicates with the GPU virtualization cluster and is used to call the GPU;
[0042] Pod module, used as a container for mapped devices;
[0043] The kube-scheduler module is used to bind the appropriate node to the Pod module;
[0044] The Kubernetes module is used to monitor the Pod module bound to the current node and allocate devices to the Pod module. By setting the above modules, the GPU can be effectively called through the following steps:
[0045] Step 1: The Device plugin module reports the current node resource status to the Kubernetes module. The user creates a load through a YAML file and defines the GPU resource request.
[0046] Step 2: The kube-scheduler module binds the appropriate node to the Pod based on the resource information synchronized from the kubelet and the resource request of the Pod module;
[0047] Step 3: The Kubernetes module listens to the Pod module bound to the current node and calls the allocate interface of the Device plugin module to allocate devices for the Pod module.
[0048] The Kubernetes module starts the container in the Pod module and maps the device to the container.
[0049] The Nvidia implementation of the Device Plugin module only supports mapping the entire card to the container. Multiple containers cannot share the same card, which often results in a large amount of resource waste. Based on the above problems, the wrap library and Linux dynamic library linking technology are used to achieve the upper limit isolation of GPU computing power and video memory.
[0050] The corresponding GPUManager module consists of three parts: the cuda encapsulation library vcuda, the device plugin gpu-manager-daemonset, and the Kubernetes scheduling plugin gpu-quota-admission. The vcuda library is a encapsulation library of the nvidia-ml and libcuda libraries. It restricts the use of the GPU and video memory by the current process in the container by hijacking the cuda calls of the user program in the container.
[0051] gpu-manager-daemonset is a standard Kubernetes device plugin that implements GPU topology awareness, device and driver mapping.
[0052] gpu-quota-admission is a Kubernetes Scheduler extender that implements the Scheduler's predicates interface. When kube-scheduler schedules a Pod that requests resources, the predicates phase calls the gpu-quota-admission's predicates interface to filter and bind nodes. At the same time, gpu-quota-admission provides GPU resource pool scheduling capabilities.
[0053] In this way, GPU information is collected through the GPU Manager module, and cuda is hijacked when the GPU is called to limit resource usage.
[0054] In summary, the Kubernetes-based GPU virtualization system of this embodiment uses the GPUManager module to limit the use of GPU and video memory by the Kubernetes cluster module, which can effectively improve resource utilization, avoid idle GPU resources, and improve the efficiency of the overall system.
[0055] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
Claims
1. A GPU virtualization system based on Kubernetes, characterized by: include: The Kubernetes cluster module communicates with the GPU virtualization cluster and is used to schedule the resources of the GPU virtualization cluster. The GPUManager module communicates with the Kubernetes cluster module and the GPU virtualization cluster to limit the use of GPU and video memory by the Kubernetes cluster module.
2. The Kubernetes-based GPU virtualization system according to claim 1, characterized in that: The Kubernets cluster module includes: Device Plugin module, which communicates with the GPU virtualization cluster and is used to call the GPU; Pod module, used as a container for mapped devices; The kube-scheduler module is used to bind the appropriate node to the Pod module; Kubernetes module, used to listen to the Pod module bound to the current node and allocate devices to the Pod module.
3. The Kubernetes-based GPU virtualization system according to claim 2, characterized in that: The specific steps of the Device Plugin module calling the GPU are as follows: Step 1: The Device plugin module reports the current node resource status to the Kubernetes module. The user creates a load through a YAML file and defines the GPU resource request. Step 2: The kube-scheduler module binds the appropriate node to the Pod based on the resource information synchronized from the kubelet and the resource request of the Pod module; Step 3: The Kubernetes module listens to the Pod module bound to the current node and calls the allocate interface of the Device plugin module to allocate devices for the Pod module. The Kubernetes module starts the container in the Pod module and maps the device to the container.
4. The Kubernetes-based GPU virtualization system according to claim 3, characterized in that: The GPUManager module includes the cuda encapsulation library vcuda, the device plugin gpu-manager-daemonset and the Kubernetes scheduling plugin gpu-quota-admission, where: The vcuda library is a wrapper library for the nvidia-ml and libcuda libraries. It restricts the use of GPU and video memory by the current process in the container by hijacking the cuda calls of the user program in the container. gpu-manager-daemonset is a standard Kubernetes device plugin that implements GPU topology awareness, device and driver mapping. gpu-quota-admission is a Kubernetes Scheduler extender that implements the Scheduler's predicates interface. When kube-scheduler schedules a Pod that requests resources, the predicates phase calls the gpu-quota-admission's predicates interface to filter and bind nodes. At the same time, gpu-quota-admission provides GPU resource pool scheduling capabilities.