GPU resource scheduling method and system based on Kubernetes

Abstract

The invention discloses a GPU (Graphics Processing Unit) resource scheduling method and system based on Kubernetes, and belongs to the technical field of cloud native and computing power scheduling, the method is based on a Kubernetes container cluster management technology, and GPU information reporting and statistics and GPU resource scheduling are realized by installing a GPU driver, an nvidia-dockertime plug-in and an nvidia-device-plug-in plug-in in each GPU node in a cluster. According to the method, GPU resource scheduling in a Kubernetes cluster is realized by introducing a Kubernetes cluster management technology and a GPU native plug-in tool. Efficient scheduling and management of GPU resources are achieved through rapid deployment of applications in a Kubernetes cluster, and the requirements for computing resources in a complex application scene are met.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud native and computing power scheduling, and specifically to a GPU resource scheduling method and system based on Kubernetes. Background Art

[0002] With the rapid development of technologies such as cloud computing, artificial intelligence, and big data, as an important computing resource, GPUs have been widely used in fields such as deep learning and high-performance computing. However, when facing GPU resources, traditional computing resource scheduling methods often have problems such as low resource utilization rate and single scheduling strategy. Therefore, how to efficiently schedule and manage GPU resources has become an urgent technical problem to be solved.

[0003] Kubernetes (abbreviated as k8s) is a container cluster management system open-sourced by Google in June 2014 and developed in the Go language. It is used to manage containerized applications on multiple hosts in a cloud platform. The goal of Kubernetes is to make the deployment of containerized applications simple and efficient. Kubernetes provides a complete set of functions such as resource scheduling, deployment management, service discovery, scaling, monitoring, and maintenance, and strives to become a platform for automatically deploying, scaling, and running application containers across host clusters. However, the native scheduler of Kubernetes mainly makes decisions based on static information of resource allocation and may not be able to fully utilize GPU resources, especially when multiple tasks share the same GPU. Summary of the Invention

[0004] The technical task of the present invention is to address the above deficiencies and provide a GPU resource scheduling method and system based on Kubernetes, which can achieve efficient scheduling and management of GPU resources by quickly deploying applications in a Kubernetes cluster and meet the computing resource requirements in complex application scenarios.

[0005] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0006] A GPU resource scheduling method based on Kubernetes. This method is based on Kubernetes container cluster management technology. By installing GPU drivers, nvidia-docker runtime, and nvidia-device-plugin plugins on each GPU node in the cluster, GPU information reporting and statistics and GPU resource scheduling are achieved. The implementation of this method includes the following steps:

[0007] 1) Deploy GPU device plugins on each node of the Kubernetes cluster to read and summarize the GPU device information on that node;

[0008] 2) Report the GPU device information to the Kubernetes API Server through the GPU device plugin, and forward it to the GPU scheduling service by the API Server;

[0009] 3) The GPU scheduling service receives and counts all the reported GPU information, generates the mapping relationship between nodes and GPU devices, and records the detailed information of the GPU devices;

[0010] 4) The user specifies the requests and limits for GPU resources in the configuration file of the Pod.

[0011] 5) The Kubernetes scheduler selects a suitable node for Pod scheduling according to the resource requirements of the Pod and the availability of GPU resources in the cluster, combined with the scheduling policies provided by the GPU scheduling service;

[0012] 6) The GPU scheduling service makes specific allocations for GPU resources according to the selected scheduling policies.

[0013] This method realizes the GPU resource scheduling in the Kubernetes cluster by introducing the Kubernetes cluster management technology and the GPU native plugin tool. This method can install the GPU driver, nvidia-docker runtime and nvidia-device-plugin plugin according to the requirements to build the GPU scheduling system, and realizes the efficient scheduling and management of GPU resources through the rapid deployment of applications in the Kubernetes cluster, meeting the requirements for computing resources in complex application scenarios.

[0014] Further, in step 2), the GPU device plugin sends the GPU information to the Kubernetes API Server through remote procedure call (RPC).

[0015] Further, the detailed information of the GPU device includes the graphics card name, graphics card ID, and video memory size.

[0016] Further, in step 4), request a specific number of GPU resources by specifying resources.limits.nvidia.com / gpu:<quantity>.

[0017] Further, the scheduling policies include the Binpack policy, the Spread policy, and the Exclusive policy.

[0018] Further, the Binpack policy preferentially schedules Pods to display nodes with less remaining GPU resources to reduce resource fragmentation;

[0019] The Spread policy evenly distributes Pods to multiple nodes to fully utilize the GPU resources of each node;

[0020] The Exclusive policy ensures that a Pod can use a certain GPU device alone and avoids interference from other Pods.

[0021] Furthermore, the method further includes installing GPU drivers and CUDA toolkits (for NVIDIA GPUs) on each node of the Kubernetes cluster to ensure the normal operation of the GPU device.

[0022] The present invention also claims protection for a GPU resource scheduling system based on Kubernetes, including:

[0023] A GPU plugin / program deployment module that deploys GPU device plugins on each node of the Kubernetes cluster for reading and summarizing GPU device information on that node;

[0024] A GPU information reporting module that reports the GPU device information to the Kubernetes APIServer through the GPU device plugin, and the API Server forwards it to the GPU scheduling service;

[0025] A GPU node mapping module that the GPU scheduling service receives and statistically analyzes all reported GPU information, generates a mapping relationship between nodes and GPU devices, and records the detailed information of the GPU devices;

[0026] A user configuration module where the user specifies requests and limits for GPU resources in the configuration file of the Pod;

[0027] A scheduling policy module where the Kubernetes scheduler selects a suitable node for Pod scheduling based on the resource requirements of the Pod and the availability of GPU resources in the cluster, in combination with the scheduling policies provided by the GPU scheduling service;

[0028] A GPU scheduling distribution module where the GPU scheduling service makes specific allocations of GPU resources according to the selected scheduling policy;

[0029] The system specifically realizes GPU resource scheduling through the above-mentioned GPU resource scheduling method based on Kubernetes.

[0030] The present invention also claims protection for a GPU resource scheduling device based on Kubernetes, including: at least one memory and at least one processor;

[0031] The at least one memory is used to store machine-readable programs;

[0032] The at least one processor is used to call the machine-readable program to implement the above method.

[0033] The present invention also claims to protect a computer-readable medium, on which computer instructions are stored. When the computer instructions are executed by a processor, the above method is implemented.

[0034] Compared with the prior art, a GPU resource scheduling method and system based on Kubernetes of the present invention have the following beneficial effects:

[0035] The present invention improves resource utilization rate. By introducing GPU native plug-in tools and various scheduling strategies, efficient scheduling and management of GPU resources are realized, and the resource utilization rate is improved. It meets the requirements of complex application scenarios, supports various scheduling strategies, and can be flexibly configured according to different application scenarios and requirements to meet the requirements for computing resources in complex application scenarios. It is easy to expand and maintain. Based on the Kubernetes cluster management system, it has good scalability and maintainability, which is convenient for subsequent function upgrade and optimization. Description of the Drawings

[0036] Figure 1 It is a schematic diagram of the principle of a GPU resource scheduling method based on Kubernetes provided by an embodiment of the present invention. Detailed Embodiments

[0037] The present invention will be further described below with reference to the drawings and specific embodiments.

[0038] A GPU resource scheduling method based on Kubernetes. This method is based on Kubernetes container cluster management technology. By installing GPU drivers, nvidia-dockerruntime, and nvidia-device-plugin plug-ins on each GPU node in the cluster, GPU information reporting and statistics and GPU resource scheduling are realized. The implementation of this method includes the following steps:

[0039] 1. Deploy GPU device plug-ins on each node of the Kubernetes cluster to read and summarize the GPU device information on this node;

[0040] 2. Through the GPU device plug-in, report the GPU device information to the Kubernetes API Server, and the API Server forwards it to the GPU scheduling service;

[0041] 3. The GPU scheduling service receives and statistics all reported GPU information, generates the mapping relationship between nodes and GPU devices, and records the detailed information of GPU devices, including the graphics card name, graphics card ID, video memory size, etc.

[0042] 4. The user specifies the requests and limits for GPU resources in the Pod's configuration file.

[0043] 5. The Kubernetes scheduler selects a suitable node for Pod scheduling based on the resource requirements of the Pod and the availability of GPU resources in the cluster, combined with the scheduling policies provided by the GPU scheduling service.

[0044] 6. The GPU scheduling service makes specific allocations of GPU resources according to the selected scheduling policy.

[0045] The scheduling policies include the Binpack policy, the Spread policy, and the Exclusive policy.

[0046] The Binpack policy preferentially schedules Pods to display nodes with less remaining GPU resources to reduce resource fragmentation.

[0047] The Spread policy evenly distributes Pods to multiple nodes to make full use of the GPU resources of each node.

[0048] The Exclusive policy ensures that a Pod can use a certain GPU device alone and avoids being interfered with by other Pods.

[0049] Among them, in step 2), the GPU device plugin sends GPU information to the Kubernetes API Server through remote procedure call (RPC).

[0050] In step 4), a specific number of GPU resources can be requested by specifying resources.limits.nvidia.com / gpu:<quantity>.

[0051] This method also includes installing GPU drivers and CUDA toolkits (for NVIDIA GPUs) on each node of the Kubernetes cluster to ensure that the GPU devices can work properly.

[0052] This method realizes the scheduling of GPU resources in the Kubernetes cluster by introducing Kubernetes cluster management technology and GPU native plugin tools. This method can install GPU drivers, nvidia-dockerruntime, and nvidia-device-plugin plugins according to requirements to build a GPU scheduling system, and achieve efficient scheduling and management of GPU resources through the rapid deployment of applications in the Kubernetes cluster, meeting the requirements for computing resources in complex application scenarios.

[0053] This method is applicable to GPU devices of different brands and models. Only the corresponding GPU device plugins and drivers need to be installed accordingly.

[0054] An embodiment of the present invention further provides a GPU resource scheduling system based on Kubernetes, including:

[0055] A GPU plugin / program deployment module that deploys GPU device plugins on each node of the Kubernetes cluster to read and summarize GPU device information on that node;

[0056] A GPU information reporting module that reports the GPU device information to the Kubernetes APIServer through the GPU device plugin, and the API Server forwards it to the GPU scheduling service;

[0057] A GPU node mapping module that the GPU scheduling service receives and statistics all reported GPU information, generates a mapping relationship between nodes and GPU devices, and records the detailed information of the GPU devices;

[0058] A user configuration module where the user specifies requests and limits for GPU resources in the configuration file of the Pod;

[0059] A scheduling policy module where the Kubernetes scheduler selects a suitable node for Pod scheduling according to the resource requirements of the Pod and the availability of GPU resources in the cluster, in combination with the scheduling policy provided by the GPU scheduling service;

[0060] A GPU scheduling distribution module where the GPU scheduling service makes specific allocations of GPU resources according to the selected scheduling policy;

[0061] This system specifically realizes GPU resource scheduling through the Kubernetes-based GPU resource scheduling method described in the above embodiments. The process is as follows:

[0062] 1. Install GPU device plugins on each node of the Kubernetes cluster, and this plugin is responsible for reading and summarizing GPU device information on the node.

[0063] 2. The GPU device plugin sends GPU information to the Kubernetes APIServer through remote procedure call (RPC), and the API Server forwards it to the GPU scheduling service.

[0064] 3. The GPU scheduling service receives and statistics all reported GPU information, records the mapping relationship between nodes and graphics cards, and the detailed information of the graphics cards (such as graphics card name, graphics card ID, video memory size, etc.).

[0065] 4. The user specifies the requested and limited GPU resources in the Pod's configuration file. For example, request a specific number of GPU resources by specifying resources.limits.nvidia.com / gpu:<quantity>.

[0066] 5. After receiving the Pod creation request, the Kubernetes scheduler selects a suitable node for scheduling based on the Pod's resource requirements and the availability of GPU resources in the cluster.

[0067] 6. The GPU scheduling service allocates GPU resources according to scheduling policies (such as Binpack policy, Spread policy, Exclusive policy, etc.). The Binpack policy preferentially schedules Pods to the graphics card nodes with less remaining GPU resources to avoid fragmentation; the Spread policy evenly distributes Pods to multiple graphics cards to make full use of the graphics card resources; the Exclusive policy ensures that a Pod can use a certain graphics card alone and avoid being interfered by other Pods.

[0068] An embodiment of the present invention also provides a Kubernetes-based GPU resource scheduling device, including: at least one memory and at least one processor;

[0069] The at least one memory is used to store machine-readable programs;

[0070] The at least one processor is used to call the machine-readable program to implement the Kubernetes-based GPU resource scheduling method described in the above embodiments.

[0071] An embodiment of the present invention also provides a computer-readable medium, on which computer instructions are stored. When the computer instructions are executed by a processor, the Kubernetes-based GPU resource scheduling method described in the above embodiments is implemented. Specifically, a system or device equipped with a storage medium can be provided, on which software program code for implementing the functions of any one of the above embodiments is stored, and the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0072] In this case, the program code read from the storage medium itself can implement the functions of any one of the above embodiments. Therefore, the program code and the storage medium storing the program code constitute a part of the present invention.

[0073] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code can be downloaded from a server computer via a communication network.

[0074] In addition, it should be clear that not only can the functions of any one of the above embodiments be realized by executing the program code read by a computer, but also by causing an operating system or the like operating on the computer to complete some or all of the actual operations based on the instructions of the program code.

[0075] In addition, it can be understood that the program code read from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion unit connected to the computer, and then the CPU or the like installed on the expansion board or the expansion unit is caused to execute some and all of the actual operations based on the instructions of the program code, thereby realizing the functions of any one of the above embodiments.

[0076] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above-mentioned multiple embodiments, those skilled in the art can know that the code review means in different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A GPU resource scheduling method based on Kubernetes, characterized in that This method is based on the Kubernetes container cluster management technology. By installing GPU drivers, nvidia-docker runtime, and nvidia-device-plugin plugins on each GPU node in the cluster, it realizes GPU information reporting, statistics, and GPU resource scheduling. The implementation of this method includes the following steps: 1) Deploy GPU device plugins on each node of the Kubernetes cluster to read and summarize GPU device information on that node; 2) Through the GPU device plugin, report the GPU device information to the Kubernetes API Server, and the API Server forwards it to the GPU scheduling service; 3) The GPU scheduling service receives and statistics all reported GPU information, generates the mapping relationship between nodes and GPU devices, and records the detailed information of GPU devices; 4) The user specifies the requests and limits for GPU resources in the Pod's configuration file; 5) The Kubernetes scheduler selects appropriate nodes for Pod scheduling based on the resource requirements of the Pod and the availability of GPU resources in the cluster, combined with the scheduling policies provided by the GPU scheduling service; 6) The GPU scheduling service makes specific allocations of GPU resources according to the selected scheduling policies.

2. The method for GPU resource scheduling based on Kubernetes according to claim 1, wherein In step 2), the GPU device plugin sends GPU information to the Kubernetes API Server through remote procedure calls.

3. A GPU resource scheduling method based on Kubernetes according to claim 1, characterized in that, The detailed information of the GPU device includes the graphics card name, graphics card ID, and video memory size.

4. A GPU resource scheduling method based on Kubernetes according to claim 1, wherein In step 4), request a specific number of GPU resources by specifying resources.limits.nvidia.com / gpu:<quantity>.

5. A GPU resource scheduling method based on Kubernetes according to claim 1, characterized in that The scheduling policies include the Binpack policy, Spread policy, and Exclusive policy.

6. The GPU resource scheduling method based on Kubernetes according to claim 5, wherein The Binpack policy preferentially schedules Pods to display nodes with less remaining GPU resources to reduce resource fragmentation; The Spread policy evenly distributes Pods to multiple nodes to make full use of the GPU resources of each node; The Exclusive policy ensures that a Pod can use a certain GPU device alone and avoids being interfered by other Pods.

7. A GPU resource scheduling method based on Kubernetes according to claim 1 or 5, characterized in that This method also includes installing GPU driver programs and CUDA toolkits on each node of the Kubernetes cluster to ensure the normal operation of GPU devices..

8. A GPU resource scheduling system based on Kubernetes, characterized in that, Including: GPU plugin / program deployment module, which deploys GPU device plugins on each node of the Kubernetes cluster to read and summarize GPU device information on that node; GPU information reporting module, which reports the GPU device information to the Kubernetes API Server through the GPU device plugin, and the API Server forwards it to the GPU scheduling service; The GPU node mapping module, where the GPU scheduling service receives and counts all reported GPU information, generates the mapping relationship between nodes and GPU devices, and records the detailed information of the GPU devices; The user configuration module, where the user specifies the requests and limits for GPU resources in the configuration file of the Pod; The scheduling policy module, where the Kubernetes scheduler selects appropriate nodes for Pod scheduling based on the resource requirements of the Pod and the availability of GPU resources in the cluster, in combination with the scheduling policies provided by the GPU scheduling service; The GPU scheduling distribution module, where the GPU scheduling service makes specific allocations of GPU resources according to the selected scheduling policy; This system specifically realizes GPU resource scheduling through the Kubernetes-based GPU resource scheduling method described in any one of claims 1 to 7.

9. A GPU resource scheduling device based on Kubernetes, characterized in that, It includes: At least one memory and at least one processor; The at least one memory is used to store machine-readable programs; The at least one processor is used to call the machine-readable program to implement the method described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that, Computer instructions are stored on the computer-readable medium, and when the computer instructions are executed by the processor, the method described in any one of claims 1 to 7 is implemented.

Images