A multi-tenant machine learning task resource scheduling method based on Kubernetes

Abstract

The present invention discloses a multi-tenant machine learning task resource scheduling method based on Kubernetes, performs quota management on the computing power resources available to different users, monitors the resource status information of each Node node in the Kubernetes platform, considers the resource utilization rate of the host machine where the node is located, avoids the problem of inaccurate scheduling results, and monitors real-time scheduling and pre-scheduling request demand information at the same time, prioritizes each Node node according to the scheduling task demand information, obtains the host label of the optimal node, and reasonably allocates the resource requirements of various machine learning model training and prediction tasks according to the label. The present invention effectively prevents and reduces the tilt problem of node resource use in the Kubernetes platform, realizes multi-node load balancing, and improves the utilization rate of node resources.

Description

Technical Field

[0001] The present invention relates to a multi-tenant machine learning task resource scheduling method based on Kubernetes, and belongs to the technical field of power regulation. Background Art

[0002] At present, the application of artificial intelligence technology in the field of power grid control has achieved initial results, but in terms of computing power resource management, it has encountered the problem of dispersed computing power, which restricts the breakthrough of application. Various applications are deployed in an "chimney-style" artificial intelligence development and operation environment, resulting in duplicate construction of underlying hardware resources, dispersed computing power and difficulty in expansion.

[0003] The IaaS layer of the cloud computing platform mainly uses virtualization technology to achieve multi-tenant resource isolation and dynamic allocation. However, traditional virtualization technology itself has a high hardware resource occupancy rate and is not suitable for scenarios with high computing resource utilization such as machine learning model training and prediction tasks. In addition, the application configuration, operation, and management aspects are relatively complex, which is not conducive to clustered coordinated management.

[0004] Kubernetes has the ability to automatically orchestrate, deploy and schedule resources for services and is very popular among developers. The present invention performs customized orchestration and scheduling of resources based on Kubernetes, supports the development of artificial intelligence applications and service support platforms in a new generation of scheduling technical support systems, and is used for resource scheduling of machine learning training and prediction tasks such as power grid fault identification and analysis, power grid operation prediction and analysis, and power grid intelligent scheduling and auxiliary decision-making. Its application results verify the technical route and reliability of the present invention. Summary of the invention

[0005] Objective: To overcome the deficiencies in the prior art, the present invention provides a multi-tenant machine learning task resource scheduling method based on Kubernetes, which uses Kubernetes and container technology to uniformly regulate the CPU, GPU and memory resources of the IaaS layer, build a standardized operating environment for multi-tenant machine learning model training and prediction applications, and improve the controllability, elastic expansion capability and resource isolation capability of the power grid control system.

[0006] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A multi-tenant machine learning task resource scheduling method based on Kubernetes includes the following steps:

[0008] The difference between the resources used by the Node in the cluster and the resources used by the created containers is calculated to obtain the resource information occupied by all processes of the Node operating system itself.

[0009] Call the Kubernetes API to obtain resource information requested by all machine learning model training and prediction task containers on the Node.

[0010] The real-time available resource information of the Node node is calculated by subtracting the resource information occupied by all processes of the Node node's operating system and the resource information requested by all machine learning model training and prediction task containers on the Node node from the Node node's inherent resource capacity.

[0011] Calculate the availability of the Node's CPU, GPU, and memory based on the Node's real-time available resource information and the Node's inherent resource capacity.

[0012] The system cluster resource management service presets the resource threshold percentage. Node nodes whose CPU, GPU, and memory availability is not less than the preset resource threshold percentage are allocated computing resources for machine learning model training and prediction tasks.

[0013] The machine learning task scheduling service sends the number of CPU, GPU, and memory resources requested by different users' machine learning model training and prediction tasks to the system cluster resource management service.

[0014] The system cluster resource management service calculates the resource difference between the multi-tenant resource quota table and the user resource usage table to obtain the remaining resources that users can apply for, and verifies whether the number of CPUs, GPUs, and memories requested by machine learning model training and prediction tasks exceeds the remaining resources that users can apply for.

[0015] Select a Node whose remaining resources do not exceed the amount that the user can apply for. The system cluster resource management service calculates the difference between the real-time available resource information of the Node and the requested CPU, GPU, and memory quantity, and divides it by the inherent resource capacity of the Node to obtain the percentage of CPU, GPU, and memory resources remaining after the resources are allocated.

[0016] Select the Node nodes whose percentage of CPU, GPU and memory resources remaining after resources are allocated is greater than the preset resource threshold percentage, calculate the percentage of CPU, GPU and memory resources remaining after resources are allocated for each Node node, and sort them from large to small according to the score.

[0017] The system cluster resource management service selects the Node node that is ranked first in the sequence as the optimal node, returns the node name of the optimal node to the machine learning task scheduling service, and stores it persistently in the user resource usage table.

[0018] The machine learning task scheduling service dynamically generates Kubernetes yaml files and calls the Kubernetes API to create containers in the optimal nodes to run machine learning model training and prediction tasks.

[0019] As a preferred solution, a CPU, GPU, and memory usage collection program is deployed on each Kubernetes Node in the cluster.

[0020] As a preferred solution, the inherent resource capacity of the Node nodes in the cluster uses the user ID as the namespace in Kubernetes to logically divide and isolate the virtual resource pool.

[0021] As a preferred solution, the multi-tenant resource quota table is as follows:

[0022]

[0023]

[0024] As a preferred solution, the user resource usage table is as follows:

[0025]

[0026] As a preferred solution, Kubernetes' role-based access control is used to grant access permissions to namespaces that different users can operate.

[0027] As a preferred solution, the Kubernetes cluster includes the following components: API Server, ControllerManager, Scheduler, Kubelet, Kube-proxy, Etcd, and Container runtime.

[0028] As a preferred solution, the method for calculating the score based on the percentage of CPU, GPU and memory resources remaining after each Node node is allocated resources is as follows:

[0029] Score i =request_cpu×percent_cpu i +request_gpu×percent_gpu i +request_mem×percent_mem i

[0030] Among them, Score i is the score of the i-th Node, percent_cpu i 、percent_gpu i、percent_mem i They are the percentages of CPU, GPU and memory resources remaining after the i-th Node node is allocated resources. request_cpu, request_gpu and request_mem are the amounts of CPU, GPU and memory requested by the i-th Node node respectively.

[0031] Beneficial effect: The present invention provides a multi-tenant machine learning task resource scheduling method based on Kubernetes, which performs quota management on the computing resources available to different users, monitors the resource status information of each Node node in the Kubernetes platform, considers the resource utilization rate of the host machine where the node is located, and avoids the problem of inaccurate scheduling results. At the same time, by monitoring the real-time scheduling and pre-scheduling request demand information, each Node node is prioritized according to the scheduling task demand information, and the host label of the optimal node is obtained. According to the label, the resource requirements of various machine learning model training and prediction tasks are reasonably allocated, which effectively prevents and reduces the problem of node resource usage tilt in the Kubernetes platform, realizes multi-node load balancing, and improves the utilization rate of node resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of multi-tenant management of cluster resources in an example of the present invention.

[0033] Figure 2 It is a schematic diagram of the Kubernetes cluster resource management architecture in the example of the present invention.

[0034] Figure 3 It is a flowchart of machine learning training and prediction task creation in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described below in conjunction with specific embodiments.

[0036] A multi-tenant machine learning task resource scheduling method based on Kubernetes includes the following steps:

[0037] 1) By calculating the used resources of the Node node (node_cpu_used i 、node_gpu_used i and node_mem_used i ) and the resources used by the created container (pod_cpu_used i , pod_gpu_used i and pod_mem_used i) to obtain the resource information occupied by all processes of the node's operating system.

[0038] 2) Obtain resource information (pod_cpu_req) requested by all machine learning model training and prediction task containers on the node by calling the Kubernetes API i , pod_gpu_req i and pod_mem_req i ).

[0039] 3) Set the Node's inherent resource capacity (node_cpu_total i 、node_gpu_total i and node_mem_total i ) minus the above two values, you can calculate the real-time available resource information of the Node node (node_cpu i 、node_gpu i and node_mem i ).

[0040] 4) Calculate the availability of CPU, GPU and memory of each Node using the following formula:

[0041] percent_cpu i =node_cpu i node_cpu_total i

[0042] percent_gpu i =node_gpu i node_gpu_total i

[0043] percent_mem i =node_mem i node_mem_total i

[0044] 5) The system cluster resource management service presets the resource threshold percentage. Nodes below the preset resource threshold percentage will no longer allocate computing resources for machine learning model training and prediction tasks, ensuring that each Node node will not be overloaded.

[0045] 6) The machine learning task scheduling service sends the number of CPU, GPU, and memory resources (request_cpu, request_gpu, and request_mem) required for different users' machine learning model training and prediction tasks to the system cluster resource management service.

[0046] 7) The system cluster resource management service calculates the resource difference between the multi-tenant resource quota table and the user resource usage table to obtain the remaining resources that the user can apply for, and verifies whether the number of request_cpu, request_gpu, and request_mem requested by the machine learning model training and prediction tasks exceeds the remaining resources that the user can apply for.

[0047] resource.

[0048] 8) The system cluster resource management service will provide the real-time available resource information of the Node node (node_cpu i 、node_gpu i and node_mem i ) and the requested request_cpu, request_gpu and request_mem

[0049] Calculate the difference in quantity and divide it by the inherent resource capacity of the Node to get the percentage of resources left after allocating resources.

[0050] percentage.

[0051] percent_cpu i =(node_cpu i -request_cpu) / node_cpu_total i

[0052] percent_gpu i =(node_gpu i -request_gpu) / node_gpu_total i

[0053] percent_mem i =(node_mem i -request_mem) / node_mem_total i

[0054] Compare the nodes whose remaining resource percentage after allocating the above resources is less than the preset resource threshold percentage

[0055] After filtering, the remaining resource percentages of the remaining nodes are scored and sorted by score.

[0056] Score i =request_cpu×percent_cpu i +request_gpu×percent_gpu i+request_mem×percent_mem i

[0057] 9) The system cluster resource management service selects the Node with the highest score from the sorting as the optimal node, returns the node name to the machine learning task scheduling service, and records it in the user resource usage table.

[0058] Persistent storage.

[0059] 10) The machine learning task scheduling service dynamically generates Kubernetes yaml files and calls the Kubernetes API to create containers in the optimal node to run machine learning model training and prediction tasks.

[0060] The purpose of the present invention is to use Kubernetes and container technology to uniformly control the CPU, GPU, memory and storage resources of the IaaS layer, build a standardized operating environment for multi-tenant machine learning model training and prediction applications, and improve the controllability, elastic expansion capability and resource isolation capability of the power grid system. The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention:

[0061] According to the available CPU, GPU, memory and storage resource information of different Node nodes in the cluster, label management is performed, cluster resources are integrated into a resource pool through Kubernetes, and user ID is used as the namespace in Kubernetes to logically divide and isolate the virtual resource pool.

[0062] Figure 1 shown.

[0063] The system administrator allocates the required resource information to different users through the cluster multi-tenant resource management interface tool. This information is persistently stored in the multi-tenant resource quota table, as shown in Table 1.

[0064] Role-based access control (RBAC) grants access rights to namespaces that different users can operate, preventing resource usage between users from interfering with each other.

[0065] Deploy CPU, GPU and memory usage collection programs on each Kubernetes Node in the cluster, and calculate the available resources and availability of all Node nodes based on the above information.

[0066] Table 1 Multi-tenant resource quota table

[0067] Field Name use user_id Unique user ID consistent with permissions and single sign-on cpu_capacity Total number of CPU logical cores memory_capacity Total memory (GB) gpu_capacity Number of GPU cards storage_capacity Storage space (GB)

[0068] Table 2 User resource usage

[0069]

[0070] like Figure 2 The figure shows a schematic diagram of the Kubernetes cluster resource management architecture of this embodiment. In this embodiment, the Kubernetes cluster consists of 2 Master nodes and 6 Node nodes. The Master node is the main control unit of the cluster, which mainly schedules and manages the cluster to prevent the increase of project demand and the increase of access volume. Therefore, this embodiment constructs a high-availability mode of dual Master nodes; the Node node is a workload node, which is mainly used to run containers of business applications, including two clusters, CPU and GPU. The CPU cluster is mainly used to create conventional pod tasks, while the GPU cluster is mainly used to create pod tasks involving image computing. The dual Node cluster mode will make the applications deployed therein run more reasonably and efficiently.

[0071] The Kubernetes cluster mainly includes seven main components: API Server, Controller Manager, Scheduler, Kubelet, Kube-proxy, Etcd, and Container runtime. The above components work together to realize the operation of the entire cluster. The scheduling strategy of the present invention mainly works on the Scheduler, by calculating the evaluation scores of real-time tasks and scheduled tasks on each Node node. The evaluation score includes two aspects. On the one hand, it refers to the actual usage of the resources of the Node node itself, and on the other hand, it takes into account the preference of the pod for CPU, GPU, and memory resource requirements. Finally, the scheduling strategy of the present invention conducts a comprehensive evaluation of each Node node based on real-time tasks and scheduled tasks, selects the Node node with the highest evaluation score as the target scheduling node, skips the Scheduler's pre-selection strategy and optimization strategy, and can directly create a pod on the specified Node node by setting a unique label. Figure 3 The flowchart of creating a pod task request in this embodiment is shown below. The specific method is as follows:

[0072] Step 1: Obtain the CPU, GPU, and memory usage information of the host machine where each Node node in the Kubernetes platform is located, as well as the CPU, GPU, and memory usage information and request allocation information of the pod on the node, and calculate the available resources and availability of each Node node based on the above information;

[0073] First, the difference between the host and pod resource usage is calculated to obtain the host resource usage outside the pod container. Second, the actual resource allocation of the pod container is obtained, and the actual availability of the Node node is calculated by summing the host usage outside the container. The actual available resources of the CPU, GPU, and memory of all Node nodes are calculated using the following formula:

[0074] node_cpu i =node_cpu_total i -(host_cpu_used i -pod_cpu_used i )-pod_cpu_req i

[0075] node_mem i =node_mem_total i -(host_mem_used i -pod_mem_used i )-pod_mem_req i

[0076] node_gpu i =node_gpu_total i -(host_gpu_used i -pod_gpu_used i )-pod_gpu_req i

[0077] Among them, the node_cpu i 、node_mem i and node_gpu i Corresponding to the actual available resource information of Node CPU, GPU and memory, the node_cpu_total i 、node_mem_total i and node_gpu_total i The total resource configuration information corresponding to the Node node CPU, GPU and memory, host_cpu_used i 、host_mem_used i and host_gpu_used i Corresponding to the host machine usage information of Node node CPU, GPU and memory, host_cpu_used i , pod_mem_usedi

[0078] and host_gpu_used i Corresponding to the pod usage information of the Node node CPU, GPU and memory, the pod_cpu_req i , pod_mem_req i and pod_gpu_req i Resource request allocation information of pods corresponding to the CPU, GPU, and memory of the current Node;

[0079] The availability of CPU, GPU and memory of each Node is calculated using the following formula:

[0080] percent_cpu i =node_cpu i node_cpu_total i

[0081] percent_mem i =node_mem i node_mem_total i

[0082] percent_gpu i =node_gpu i node_gpu_total i

[0083] Step 2: Compare the CPU, GPU and memory availability of each Node with the preset threshold. If a node is lower than the specified threshold, it indicates that the node is overloaded. Filter the node. If the number of filtered nodes is 0, return scheduling failure; if the number of filtered nodes is greater than 0, proceed to step 3;

[0084] Step 3: Obtain the request information of the real-time task and scheduled task pod for CPU, GPU and memory resources through the K8s scheduler, which are request_cpu, request_gpu, request_mem and user ID respectively. Look up the table based on the user ID to obtain the remaining information of the current user resources. By comparing, it can be determined whether to support the continued creation of pods. If not, the scheduling failure is returned. If satisfied, proceed to the next step.

[0085] Step 4: Compare the task resource request information obtained in step 3 with the available resources of the Node node, and filter out the Node nodes with insufficient CPU, GPU, and memory resources. If the number of filtered nodes is 0, a scheduling failure is returned. If the number of filtered nodes is equal to 1, the Node node is set as the host of the pod to be created. If the number of filtered nodes is greater than 1, proceed to the next step.

[0086] Step 5: Score the filtered Node nodes and calculate the percentage of resources remaining after the requested task is allocated to each Node node using the following formula.

[0087] percent_cpu i =(node_cpu i -request_cpu) / node_cpu_total i

[0088] percent_gpu i =(node_gpu i -request_gpu) / node_gpu_total i

[0089] percent_mem i =(node_mem i -request_mem) / node_mem_total i

[0090] The nodes whose remaining resource percentage after the above resources are allocated is less than the remaining resource threshold percentage are excluded, and the remaining resource percentages of the CPU, GPU and memory of all nodes after the resources are allocated are accumulated and sorted.

[0091] Prioritize each Node node and determine the optimal number of nodes based on the ranking. If the number of nodes is 1, the Node node is the optimal node and its label is obtained. If the number of nodes is greater than 1, the optimal Node node is selected based on the ranking and its label is obtained. Finally, the pod is started by specifying the label in the yaml file of the machine learning task.

[0092] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-tenant machine learning task resource scheduling method based on Kubernetes, characterized by: The steps include: Calculate the difference between the resources used by the Node nodes in the cluster and the resources used by the created containers to obtain the resource information occupied by all processes of the Node node operating system itself; Call the Kubernetes API to obtain resource information requested by all machine learning model training and prediction task containers on the Node node; The real-time available resource information of the Node node is calculated by subtracting the resource information occupied by all processes of the Node node's operating system and the resource information requested by all machine learning model training and prediction task containers on the Node node from the Node node's inherent resource capacity. Calculate the availability of the Node's CPU, GPU, and memory based on the Node's real-time available resource information and the Node's inherent resource capacity; The system cluster resource control service presets the resource threshold percentage. The Node nodes whose CPU, GPU and memory availability is not less than the preset resource threshold percentage are allocated computing resources for machine learning model training and prediction tasks. The machine learning task scheduling service sends the number of CPU, GPU, and memory resources requested by different users' machine learning model training and prediction tasks to the system cluster resource management service; The system cluster resource control service calculates the resource difference between the multi-tenant resource quota table and the user resource usage table to obtain the remaining resources that users can apply for, and verifies whether the number of CPUs, GPUs, and memories requested by machine learning model training and prediction tasks exceeds the remaining resources that users can apply for. Select a Node whose remaining resources do not exceed the amount that the user can apply for. The system cluster resource management service calculates the difference between the real-time available resource information of the Node and the requested CPU, GPU, and memory quantity, and divides it by the inherent resource capacity of the Node to obtain the percentage of CPU, GPU, and memory resources remaining after the resources are allocated. Select the Node nodes whose percentage of CPU, GPU and memory resources remaining after resources are allocated is greater than the preset resource threshold percentage, calculate the percentage of CPU, GPU and memory resources remaining after resources are allocated for each Node node, and sort them from large to small according to the score; The system cluster resource management service selects the Node node that is ranked first in the sequence as the optimal node, returns the node name of the optimal node to the machine learning task scheduling service, and stores it persistently in the user resource usage table; The machine learning task scheduling service dynamically generates Kubernetes yaml files and calls the Kubernetes API to create containers in the optimal nodes to run machine learning model training and prediction tasks.

2. According to a Kubernetes-based multi-tenant machine learning task resource scheduling method according to claim 1, it is characterized in that: Deploy CPU, GPU, and memory usage collection programs on each Kubernetes Node in the cluster.

3. According to a Kubernetes-based multi-tenant machine learning task resource scheduling method according to claim 1, it is characterized in that: The inherent resource capacity of the Node nodes in the cluster uses the user ID as the namespace in Kubernetes to logically divide and isolate the virtual resource pool.

4. According to a Kubernetes-based multi-tenant machine learning task resource scheduling method according to claim 1, it is characterized in that: The multi-tenant resource quota table is as follows:

5. According to a Kubernetes-based multi-tenant machine learning task resource scheduling method according to claim 1, it is characterized in that: The user resource usage table is as follows:

6. According to a Kubernetes-based multi-tenant machine learning task resource scheduling method according to claim 1, it is characterized in that: Use Kubernetes' role-based access control to grant access permissions to namespaces that different users can operate.

7. According to a Kubernetes-based multi-tenant machine learning task resource scheduling method according to claim 1, it is characterized in that: A Kubernetes cluster consists of the following components: API Server, Controller Manager, Scheduler, Kubelet, Kube-proxy, Etcd, Container runtime.

8. According to a Kubernetes-based multi-tenant machine learning task resource scheduling method according to claim 1, it is characterized in that: The remaining resources of CPU, GPU and memory after each Node node is allocated resources The percentage scoring method is as follows: Score i =request_cpu×percent_cpu i +request_gpu×percent_gpu i +request_mem×percent_mem i Among them, Score i is the score of the i-th Node, percent_cpu i 、percent_gpu i , percent_mem i The remaining resources of CPU, GPU and memory after allocating resources to the i-th Node node The percentage of the source, request_cpu, request_gpu, and request_mem are the i-th Node section The number of CPUs, GPUs, and memory requested by the node.

Images