A container scheduling method and device for a shared GPU cluster
By sorting Pods in Kubernetes clusters for business priority and resource requirements, calculating node scores, matching the optimal node for scheduling, the problem of low GPU resource utilization in traditional Kubernetes clusters is solved, and more efficient resource utilization and loading is achieved balanced.
Patent Information
- Application Number
- CN202210535352.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-17
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-05-17
AI Technical Summary
In distributed computing clusters, traditional Kubernetes clusters fail to make rational use of GPU resources, resulting in unreasonable scheduling of diversified computing services, low GPU resource utilization, affecting cluster performance.
A container scheduling method for a shared GPU cluster is adopted to obtain resource information in the Pod creation request, sort the Pods according to business priorities and resource requirements, calculate the node scores, match the optimal nodes for scheduling, and ensure that GPUs are not required. The pods of resources are allocated to nodes with insufficient GPU resources to improve resource utilization.
It improves task processing efficiency and resource utilization, avoids unbalanced CPU and memory consumption, realizes load balancing, and improves the overall utilization rate of GPU resources.
Smart Images

Figure CN114968566B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a container scheduling method and device for a shared GPU cluster, belonging to the technical field of cluster scheduling. Background Art
[0002] The lower-level nodes of the distributed computing cluster Kubernetes (hereinafter referred to as k8s) are mainly composed of GPU (microprocessors for image computing, which are important computing engines like CPU) servers, combined with CPU, storage, etc. to provide computing resources for tasks such as deep learning. As artificial intelligence and engineering computing platforms are more widely used in various industries, their demand for GPU computing resources is also gradually increasing.
[0003] The container scheduling of distributed clusters needs to allocate and schedule containers based on current business indicator requirements and resource pool status. Traditional K8s clusters rely on the number of GPUs rather than the granularity of using GPUs to achieve sharing, resulting in poor adaptation performance of distributed cluster scheduling for complex and diverse computing scenarios and resource requirements. In related technologies, the most suitable node is selected for container scheduling based on the needs of scheduling containers, and the scheduling queue and GPU resource scoring are not reasonable, resulting in reduced task allocation efficiency and low GPU resource utilization in the entire distributed cluster, affecting cluster performance.
[0004] In order to solve the problems of unreasonable scheduling and low GPU resource utilization in diversified computing services, the present invention proposes a container scheduling method for a shared GPU cluster. Summary of the invention
[0005] Purpose: In order to overcome the problems of unreasonable scheduling and low GPU resource utilization in the diversified computing services in the prior art, the present invention provides a container scheduling method and device for a shared GPU cluster.
[0006] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:
[0007] In a first aspect, a container scheduling method for a shared GPU cluster includes the following steps:
[0008] Get the Pod creation request, verify the Pod according to the Pod creation information in the Pod creation request to obtain the Pod tag, and obtain the total required resource information of the Pod according to the Pod creation information.
[0009] Sort the Pods according to the Pod business priority label and the total required resource information of the Pod in the Pod creation information, and obtain the Pod to be scheduled at the head of the scheduling queue.
[0010] Filter the nodes according to the CPU label, memory label, and GPU label required by the Pod in the Pod creation information of the Pod to be scheduled, the Pod tag, and the cluster node status information to obtain the Pod schedulable nodes.
[0011] According to the number of Pod schedulable nodes and the Pod tag of the Pod to be scheduled, the score of the Pod corresponding to the schedulable node is calculated, and the Pod to be scheduled is matched with the optimal Pod schedulable node based on the score.
[0012] In a second aspect, a container scheduling device for a shared GPU cluster includes the following modules:
[0013] The Pod tag and resource information acquisition module is used to obtain a Pod creation request, verify the Pod according to the Pod creation information in the Pod creation request to obtain the Pod tag, and obtain the total required resource information of the Pod according to the Pod creation information.
[0014] The Pod sorting module is used to sort the Pods according to the Pod business priority label and the total required resource information of the Pod in the Pod creation information, and obtain the Pod to be scheduled at the head of the scheduling queue.
[0015] The Pod schedulable node acquisition module is used to filter nodes according to the CPU label, memory label and GPU label required by the Pod in the Pod creation information of the Pod to be scheduled, the Pod mark and the cluster node status information, and obtain the Pod schedulable nodes.
[0016] The best node acquisition module is used to calculate the score of the Pod corresponding to the schedulable node according to the number of Pod schedulable nodes and the Pod tag of the Pod to be scheduled, and match the Pod to be scheduled with the best Pod schedulable node according to the score.
[0017] As a preferred solution, the Pod creation request is obtained, the Pod is verified according to the Pod creation information in the Pod creation request to obtain a Pod tag, and the total required resource information of the Pod is obtained according to the Pod creation information, including:
[0018] Get the GPU label of the Pod creation information in the Pod creation request. If the Pod does not contain the GPU label, the Pod is marked as a Pod that does not require GPU resources; if the Pod contains the GPU label, the Pod is marked as a Pod that requires GPU resources.
[0019] Based on the resource information required for each container in the Pod creation information, similar resources in all containers are accumulated to obtain the total resource information required by the Pod.
[0020] As a preferred solution, the total required resource information of the Pod includes: the amount of GUP resource application, the amount of CPU resource application, the amount of memory resource application, and the amount of GPU video memory application.
[0021] As a preferred solution, sorting the Pods according to the Pod service priority label and the total required resource information of the Pod in the Pod creation information to obtain the Pod to be scheduled at the head of the scheduling queue includes:
[0022] Sort the Pods from high to low according to the Pod service priority label value.
[0023] If the difference in the Pod service priority label values between m Pods among n Pods to be scheduled is less than the preset threshold, m < n, screen the amount of GPU resource application for the m Pods, and give priority to sorting the Pod with the smaller amount of GPU resource application.
[0024] If the amount of GPU resource application is the same, screen the amount of CPU resource application, and give priority to sorting the Pod with the smaller amount of CPU resource application.
[0025] If the amount of CPU resource application is the same, screen the amount of memory resource application, and give priority to sorting the Pod with the smaller amount of memory resource application.
[0026] If the amounts of GPU, CPU, and memory resource applications are all the same, do not change the sorting order.
[0027] The Pod with the highest sorting is the Pod to be scheduled at the head of the scheduling queue.
[0028] As a preferred solution, filtering the nodes according to the CPU label, memory label, and GPU label required by the Pod, the Pod label, and the cluster node status information in the Pod creation information of the Pod to be scheduled to obtain the Pod schedulable nodes includes:
[0029] Obtain the CPU label, memory label, and GPU label required by the Pod to be scheduled.
[0030] Obtain the status of all nodes in the current cluster, and obtain the idle node resource information. The idle node resource information includes: the CPU clock frequency, CPU usage rate, available memory, number of GPU cores, GPU clock frequency, and GPU usage rate held by the node.
[0031] Traverse all idle nodes in the cluster. When the CPU clock frequency held by the node is greater than the CPU clock frequency label value of the Pod to be scheduled, mark the node as a schedulable node, and mark the node schedulable label value as 1; otherwise, mark it as 0.
[0032] Traverse all nodes whose schedulable node label value is 1. When the available memory value held by the node is greater than the memory label value of the Pod to be scheduled, the schedulable label value of the node is continuously marked as 1, otherwise it is marked as 0.
[0033] If the Pod to be scheduled is not a Pod that requires GPU resources, all nodes with the schedulable node label value of 1 will be used as Pod schedulable nodes.
[0034] If the Pod to be scheduled is a Pod that requires GPU resources, all nodes with the schedulable node label value of 1 are traversed. When the number of GPU cores held by the node is greater than the GPU quantity label value, and the total value of the node GPU available memory is greater than the GPU memory label value, the node is considered a Pod schedulable node.
[0035] As a preferred solution, it also includes:
[0036] Check whether the requested volumes of all Pod-schedulable nodes conflict with the volumes used by other Pods on the node. If there is a conflict, filter the Pod-schedulable node.
[0037] As a preferred solution, the score of the Pod corresponding to the schedulable node is calculated according to the number of Pod schedulable nodes and the Pod tag of the Pod to be scheduled, and the Pod to be scheduled is matched with the optimal Pod schedulable node according to the score, including:
[0038] When the number of Pod schedulable nodes is equal to 1, the schedulable node is used as the best node.
[0039] When the number of Pod schedulable nodes is greater than 1, if the Pod to be scheduled is marked as a Pod that does not require GPU resources, calculate the score Score1 of the schedulable node corresponding to the Pod that does not require GPU resources, select the node with the highest score as the best node, and bind the Pod to be scheduled to the best node.
[0040] The calculation formula for the score Score1 of the schedulable node corresponding to the non-demand GPU resource Pod is as follows:
[0041]
[0042] in, The weight of the first score for CPU and memory resource balancing, is the weight of the GPU idle rate score, The CPU and memory resource balance score is the first. GPU idle rate score.
[0043] Said
[0044]
[0045] in, Indicates the CPU resource application amount of the Pod to be scheduled. Represents the total amount of remaining idle CPUs in the schedulable nodes. Indicates the memory resource application amount of the Pod to be scheduled. It represents the sum of the remaining free memory resources in the schedulable nodes. abs(*) represents the absolute value.
[0046] Said The calculation formula is as follows:
[0047] (2)
[0048] in, Represents the total amount of free GPU memory remaining in the schedulable nodes. Indicates the maximum total amount of GPU memory in the schedulable node.
[0049] When the number of Pod schedulable nodes is greater than 1, if the Pod to be scheduled is marked as a Pod requiring GPU resources, the score Score2 of the schedulable node corresponding to the Pod requiring GPU resources is calculated, the node with the highest score is selected as the best node, and the Pod to be scheduled is bound to the best node.
[0050] The calculation formula for the score Score2 of the Pod that requires GPU resources and corresponds to the schedulable node is as follows:
[0051]
[0052] in, is the weight of the GPU score, The weight of the Pod quota score. is the weight of the second score of CPU and memory resources, The second score is for CPU and memory resource balance.
[0053] Said The calculation formula is as follows:
[0054]
[0055] in, It is expressed as the sum of the scores of each GPU in the schedulable node, Indicates the GPU clock frequency of the schedulable node, Indicates the maximum clock frequency of all GPUs in the schedulable node that meet the Pod request to be scheduled. Indicates the GPU power of the schedulable node, Indicates the maximum power of all GPUs in the schedulable node that meet the Pod request to be scheduled. Indicates the free video memory of each GPU in the schedulable node, Indicates the total amount of free video memory in the schedulable nodes. Indicates the total amount of GPU memory of the schedulable nodes. Indicates the total amount of memory in the schedulable nodes that can satisfy the requests of the Pods to be scheduled.
[0056] Said The calculation formula is as follows:
[0057]
[0058] in, Indicates the total amount of GPU memory requested by the Pods to be scheduled.
[0059] Said The calculation formula is as follows:
[0060]
[0061] in, 2 is the second score of CPU and memory resource balance, It is expressed as the CPU idleness of the schedulable node. It is represented as the memory idleness of the schedulable node. It is expressed as the maximum capacity of CPU resources of schedulable nodes. Indicates the total amount of CPU requested by the Pod to be scheduled. It is represented by the maximum capacity of the memory resources of the schedulable nodes. Indicates the total amount of memory requested by the Pod to be scheduled.
[0062] As a preferred solution, if there are multiple best nodes, a best node is randomly selected to bind to the Pod to be scheduled.
[0063] As a preferred solution, the score of the Pod corresponding to the schedulable node is calculated according to the number of Pod schedulable nodes and the Pod tag of the Pod to be scheduled, and the Pod to be scheduled is matched with the optimal Pod schedulable node according to the score, including:
[0064] When the number of nodes that can be scheduled by the pod is 0, the scheduling failure time and the corresponding Pod information to be scheduled are written into the event log, and the head Pod of the scheduling queue is reselected, and the Pod that failed to be scheduled is put back into the queue to be scheduled.
[0065] Beneficial effects: The present invention provides a container scheduling method and device for a shared GPU cluster. When monitoring Pod creation events, screening and inspecting required resources in a distributed cluster, labels are added according to task attributes. Subsequent node scoring uses different scoring standards according to label attributes, and appropriate nodes are allocated according to the final results, so that Pods that do not require GPU resources can be preferentially allocated to nodes that lack GPU resources, thereby improving the utilization rate of each node resource and achieving load balancing. Its advantages are as follows:
[0066] 1. To address the scheduling queue arrangement problem, a resource application threshold is introduced in the business priority arrangement for secondary sorting to prevent GPU, CPU and other resources on cluster nodes from being occupied for a long time by Pods with high preset priorities, thereby preventing Pods with low business priorities from waiting for a long time, and improving task processing efficiency and utilization of resources within the node.
[0067] 2. After the Pod is created and enters the scheduling queue for scheduling, the same scoring mechanism is used to allocate nodes for all Pods. In a shared GPU cluster, Pods that do not require GPU resources are prioritized on nodes with idle CPU and memory resources and fewer idle GPU resources, so that other Pods that require GPUs can better select the best node, taking into account load balancing, improving resource utilization, and avoiding uneven CPU and memory consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] Figure 1 This is a system topology diagram of the scheduling method in the present invention.
[0069] Figure 2 It is a flowchart of the steps of the scheduling method in the present invention.
[0070] Figure 3 This is a scheduling queue update flow chart of the scheduling method in the present invention.
[0071] Figure 4 The flowchart of the node filtering and selection operation of the scheduling method in the present invention. DETAILED DESCRIPTION
[0072] The present invention will be further described below in conjunction with specific embodiments.
[0073] The present invention adopts the following technical scheme to achieve the above-mentioned object of the invention.
[0074] like Figure 1As shown, the system for executing the scheduling method of the present invention includes: etcd, Master API-Server, listener, scheduler and executor, etcd is a highly available key-value storage system for storing all data that needs to be persisted. Master API-Server is the master node in the cloud in K8s. The listener is used to listen to Pod creation events. The scheduler is used to carry the scheduling function of cluster resources and schedule the Pod to the specified node according to a specific scheduling algorithm and strategy. The executor is used to execute the Pod bound to the node and run it on the bound node.
[0075] like Figure 2 As shown, a container scheduling method for a shared GPU cluster includes the following steps:
[0076] (1) Obtain a Pod creation request, verify the Pod according to the Pod creation information in the Pod creation request to obtain the Pod tag, and obtain the total required resource information of the Pod according to the Pod creation information.
[0077] (2) Sort the Pods according to the Pod business priority label and the total resource information required by the Pod in the Pod creation information, and obtain the Pod to be scheduled at the head of the scheduling queue.
[0078] (3) Filter the nodes based on the CPU label, memory label, and GPU label required by the Pod in the Pod creation information of the Pod to be scheduled, the Pod tag, and the cluster node status information to obtain the Pod schedulable nodes.
[0079] (4) Based on the number of Pod schedulable nodes and the Pod tag of the Pod to be scheduled, the score of the Pod corresponding to the schedulable node is calculated, and the Pod to be scheduled is matched with the optimal Pod schedulable node based on the score.
[0080] Preferably, the specific steps of step (1) are as follows:
[0081] (1-1): The Master API-Server monitors the Pod creation requests in the server in real time. Users can send requests to the Master API-Server through kubectl (command line) to create the required Pod. Before creation, users can create a Yaml file according to task requirements and configure the Pod creation information in the Yaml file. The Pod creation information includes: Pod image name, Pod scheduling task priority label, startup command, startup parameters, CPU label, memory label and GPU label required by the Pod, among which the GPU label includes GPU quantity label, GPU video memory label and GPU clock frequency label; when a new Pod creation request is monitored, the Pod creation information is obtained through the Yaml file, and the Pod is verified according to the Pod creation information to obtain the Pod tag. According to the Pod creation information, the total required resource information of the Pod is obtained, and the Pod tag and the total required resource information of the Pod are stored, which is used as a basis for scoring subsequent nodes.
[0082] (1-2): Verify the created Pod and obtain the total resource information required by the Pod. The specific process is as follows:
[0083] Determine whether the Pod contains the GPU label: If the Pod does not contain the GPU label, it means that the environment required by the Pod does not need GPU resources, and the Pod is marked as a Pod that does not require GPU resources; if the Pod contains the GPU label, it means that the environment required by the Pod requires GPU resources, and the Pod is marked as a Pod that requires GPU resources.
[0084] According to the Pod creation information, the resource information required for the operation of each container contained in it is obtained, such as the number of CPUs, running memory, number of GPUs, GPU video memory, and GPU clock frequency. The same resources in all containers in the Pod are accumulated to obtain the total resource information required for this Pod, including the GPU resource application amount, CPU resource application amount, memory resource application amount, and GPU video memory application amount.
[0085] (1-3): Store the verification results and the total resource information required by the Pod, providing a basis for subsequent node scoring and node allocation.
[0086] like Figure 3 As shown, preferably, the specific steps of step (2) are as follows:
[0087] (2-1) Send the Pod verified in step (1) to the scheduling queue, generate a new scheduling queue, and update the scheduling queue.
[0088] (2-2) Get the current queue information, get the newly sent Pod business priority label, and sort from high to low according to the value of each Pod business priority label in the current scheduling queue. If there are m Pods among the n Pods to be scheduled whose Pod business priority label value difference is less than the preset threshold, then further sort according to the total required resource information of the Pod, including the size of the GPU, CPU, and memory resource application amount: filter the GPU resource application amount, and prioritize the Pod with the least GPU resource application amount; when the GPU resource application amounts are the same, filter the CPU resource application amount, and prioritize the Pod with the least CPU resource application amount; when the CPU resource application amounts are the same, filter the memory resource application amount, and prioritize the Pod with the least memory resource application amount; if the GPU, CPU, and memory resource application amounts are the same, do not change the sorting order. This prevents Pods with high business priority from constantly entering the queue and lining up at the head of the queue, causing low-priority Pods to wait for a long time, causing the cluster GPU resources to be occupied by Pods with too high business priority for a long time, affecting the load balancing of resources within the node, thereby improving task processing efficiency and the utilization of resources within the node.
[0089] like Figure 4 As shown, the specific steps of step (3) are as follows:
[0090] (3-1) Monitor the scheduling queue in real time to see if it is empty. If the scheduling queue is not empty, read the Pod to be scheduled at the head of the queue, send the Pod to the scheduler, and obtain the CPU label, memory label, and GPU label required by the Pod.
[0091] (3-2) Initiate a cluster node status request to the Master API-Server to obtain the status of all nodes in the current Kubernetes cluster and obtain idle node resource information, including: the CPU clock frequency, CPU usage, available memory, number of GPU cores, GPU clock frequency, and GPU usage (i.e., the total amount of GPU memory and the total amount of available memory) held by the node.
[0092] (3-3) Filter nodes based on the label information carried by the Pod to be scheduled and the cluster node status information to obtain nodes that meet the requirements for running the Pod to be scheduled. This includes the following four steps:
[0093] S301: Node screening based on CPU clock frequency label: Traverse all idle nodes in the cluster. When the CPU clock frequency held by the node is greater than the CPU clock frequency label value of the Pod to be scheduled, the node is marked as a schedulable node, and the node schedulable label value is marked as 1, otherwise it is marked as 0;
[0094] S302: Filter nodes according to memory labels based on S301: traverse all nodes with schedulable node label values of 1 after S301 filtering. When the available memory value held by the node is greater than the running memory label value of the Pod to be scheduled, the schedulable label value of the node is continuously marked as 1, otherwise it is marked as 0.
[0095] S303: Based on S302, perform node screening according to the verification result, GPU quantity label value and GPU memory label value: If the Pod to be scheduled is a Pod that does not require GPU resources, all nodes whose label value of schedulable nodes after filtering in S302 is still 1 are used as Pod schedulable nodes; If the Pod to be scheduled is a Pod that requires GPU resources, traverse all nodes whose label value of schedulable nodes after filtering in S302 is still 1, and when the number of GPU cores held by the node is greater than the GPU quantity label value, and the node's available GPU memory value is greater than the GPU memory label value, the node is used as a Pod schedulable node.
[0096] S304: For all Pod schedulable nodes filtered out based on S303, use NoDiskConflict to check whether the Pod requested volume conflicts with the volumes used by other Pods on the node. If there is a conflict, filter the Pod schedulable node.
[0097] The specific steps of step (4) are as follows:
[0098] (4-1) According to the number of Pod schedulable nodes after screening in step (3-3), further select Pod schedulable nodes.
[0099] S401: When the number of pod schedulable nodes is 0, the scheduling failure time and the corresponding Pod information are written into the event log, and the head Pod of the queue to be scheduled is reselected and the Pod that failed to be scheduled is put back into the queue to be scheduled, and steps (3-1) to (3-3) are repeated to filter out the schedulable nodes.
[0100] S402: When the number of Pod schedulable nodes is equal to 1, the schedulable node is used as the best node.
[0101] S403: When the number of Pod schedulable nodes is greater than 1, the score of each schedulable node is calculated based on the verification result and the total required resource information of the Pod, and the node with the highest score is selected as the best node.
[0102] (4-2) If the node with the highest score in step S403 is not unique, a node with the highest score is randomly selected as the best node, and the Pod to be scheduled is bound to the best node. This allows Pods that do not require GPU resources to be preferentially allocated to nodes that lack GPU resources, thereby improving the utilization of each node's resources and achieving load balancing; and repeat the above steps until the queue to be scheduled is empty.
[0103] The schedulable nodes screened out in step S403 are scored, the node with the highest score is selected as the best node, and the Pod is bound to the best node to complete the Pod scheduling.
[0104] The scoring specifically considers the cluster load balancing situation. The node score is mainly based on whether the Pod to be scheduled uses shared GPU resources, the node GPU score, and the CPU and memory scores. The specific steps are as follows:
[0105] The current schedulable nodes are scored according to the Pod category marked in S101. If it is marked as a Pod that does not require GPU resources, the node does not need GPU resources. The scoring of the schedulable nodes is mainly divided into two parts: 1. CPU and memory resource balance score; 2. GPU idle rate score; If it is marked as a Pod that requires GPU resources, the node requires shared GPU resources. The scoring of the schedulable nodes is mainly divided into three parts: 1. GPU score within the node; 2. Pod quota score; 3. Node CPU and memory resource score.
[0106] S411: In a shared GPU cluster, the GPU utilization rate will be higher than other clusters, while the CPU and memory utilization rates will be relatively low; Pods that do not require GPU resources will be placed first on nodes with idle CPU and memory resources and fewer idle GPU resources, so that other Pods that need GPUs can better choose the best node, which not only takes into account load balancing, but also improves resource utilization and avoids uneven CPU and memory consumption. Its score is mainly divided into two parts: 1. The first CPU and memory resource balance score; 2. The GPU idle rate score.
[0107] The calculation formula for the first score of CPU and memory resource balance is as follows:
[0108] (1)
[0109] in, Indicates the CPU resource application amount of the Pod to be scheduled. Represents the total amount of remaining idle CPUs in the schedulable nodes. Indicates the memory resource application amount of the Pod to be scheduled. It represents the sum of the remaining free memory resources in the schedulable nodes. abs(*) represents the absolute value.
[0110] The GPU idle rate score calculation formula is as follows:
[0111] (2)
[0112] in, Represents the total amount of free GPU memory remaining in the schedulable nodes. Indicates the maximum total amount of GPU memory in the schedulable node.
[0113] The calculation formula for the score Score1 of the schedulable node corresponding to the Pod that meets the non-demand GPU resources is as follows:
[0114] (3)
[0115] in, The weight of the first score for CPU and memory resource balancing. The default value is 2. The weight of the GPU idle rate score. The default value is 1.
[0116] S412: The score of the schedulable node corresponding to the Pod that needs GPU resources is mainly divided into three parts: 1. The GPU score in the node; 2. The Pod quota score; 3. The second score of CPU and memory resource balance.
[0117] The calculation formula for the GPU score within a node is as follows:
[0118] (4)
[0119] in, It is expressed as the sum of the scores of each GPU in the schedulable node, Indicates the GPU clock frequency of the schedulable node, Indicates the maximum clock frequency of all GPUs in the schedulable node that meet the Pod request to be scheduled. Indicates the GPU power of the schedulable node, Indicates the maximum power of all GPUs in the schedulable node that meet the Pod request to be scheduled. Indicates the free video memory of each GPU in the schedulable node, Indicates the total amount of free video memory in the schedulable nodes. Indicates the GPU memory application amount of the schedulable node. Indicates the total amount of memory in the schedulable nodes that can satisfy the requests of the Pods to be scheduled.
[0120] The calculation formula for the Pod quota score is as follows:
[0121] (5)
[0122] in, Indicates the GPU memory application amount of the Pod to be scheduled.
[0123] The calculation formula for the second score of node CPU and memory resource balance is as follows:
[0124] (6)
[0125] in, 2 is the second score of CPU and memory resource balance, It is expressed as the CPU idleness of the schedulable node. It is represented as the memory idleness of the schedulable node. It is expressed as the maximum capacity of CPU resources of schedulable nodes. Indicates the total amount of CPU requested by the Pod to be scheduled. It is represented by the maximum capacity of the memory resources of the schedulable nodes. Indicates the total amount of memory resources requested by the Pod to be scheduled.
[0126] The calculation formula for the score Score2 of the schedulable node corresponding to the Pod that requires GPU resources is as follows:
[0127] (7)
[0128] in, is the weight of the GPU score, and its default value is 2. The weight of the Pod quota score, the default value is 1. The weight of the second score of CPU and memory resources. The default value is 1.
[0129] Implementation Example 1:
[0130] A queue to be scheduled is constructed, which contains tasks to be scheduled Pod1-Pod4, and the values of their Pod scheduling task priority labels are Pod1: 10000; Pod2: 9500; Pod3: 14000; Pod4: 9200; the preset threshold is 1000; add the scheduling task Pod5 that has completed verification to the end of the queue, and the value of its Pod scheduling task priority label is 12500.
[0131] Step 1: Get the scheduling queue information, get the Pod scheduling task priority label value of Pod5, and sort them from high to low according to the preset Pod scheduling task priority label value of each Pod in the current scheduling queue. The order is Pod3→Pod5→Pod1→Pod2→Pod4.
[0132] Step 2: Get the preset threshold of 1000, get the secondary ranking scheduling tasks Pod1, Pod2, and Pod4, and perform further sorting based on their verification results and the recorded resources requested. See Table 1 for the resource application status of Pod1, Pod2, and Pod4:
[0133] Pod Label GPU Resources CPU Resources Memory resources Pod1 1m 2m 2Gi Pod2 0.5m 2m 512Mi Pod4 1m 1m 1Gi
[0134] Table 1 shows the total amount of Pod resources to be scheduled
[0135] Step 3: Re-sort the Pods according to the resource application of each Pod in Table 1: From Table 1, we can see that Pod2 has the smallest GPU resource application, and Pod1 and Pod4 have the same application amount, so Pod2 is ranked before Pod1 and Pod4; comparing the CPU resource application of Pod1 and Pod4, we can see that Pod4 has the smallest CPU resource application amount, so Pod4 is ranked before Pod1. Based on the secondary sorting, the latest business priority ranking is: Pod3→Pod5→Pod2→Pod4→Pod1.
[0136] Implementation Example 2:
[0137] A queue to be scheduled is constructed, which contains tasks to be scheduled Pod1-Pod4, and the values of their Pod scheduling task priority labels are Pod1: 10000; Pod2: 9500; Pod3: 14000; Pod4: 9200; the preset threshold is 1000; add the successfully verified scheduling task Pod5 to the end of the queue, and the value of its Pod scheduling task priority label is 12500.
[0138] Step 1: Get the scheduling queue information, get the Pod scheduling task priority label value of Pod5, and sort the Pods in the current scheduling queue from high to low according to the preset Pod scheduling task priority label value of each Pod. The order is Pod3→Pod5→Pod1→Pod2→Pod4.
[0139] Step 2: Get the preset threshold of 1000, get the secondary arrangement scheduling tasks Pod1, Pod2, and Pod4, and further sort them according to the resources applied for by their verification process records. See Table 2 for the application resource status of Pod1, Pod2, and Pod4:
[0140] Pod Label GPU Resources CPU Resources Memory resources Pod1 1m 2m 2Gi Pod2 0.5m 1m 1Gi Pod4 0.5m 1m 512Mi
[0141] Table 2 shows the total amount of Pod resources to be scheduled
[0142] Step 3: Re-sort the Pods according to the resource applications of each Pod in Table 2: Table 2 shows that Pod1 has the largest GPU resource application, and Pod2 and Pod4 have the same application, so Pod2 and Pod4 are ranked before Pod1; Comparing the CPU resource applications of Pod2 and Pod4, we know that their CPU application applications are the same; Comparing the memory resource applications of Pod2 and Pod4, we know that Pod4 has the smallest memory resource application, so Pod4 is ranked before Pod2. The latest business priority ranking based on the secondary sorting is: Pod3→Pod5→Pod4→Pod2→Pod1.
[0143] Implementation Example 3:
[0144] In a shared GPU cluster, apply for a Pod that does not require GPU resources according to the preset yaml file.
[0145] Step 1: The Master API-Server listens to the Pod creation request, obtains all the information about the Pod creation, verifies the Pod and stores the required resource information. The verification finds that the Pod does not contain the GPU tag, adds an attribute tag to indicate whether GPU resources are required and sets it to 0, indicating that the Pod does not need GPU resources.
[0146] Step 2: Send the Pods that have been successfully verified in Step 1 to the scheduling queue and perform secondary sorting based on the preset Pod scheduling task priority labels and thresholds (refer to Implementation Examples 1 and 2 for sorting).
[0147] Step 3: Monitor the scheduling queue in real time to see if it is empty. When the scheduling queue is not empty, read the Pod to be scheduled at the head of the queue and send the Pod to the scheduler; obtain the status of all nodes in the current Kubernetes cluster and the idle node resource information, and filter out nodes that do not meet the requirements of Pod operation based on the sent Pod label information and cluster node status information.
[0148] Step 4: When the number of schedulable nodes for the container is 0, the scheduling failure time and the corresponding Pod information are written to the event log, and the Pod at the head of the queue to be scheduled is reselected and the Pod that failed to be scheduled is put back into the queue to be scheduled; when the number of schedulable nodes for the container is equal to 1, the schedulable node is used as the best node; when the number of schedulable nodes for the container is greater than 1, the score of each schedulable node is calculated based on the Pod label information and node resource conditions.
[0149] Step 5: Based on the label added in step 1, we know that this Pod does not require GPU resources, so we use the following formula to score each node:
[0150]
[0151] Step 6: Based on the scores in step 5, select the node with the highest score as the best node. If the node with the highest score is not unique, randomly select a node as the best node and bind the Pod to be scheduled to the best node.
[0152] Implementation Example 4:
[0153] In a shared GPU cluster, apply for a Pod that requires GPU resources according to the preset yaml file.
[0154] Step 1: The Master API-Server listens to the Pod creation request, obtains all the information about the Pod creation, verifies the Pod and stores the required resource information; after verification, it is found that the Pod contains a GPU tag, and an attribute tag is added to indicate whether GPU resources are required and set to 1, indicating that this Pod requires GPU resources.
[0155] Step 2: Send the Pods that have been successfully verified in Step 1 to the scheduling queue and perform secondary sorting based on the preset Pod scheduling task priority labels and thresholds (refer to Implementation Examples 1 and 2 for sorting).
[0156] Step 3: Monitor the scheduling queue in real time to see if it is empty. When the scheduling queue is not empty, read the Pod to be scheduled at the head of the queue and send the Pod to the scheduler; obtain the status of all nodes in the current Kubernetes cluster and the idle node resource information, and filter out nodes that do not meet the requirements of Pod operation based on the sent Pod label information and cluster node status information.
[0157] Step 4: When the number of schedulable nodes for the container is 0, the scheduling failure time and the corresponding Pod information are written to the event log, and the Pod at the head of the queue to be scheduled is reselected and the Pod that failed to be scheduled is put back into the queue to be scheduled; when the number of schedulable nodes for the container is equal to 1, the schedulable node is used as the best node; when the number of schedulable nodes for the container is greater than 1, the score of each schedulable node is calculated based on the Pod label information and node resource conditions.
[0158] Step 5: Based on the label added in step 1, we know that this Pod requires GPU resources, so we use the following formula to score each node:
[0159] 2
[0160] Step 6: Based on the scores in step 5, select the node with the highest score as the best node. If the node with the highest score is not unique, randomly select a node as the best node and bind the Pod to be scheduled to the best node.
[0161] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0162] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0163] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0165] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A container scheduling method for a shared GPU cluster. Features: The steps include: Get the Pod creation request, verify the Pod according to the Pod creation information in the Pod creation request to obtain the Pod tag, and obtain the total required resource information of the Pod according to the Pod creation information; Sort the Pods according to the Pod business priority label and the total required resource information of the Pod in the Pod creation information, and obtain the Pod to be scheduled at the head of the scheduling queue; Filter the nodes according to the CPU label, memory label, and GPU label required by the Pod in the Pod creation information of the Pod to be scheduled, the Pod tag, and the cluster node status information to obtain the Pod schedulable nodes; According to the number of Pod schedulable nodes and the Pod tag of the Pod to be scheduled, the score of the Pod corresponding to the Pod is calculated, and the Pod to be scheduled is matched with the optimal Pod schedulable node based on the score; The method of calculating the score of the Pod corresponding to the schedulable node according to the number of Pod schedulable nodes and the Pod tag of the Pod to be scheduled, and matching the Pod to be scheduled with the optimal Pod schedulable node according to the score includes: When the number of Pod schedulable nodes is equal to 1, the schedulable node is used as the best node; When the number of Pod schedulable nodes is greater than 1, if the Pod to be scheduled is marked as a Pod that does not require GPU resources, calculate the score Score 1 of the schedulable node corresponding to the Pod that does not require GPU resources, select the node with the highest score as the best node, and bind the Pod to be scheduled to the best node; The calculation formula for the score Score1 of the schedulable node corresponding to the non-demand GPU resource Pod is as follows: Score1=CpuMemoryScore1×CpuMemoryWeight1+GpuFreeScore ×GpuFreeWeight Among them, CpuMemoryWeight1 is the weight of the first score of CPU and memory resource balance, GpuFreeWeight is the weight of the GPU idle rate score, CpuMemoryScore1 is the first score of CPU and memory resource balance, and GpuFreeScore is the GPU idle rate score; The CpuMemoryScore1 calculation formula is as follows: RequestedCPUSum indicates the amount of CPU resources requested by the Pod to be scheduled, NodeFreeCpuSum indicates the sum of the total amount of free CPU resources remaining in the schedulable nodes, RequestedMemorySum indicates the amount of memory resources requested by the Pod to be scheduled, NodeFreeMemorySum indicates the sum of the remaining free memory resources in the schedulable nodes, and abs(*) indicates taking the absolute value. The GpuFreeScore calculation formula is as follows: Among them, NodeFreeGpuMemory represents the sum of the total amount of free GPU memory in the schedulable nodes, and NodeMaxGpuMemory represents the maximum value of the total amount of all GPU memory in the schedulable nodes; When the number of nodes that can be scheduled by a Pod is greater than 1, if the Pod to be scheduled is marked as a Pod that requires GPU resources, Calculate the score Score2 of the schedulable node corresponding to the GPU resource Pod in need, select the node with the highest score as the best node, and bind the Pod to be scheduled to the best node; The calculation formula for the score Score2 of the Pod that requires GPU resources and corresponds to the schedulable node is as follows: Score2=GpuScore×GpuWeight+NodeGPUSocre×NodeGpuWeight +CpuMemoryScore2×CpuMemoryWeight2 Among them, GpuWeight is the weight of the GPU score, NodeGpuWeight is the weight of the Pod quota score, CpuMemoryWeight2 is the weight of the second score of CPU and memory resources, GpuScore is the GPU score in the node, NodeGPUSocre is the Pod quota score, and CpuMemoryScore2 is the second score of CPU and memory resource balance; The GpuScore calculation formula is as follows: Among them, GpuScore is represented by the sum of the scores of each GPU in the schedulable node, GpuDomain is the clock frequency of the GPU in the schedulable node, and PerMaxGpuDomain is the number of Pods in the schedulable node that meet the requirements of the Pods to be scheduled. The maximum clock frequency of all GPUs requested, GpuPower indicates the GPU power of the schedulable node, PerMaxGpuPower indicates the maximum power of all GPUs in the schedulable node that meet the Pod request to be scheduled, PerFreeGpuMemory indicates the free video memory of each GPU in the schedulable node, NodeFreeGpuMemory indicates the total amount of free video memory in the schedulable node, TotalGpuMemory Indicates the total amount of GPU memory in the schedulable node. NodeMaxGpuMemory indicates the total amount of memory in the schedulable node that satisfies the Pod request to be scheduled. The NodeGPUSocre calculation formula is as follows: Among them, PodGpuMemorySum represents the total amount of PodGPU memory requests to be scheduled; The CpuMemoryScore2 calculation formula is as follows: Among them, CpuMemoryScore2 is the second score for the balance of CPU and memory resources. "cpu" represents the CPU idle rate of schedulable nodes, "memory" represents the memory idle rate of schedulable nodes, "NodeMaxCpuSum" represents the maximum capacity of CPU resources of schedulable nodes, "RequestedCPUSum" represents the total CPU application of pods to be scheduled, "NodeMaxMemorySum" represents the maximum capacity of memory resources of schedulable nodes, and "RequestedMemorySum" represents the total memory application of pods to be scheduled.
2. A container scheduling method for a shared GPU cluster according to claim 1, characterized in that: the obtaining of the Pod creation request, the verification of the Pod according to the Pod creation information in the Pod creation request to obtain the Pod label, and the obtaining of the total required resource information of the Pod according to the Pod creation information, including: obtaining the GPU label of the Pod creation information in the Pod creation request. If the Pod does not contain a GPU label, the Pod is marked as a Pod that does not require GPU resources; if the Pod contains a GPU label, the Pod is marked as a Pod that requires GPU resources; accumulating the same type of resources in all containers according to the required resource information for each container operation in the Pod creation information to obtain the total required resource information of the Pod.
3. A container scheduling method for a shared GPU cluster according to claim 2, characterized in that: the total required resource information of the Pod includes: GUP resource application amount, CPU resource application amount, memory resource application amount, and GPU video memory application amount.
4. A container scheduling method for a shared GPU cluster according to claim 1, characterized in that: the sorting of the Pod according to the Pod service priority label and the total required resource information in the Pod creation information to obtain the Pod to be scheduled at the head of the scheduling queue, including: sorting the Pods from high to low according to the Pod service priority label values; if there are m Pods among n Pods to be scheduled with a difference in Pod service priority label values less than a preset threshold, m < n, screening the GPU resource application amounts of the m Pods, and preferentially sorting the Pod with the smaller GPU resource application amount; if the GPU resource application amounts are the same, screening the CPU resource application amounts, and preferentially sorting the Pod with the smaller CPU resource application amount; if the CPU resource application amounts are the same, screening the memory resource application amounts, and preferentially sorting the Pod with the smaller memory resource application amount; if the GPU, CPU, and memory resource application amounts are all the same, the sorting order remains unchanged; the Pod with the highest sorting is the Pod to be scheduled at the head of the scheduling queue.
5. A container scheduling method for a shared GPU cluster according to claim 1, characterized in that: The filtering of nodes according to the CPU label, memory label and GPU label required by the Pod in the Pod creation information of the Pod to be scheduled, the Pod tag and the cluster node status information to obtain the Pod schedulable nodes includes: Get the CPU label, memory label, and GPU label required by the Pod to be scheduled; Get the status of all nodes in the current cluster and get the idle node resource information. The idle node resource information includes: the CPU clock frequency, CPU usage, available memory, number of GPU cores, GPU clock frequency, and GPU usage held by the node; Traverse all idle nodes in the cluster. When the CPU clock frequency of the node is greater than the CPU clock frequency label value of the Pod to be scheduled, the node is marked as a schedulable node, and the node schedulable label value is marked as 1, otherwise it is marked as 0; Traverse all nodes whose schedulable node label value is 1. When the available memory value held by the node is greater than the memory label value of the Pod to be scheduled, the schedulable label value of the node is continuously marked as 1, otherwise it is marked as 0; If the Pod to be scheduled is a Pod that does not require GPU resources, all nodes with the schedulable node label value of 1 are used as Pod schedulable nodes; If the Pod to be scheduled is a Pod that requires GPU resources, all nodes with the schedulable node label value of 1 are traversed. When the number of GPU cores held by the node is greater than the GPU quantity label value, and the total value of the node GPU available memory is greater than the GPU memory label value, the node is considered a Pod schedulable node.
6. According to claim 5, a container scheduling method for a shared GPU cluster, Features: Also includes: Check whether the requested volumes of all Pod-schedulable nodes conflict with the volumes used by other Pods on the node. If there is a conflict, filter the Pod-schedulable node.
7. The method for scheduling containers in a shared GPU cluster according to claim 1, Features: If there are multiple best nodes, a best node is randomly selected and bound to the Pod to be scheduled.
8. The method for scheduling containers in a shared GPU cluster according to claim 1, Features: The method calculates the score of the Pod corresponding to the Pod schedulable node according to the number of Pod schedulable nodes and the Pod tag of the Pod to be scheduled, and matches the Pod to be scheduled with the optimal Pod schedulable node according to the score, including: when the number of Pod schedulable nodes is 0, writing the scheduling failure time and the corresponding Pod to be scheduled information into the event log, reselecting the head Pod of the scheduling queue, and putting the Pod that failed to be scheduled back into the queue to be scheduled.
9. A container scheduling device for a shared GPU cluster. Features: It includes the following modules: a Pod tag and resource information acquisition module, which is used to obtain a Pod creation request, verify the Pod according to the Pod creation information in the Pod creation request to obtain the Pod tag, and obtain the total required resource information of the Pod according to the Pod creation information; The Pod sorting module is used to sort the Pods according to the Pod business priority label and the total required resource information of the Pod in the Pod creation information, and obtain the Pod to be scheduled at the head of the scheduling queue; The Pod schedulable node acquisition module is used to filter nodes according to the CPU label, memory label and GPU label required by the Pod in the Pod creation information of the Pod to be scheduled, the Pod tag and the cluster node status information, and obtain the Pod schedulable nodes; The best node acquisition module is used to calculate the score of the Pod corresponding to the schedulable node according to the number of Pod schedulable nodes and the Pod tag of the Pod to be scheduled, and match the Pod to be scheduled with the best Pod schedulable node according to the score; The method of calculating the score of the Pod corresponding to the schedulable node according to the number of Pod schedulable nodes and the Pod tag of the Pod to be scheduled, and matching the Pod to be scheduled with the optimal Pod schedulable node according to the score includes: When the number of Pod schedulable nodes is equal to 1, the schedulable node is used as the best node; When the number of Pod schedulable nodes is greater than 1, if the Pod to be scheduled is marked as a Pod that does not require GPU resources, calculate the score Score 1 of the schedulable node corresponding to the Pod that does not require GPU resources, select the node with the highest score as the best node, and bind the Pod to be scheduled to the best node; The calculation formula for the score Score1 of the schedulable node corresponding to the non-demand GPU resource Pod is as follows: Score1=CpuMemoryScore1×CpuMemoryWeight1+GpuFreeScore ×GpuFreeWeight Among them, CpuMemoryWeight1 is the weight of the first score of CPU and memory resource balance, GpuFreeWeight is the weight of the GPU idle rate score, CpuMemoryScore1 is the first score of CPU and memory resource balance, and GpuFreeScore is the GPU idle rate score; The CpuMemoryScore1 calculation formula is as follows: RequestedCPUSum indicates the amount of CPU resources requested by the Pod to be scheduled, NodeFreeCpuSum indicates the sum of the remaining free CPU resources in the schedulable nodes, RequestedMemorySum indicates the amount of memory resources requested by the Pod to be scheduled, NodeFreeMemorySum indicates the sum of the remaining free memory resources in the schedulable nodes, and abs(*) indicates the amount of memory resources requested by the Pod to be scheduled. Absolute value; The GpuFreeScore calculation formula is as follows: Among them, NodeFreeGpuMemory represents the sum of the total amount of free GPU memory in the schedulable nodes, and NodeMaxGpuMemory represents the maximum value of the total amount of all GPU memory in the schedulable nodes; When the number of nodes that can be scheduled by a Pod is greater than 1, if the Pod to be scheduled is marked as a Pod that requires GPU resources, Calculate the score Score2 of the schedulable node corresponding to the GPU resource Pod in need, select the node with the highest score as the best node, and bind the Pod to be scheduled to the best node; The calculation formula for the score Score2 of the Pod that requires GPU resources and corresponds to the schedulable node is as follows: Score2=GpuScore×GpuWeight+NodeGPUSocre×NodeGpuWeight +CpuMemoryScore2×CpuMemoryWeight2 Among them, GpuWeight is the weight of the GPU score, NodeGpuWeight is the weight of the Pod quota score, CpuMemoryWeight2 is the weight of the second score of CPU and memory resources, GpuScore is the GPU score in the node, NodeGPUSocre is the Pod quota score, and CpuMemoryScore2 is the second score of CPU and memory resource balance; The GpuScore calculation formula is as follows: Among them, GpuScore is represented by the sum of the scores of each GPU in the schedulable node, and GpuDomain is represented by The GPU clock frequency of the schedulable node. PerMaxGpuDomain indicates the number of pods that can be scheduled in the schedulable node. The maximum clock frequency of all GPUs requested, GpuPower indicates the GPU power of the schedulable node, PerMaxGpuPower indicates the maximum power of all GPUs in the schedulable node that meet the Pod request to be scheduled, PerFreeGpuMemory indicates the free video memory of each GPU in the schedulable node, NodeFreeGpuMemory indicates the total free video memory in the schedulable node, TotalGpuMemory indicates the total GPU memory of the schedulable node, and NodeMaxGpuMemory indicates the total video memory in the schedulable node that meets the Pod request to be scheduled; The NodeGPUSocre calculation formula is as follows: Among them, PodGpuMemorySum represents the total amount of PodGPU memory requests to be scheduled; The CpuMemoryScore2 calculation formula is as follows: Among them, CpuMemoryScore2 is the second score of CPU and memory resource balance, cpu represents the CPU idleness of the schedulable node, memory represents the memory idleness of the schedulable node, NodeMaxCpuSum represents the maximum capacity of the CPU resources of the schedulable node, RequestedCPUSum represents the total CPU application of the Pod to be scheduled, NodeMaxMemorySum represents the maximum capacity of the memory resources of the schedulable node, and RequestedMemorySum represents the total memory application of the Pod to be scheduled.
Citation Information
Patent Citations
GPU resource using method and device and storage medium
CN110888743A
Multi-dimensional resource scheduling method under Kubernetes cluster architecture system
CN111522639A
Distributed container scheduling method and system based on shared GPU
CN112925611A
Cluster node resource scheduling method and device
CN113157379A
