Pod batch scheduling method and system based on GPU selection

Abstract

The invention provides a Pod batch scheduling method and system based on GPU selection, and the method comprises the steps: receiving a Pod group carrying a unique custom resource identifier in real time through a custom scheduler in a container cluster, the Pod group carrying the unique custom resource identifier is formed by determining a corresponding Pod group according to a calculation task received in real time and marking the corresponding unique custom resource identifier for the Pod group; based on the corresponding relationship between the type of the calculation task and the model information of the GPU, selecting the GPU carrying the model information corresponding to the Pod group with the unique custom resource identifier from the model information of the GPU pre-configured by each node as a target GPU; and according to the residual resource quantity of the GPUs in each node where the target GPU is located, determining one or more nodes where the target GPU corresponding to each Pod in the Pod group carrying the unique self-defined resource identifier is located as target nodes, so that the API server schedules all the Pods in the Pod group carrying the unique self-defined resource identifier to the corresponding target nodes in batches.

Description

Technical Field

[0001] The present invention relates to the technical field of scheduling in K8s, and in particular to a Pod batch scheduling method and system based on GPU selection. Background Art

[0002] Kubernetes (often referred to as K8s) is a container-based management platform, that is, a container cluster management platform. In actual applications, when executing a task, especially in deep learning model training, a Pod group consisting of multiple Pods needs to be batch-scheduled as a whole to complete the overall allocation of computing resources to complete the task. However, as a mainstream container orchestration platform, K8s's default scheduler is mainly used for scheduling a single Pod (Pod is a collection of a group of containers and is the smallest unit of scheduling in Kubernetes), and cannot implement batch scheduling of Pods. For the current remaining resources, when only a single Pod or some Pods in a Pod group may be able to allocate sufficient resources, the scheduling strategy of the default scheduler of K8s is to allocate the current remaining resources only to a single Pod or some Pods that meet the resources, resulting in all Pods in the Pod group not being successfully scheduled, low resource utilization efficiency, and thus the task cannot be completed.

[0003] As deep learning tasks require more computing power, many K8s clusters have deployed multiple models of GPUs (Graphics Processing Units). However, the default scheduler of K8s cannot schedule specific models or multiple models of GPUs. Since some tasks are too dependent on specific models of GPUs, existing methods cannot schedule the tasks to the specific models of GPUs, resulting in overall low efficiency and the so-called "short board effect". Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a Pod batch scheduling method and system based on GPU selection to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present invention provides a Pod batch scheduling method based on GPU selection, the method comprising the following steps:

[0006] A custom scheduler in a container cluster receives in real time a Pod group carrying a unique custom resource identifier, wherein the Pod group carrying the unique custom resource identifier is formed by determining a corresponding Pod group according to a computing task received in real time in the container cluster, and marking the Pod group with a corresponding unique custom resource identifier, wherein the Pod group includes multiple Pods;

[0007] Based on the correspondence between the type of computing task and the model information of the GPU, the GPU corresponding to the model information of the Pod group carrying the unique custom resource identifier is selected from the model information of the GPU pre-configured in each node in the container cluster as the target GPU of the corresponding computing task;

[0008] According to the remaining resources of the GPU in each node where the target GPU is located, one or more nodes where the target GPU corresponding to each Pod in the Pod group carrying the unique custom resource identifier is located are determined as target nodes for executing the corresponding computing task, so that the API server in the container cluster batch schedules all Pods in the Pod group carrying the unique custom resource identifier to the corresponding target nodes.

[0009] In some embodiments of the present invention, each Pod includes a name and resource requirements of the corresponding Pod.

[0010] In some embodiments of the present invention, determining one or more nodes where the target GPU is located corresponding to each Pod in the Pod group carrying the unique custom resource identifier according to the remaining resources of the GPU in each node where the target GPU is located includes:

[0011] Determine whether the remaining resources of the GPU in each node where the target GPU is located meet the resource requirements of each Pod in the Pod group carrying the unique custom resource identifier;

[0012] If the remaining resources of the GPUs in each node where the target GPU is located cannot meet the resource requirements of each Pod in the Pod group carrying the unique custom resource identifier, all Pods in the Pod group carrying the unique custom resource identifier are in a waiting resource allocation state until the remaining resources of the GPUs in each node where the target GPU is located meet the resource requirements of each Pod in the Pod group carrying the unique custom resource identifier;

[0013] If the remaining resources of the GPU in each node where the target GPU is located meet the resource requirements of each Pod in the Pod group carrying the unique custom resource identifier, then one or more nodes where the target GPU is located that can meet the corresponding Pod resource requirements are allocated to each Pod in the Pod group carrying the unique custom resource identifier.

[0014] In some embodiments of the present invention, when the model information of the target GPUs corresponding to multiple computing tasks is the same, when the waiting time of a Pod group in a state waiting for resource allocation exceeds a preset time, the resource allocation priority of the Pod group in a state waiting for resource allocation is increased or a reserved resource amount is set for the Pod group in a state waiting for resource allocation in the GPUs of one or more nodes where the target GPU is located.

[0015] In some embodiments of the present invention, the target node of the Pod group corresponding to the computing task is determined according to the order in which the computing tasks arrive at the task queue and the priority of the computing tasks to batch schedule the Pod group.

[0016] In some embodiments of the present invention, based on the correspondence between the type of computing task and the model information of the GPU, in the step of selecting the GPU with model information corresponding to the Pod group carrying a unique custom resource identifier from the model information of the GPU pre-configured in each node in the container cluster, if the computing task depends on a GPU with specific model information, the GPU of the specific model is selected; if the computing task does not depend on a GPU of a specific model, the comprehensive performance score of each GPU is obtained based on the weighted sum of the performance parameters of GPUs with different model information and the total amount of remaining resources, and the GPU with a higher comprehensive performance score is selected for the computing task with a higher priority.

[0017] In some embodiments of the present invention, the model information of the GPU pre-configured in each node in the container cluster and the remaining resource amount of the GPU in each node where the target GPU is located are obtained through real-time monitoring.

[0018] Another aspect of the present invention provides a Pod batch scheduling system based on GPU selection, the system comprising: a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps of the aforementioned method.

[0019] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the aforementioned method.

[0020] Another aspect of the present invention provides a computer program product, comprising computer instructions, which implement the steps of the above method when executed by a processor.

[0021] The Pod batch scheduling method and system based on GPU selection of the present invention, through the scheduling logic including GPU model selection and target node determination in the custom scheduling process of the custom scheduler, can dynamically select the GPU of the corresponding model (including the specific model) according to the different types and requirements of the computing task adaptively, and can batch schedule all Pods in the Pod group corresponding to the computing task to the target node of the selected GPU of the same model, which can optimize and significantly improve the execution efficiency of various computing tasks in Kubernetes, especially distributed computing tasks, and improve the utilization efficiency of computing resources. This method effectively overcomes the defects of the traditional Kubernetes scheduler in processing multiple models of GPU and Pod batch scheduling, such as scheduling computing tasks that rely on specific models of GPU to common models of GPU, being unable to realize the scheduling of multiple models of GPU, and insufficient Pod resources allocated for executing computing tasks, resulting in the inability to execute the computing tasks successfully.

[0022] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0023] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0025] Figure 1 A schematic diagram of a process of a Pod batch scheduling method based on GPU selection in one embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of the architecture of Pod batch scheduling based on GPU selection in one embodiment of the present invention;

[0027] Figure 3 The figure is a schematic diagram of a Pod batch scheduling strategy in one embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0029] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0030] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0031] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection with an intermediate.

[0032] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0033] In Kubernetes (container cluster), Pod is a collection of a group of containers, and Pod group is a collection of a group of Pods. Controllers are often used for distributed computing tasks. For example, the training task of a deep learning model is performed by a Pod group composed of multiple Pods, some of which are used to collect data, and other Pods are used for model training and updating and iteration of model parameters. These Pods work together to complete the model training task. At the same time, the completion of some special computing tasks may rely on a specific model of GPU due to its high computational complexity, high latency requirements, or the particularity of the task itself. Therefore, in order to achieve GPU adaptive selection of various computing tasks and Pod batch scheduling based on GPU selection, the embodiment of the present invention provides a Pod batch scheduling method and system based on GPU selection, which can effectively solve the shortcomings of the current k8s default scheduler in processing Pod batch scheduling and matching and scheduling of multiple models of GPUs with various computing tasks. While realizing Pod batch scheduling, the corresponding specific model of GPU with special performance can be selected according to the specific execution requirements and particularity of various computing tasks, thereby achieving efficient execution of various computing tasks and effectively improving the utilization efficiency of computing resources.

[0034] In a container cluster, the API server is the central management component of the cluster, which is used to handle internal and external requests and manage the state of the cluster. The API server is the core of the Kubernetes control plane, which is used to provide a RESTful API interface for interacting with other components in the cluster as follows: ① Request processing: The API server receives requests from users, Kubernetes control plane components (such as kube-controller-manager, kube-scheduler), and Kubernetes worker nodes (such as kubelet); ② Admission control: After the request is authenticated and authorized, the API server will further verify and modify the request through a series of admission webhooks; ③ Resource management: The API server maintains the declarative state of all Kubernetes resources, such as Pods, Services, Deployments, etc., to ensure that the actual state of the cluster is consistent with the expected state defined by the user.

[0035] Kubelet is a key component in the Kubernetes cluster. It runs on each worker node and is used to manage containerized applications on the node, ensure that the containers on the node (usually Docker containers) run according to the definition of the Pod, and communicate with the API Server: ① Pod management: Kubelet monitors the status of the Pod on the node to ensure that it runs according to the Pod specification (PodSpec); if the containers in the Pod do not run as expected, Kubelet will try to restart them; ② Device Plugin interface: Kubelet obtains hardware-related information such as the number and model of GPUs through the Device Plugin; ③ Resource monitoring: Kubelet regularly collects resource usage of nodes and Pods (such as CPU, memory, disk, network, etc.) and reports this information to the API Server; ④ Pod lifecycle events: Kubelet handles various events in the Pod lifecycle, such as Pod creation, update, deletion, etc., and synchronizes the Pod status with the API Server; ⑤ Node status management: Kubelet regularly reports the status of the node to the API Server, including the health of the node, resource availability, etc.; if there is a problem with the node, Kubelet will mark it as unavailable and notify the control plane to make corresponding scheduling decisions. In short, Kubelet is a daemon process for each working node in the Kubernetes cluster. It is used to manage containers and Pods on the nodes, ensure that they run as defined, and communicate with the control plane. It is one of the key components of Kubernetes to implement container orchestration and management.

[0036] Custom Scheduler is an extension mechanism of Kubernetes Scheduler, which allows users to customize and extend the scheduler to meet specific scheduling requirements. Through the custom scheduler, users can insert custom scheduling logic into the standard scheduling process to achieve more flexible and efficient scheduling strategies. The embodiment of the present invention implements Pod batch scheduling based on GPU selection through custom scheduling logic. The custom scheduler is deployed to the container cluster as a Deployment and is part of the control plane in the cluster. The device plug-in is deployed to each working node as a Daemonset to manage the GPU of each node.

[0037] Custom Resource Definitions (CRD) is an extension mechanism provided by Kubernetes to extend its functionality, allowing users to extend the API by defining custom resources. CRD enables users to create and manage resource types that meet their needs without modifying the core code of Kubernetes. CRD allows users to define new Kubernetes resource types that can be created, read, updated, and deleted like Kubernetes' built-in resources Pod, Service, etc. CR (Custom Resource, CR) is a specific instance created based on CRD. These instances follow the specifications defined in CRD and contain data specific to the resource.

[0038] Figure 1 and Figure 2 The flowchart of the Pod batch scheduling method based on GPU selection and the architecture diagram of the Pod batch scheduling based on GPU selection in one embodiment of the present invention are shown respectively. Figure 1 As shown, the method consists of Figure 2 The architecture shown is executed by a custom scheduler of the container cluster, including the following steps:

[0039] In step S110, a custom scheduler in the container cluster receives in real time a Pod group carrying a unique custom resource identifier, wherein the Pod group carrying a unique custom resource identifier is formed by determining a corresponding Pod group based on a computing task received in real time in the container cluster, and marking the Pod group with a corresponding unique custom resource identifier, and the Pod group includes multiple Pods.

[0040] The API server in the container cluster maintains a task queue, receives computing tasks submitted and sent by user requests in real time, and sends computing task information to the Mutating Admission Webhook plug-in in the cluster for processing. The Mutating Admission Webhook plug-in defines and creates the corresponding Pod group according to the received computing tasks (i.e., determines whether to modify the Pod parameters as needed), including defining the number of Pods in the Pod group, the Pod name of each Pod (such as Pod1, Pod2, Pod3, etc.) and resource requirements through CRD, and creates the corresponding Pod group according to the defined Pod group information, that is, obtains the Pod group with the specified number of Pods, Pod name and resource requirements (the resource requirement of the Pod group is the sum of the resource requirements of all Pods in the Pod group); then the CR is used to mark the created Pod group with a unique custom resource identifier, forming a Pod group with a unique custom resource identifier, that is, all Pods in the Pod group have the same CR identifier. The Mutating Admission Webhook plug-in then sends the Pod group with a unique custom resource identifier back to the API server, and the custom scheduler receives the Pod group with a unique custom resource identifier corresponding to the computing task from the API server.

[0041] By defining CRD, you can describe and manage a group of related Pods. Specifically, you can bind batch scheduling objects, that is, multiple Pods used to execute a computing task, to the same CR. By identifying all Pods in a Pod group with CR, it is easier to classify all Pods identified with the same CR into the same group during scheduling, so that all Pods in the Pod group can be assigned or scheduled to GPUs of the same model (for example, a specific model specified by the user).

[0042] Step S120, based on the correspondence between the type of computing task and the model information of the GPU, select the GPU with the corresponding model information of the Pod group carrying the unique custom resource identifier from the model information of the GPU pre-configured in each node in the container cluster as the target GPU of the corresponding computing task.

[0043] The embodiment of the present invention pre-registers and configures multiple GPUs of various models to each node of the container cluster through a self-developed device plug-in (in actual application scenarios, existing plug-ins can only register a single specific model of GPU to the node of the container cluster, and cannot realize the registration of multiple different models of GPUs). Specifically, the device plug-in reports the model, quantity and resource information of the GPU configured in each node to the Kubelet of each node. After Kubelet receives the model, quantity and resource information of the GPU reported by the device plug-in of each node, it registers the GPU device with the corresponding quantity, model and resource information to the API server. At this time, the GPU device information can be seen in the cluster. The custom scheduler obtains the quantity and model information of the GPU pre-configured for each node and the remaining resources of each GPU from the API server in real time, and selects the corresponding model of GPU as the target GPU for the received Pod group carrying a unique custom resource identifier from the obtained model information of the GPU pre-configured for each node.

[0044] For different computing tasks, different GPU models with different performance requirements are selected according to their respective task types and task requirements to enable each computing task to be successfully executed. Since there are multiple GPU models configured for each node in the container cluster, computing tasks using different GPU models can be scheduled and executed in parallel.

[0045] In some embodiments, based on the correspondence between the type of computing task and the model information of the GPU, in the step of selecting the GPU with the model information corresponding to the Pod group carrying the unique custom resource identifier from the model information of the GPU pre-configured in each node in the container cluster, if the computing task depends on a GPU with specific model information, the GPU of the specific model is selected; if the computing task does not depend on a GPU of a specific model, the comprehensive performance score of each GPU is obtained based on the weighted sum of the performance parameters of GPUs with different model information and the total amount of remaining resources, and the GPU with the higher comprehensive performance score is selected for the computing task with a higher priority.

[0046] Specifically, the specific model of GPU can be a specific model of GPU specified by the user, or a specific model of GPU with special performance, wherein the special performance can refer to the characteristics of being able to efficiently carry and execute computing tasks with large amount of computation, high computational complexity, and large amount of required computing resources. When the computing task has no special requirements and dependence on the GPU model, the priority of the computing task is set according to the specific needs of the computing task, for example, computing tasks with high latency requirements and / or high computational complexity are pre-set to a higher priority, and GPUs with higher comprehensive performance scores are preferentially selected for such higher priority computing tasks to meet the computing requirements of the task, and to achieve reasonable allocation and effective utilization of computing resources, thereby improving the execution success rate of various computing tasks.

[0047] Step S130, based on the remaining resources of the GPU in each node where the target GPU is located, determine one or more nodes where the target GPU corresponding to each Pod in the Pod group carrying the unique custom resource identifier is located as the target node for executing the corresponding computing task, so that the API server in the container cluster batch schedules all Pods in the Pod group carrying the unique custom resource identifier to the corresponding target nodes.

[0048] After the custom scheduler obtains the Pod group with a unique custom resource identifier corresponding to the computing task sent by the API server, it allocates the computing task to one or more nodes where the target GPU is located according to the GPU usage (remaining resources) reported by each node where the target GPU is located and the resource requirements of the Pod group corresponding to the computing task, thereby determining the target node for executing the computing task and running the Pod group, and returning the determined target node and the corresponding Pod group to the API server. The API server sends the Pod group, the GPU model selected by the custom scheduler for the computing task, and the determined target node of the GPU model to the Kubelet of the corresponding node to schedule the Pod group to the target node. The Kubelet calls the device plug-in in the same node to configure the environment information for the Pod group, deploys the Pod group with the configured environment to the node, and specifies the GPU and video memory (in the container, the video memory of the computing task is limited. The purpose is to reasonably allocate GPU resources in a multi-task environment to avoid a single task occupying too much video memory and causing other tasks to fail to run). After scheduling the Pod group corresponding to each computing task to the corresponding target node, the target node will pull the required container image and start the container. After all Pods in the Pod group enter the running state, Kubernetes will continue to monitor and manage the life cycle of the Pod to ensure that the Pod runs in the expected state.

[0049] Specifically, in step S130, according to the remaining resources of the GPU in each node where the target GPU is located, determining one or more nodes where the target GPU is located corresponding to each Pod in the Pod group carrying the unique custom resource identifier includes the following steps:

[0050] Step S131, determining whether the remaining resources of the GPU in each node where the target GPU is located meet the resource requirements of each Pod in the Pod group carrying the unique custom resource identifier;

[0051] Step S132: If the remaining resources of the GPUs in the nodes where the target GPU is located cannot meet the resource requirements of the Pods in the Pod group carrying the unique custom resource identifier, all Pods in the Pod group carrying the unique custom resource identifier are in a waiting resource allocation state until the remaining resources of the GPUs in the nodes where the target GPU is located meet the resource requirements of the Pods in the Pod group carrying the unique custom resource identifier.

[0052] Step S133: If the remaining resources of the GPU in each node where the target GPU is located meet the resource requirements of each Pod in the Pod group carrying the unique custom resource identifier, one or more nodes where the target GPU is located that can meet the corresponding Pod resource requirements are allocated to each Pod in the Pod group carrying the unique custom resource identifier.

[0053] like Figure 3As shown, the custom scheduler completes the batch scheduling of the Pod group in accordance with steps S131 to S133 and in conjunction with the API server. Exemplarily, assume that there are three Pods that need to be scheduled simultaneously in the Pod group, namely Pod1, Pod2 and Pod3, and the three Pods have the same CR identifier. When judging whether the remaining resources of the GPU in each node where the target GPU is located meet the resource requirements of the Pod group, if the remaining resources of the GPU of one of the nodes where the target GPU is located meet the resource requirements of all Pods in the Pod group, Pod1, Pod2 and Pod3 are scheduled to the one node, and the node allocates the required computing resources to Pod1, Pod2 and Pod3 from the remaining resources of the GPU on the node; if the remaining resources of the GPU in two of the nodes where the target GPU is located meet the resource requirements of all Pods in the Pod group, for example, the remaining resources of the GPU of one of the nodes only meet the computing resources required by Pod1, and the remaining resources of the GPU of the other node can meet the computing resources required by Pod2 and Pod3, then Pod1 is scheduled to the node. There are many situations. If the remaining resources of the GPUs in three of the nodes where the target GPU is located meet the resource requirements of all Pods in the Pod group, that is, the remaining resources of the GPUs in the three nodes can only meet the computing resources required by the three Pods in the Pod group, then Pod1, Pod2 and Pod3 are scheduled to the three nodes respectively, and the three nodes allocate the computing resources required by Pod1, Pod2 and Pod3 respectively from the remaining resources of their respective GPUs. If the remaining resources of the GPUs in the nodes where the target GPU is located cannot meet the resource requirements of all Pods in the Pod group, then the Pod group waits for the GPUs on the nodes to release computing resources until the remaining resources of the GPUs in the nodes where the target GPU is located can meet the resource requirements of all Pods in the Pod group.

[0054] In some embodiments, when the model information of the target GPUs corresponding to multiple computing tasks is the same, when the waiting time of a Pod group in a state waiting for resource allocation exceeds a preset time, the resource allocation priority of the Pod group in a state waiting for resource allocation is increased or a reserved resource amount is set for the Pod group in a state waiting for resource allocation in the GPUs of one or more nodes where the target GPU is located.

[0055] When there are multiple computing tasks, for example, computing tasks A and B correspond to the same target GPU model, model C, where the resource demand of Pod group A corresponding to computing task A is greater than the resource demand of Pod group B corresponding to computing task B, in the case where the current remaining resources of the GPUs of each node where the target GPU model C is located in the container cluster only meet the resource demand of Pod group B and cannot meet the resource demand of Pod group A, Pod group B is successfully scheduled, and Pod group A waits for the release of computing resources. If a task similar to computing task B appears after computing task B is executed, and the resource demand of the Pod group of this task is the same as the resource demand of Pod group B, then the Pod group is immediately scheduled and the corresponding computing task is executed, and Pod group A continues to wait, resulting in the inability to execute computing task A. Then, in order to avoid the waiting time of computing task A being too long, when the waiting time of Pod group A exceeds the preset time, the priority of computing task A is increased, and computing task A is executed first after sufficient resources are released; or, sufficient reserved resources are set for the GPUs of one or more nodes where the target GPU is located to meet the resource demand of Pod group A, thereby avoiding the waiting of computing task A.

[0056] In some embodiments, according to the order in which the computing tasks arrive at the task queue and the priority of the computing tasks, the target node of the Pod group corresponding to the computing tasks is determined to batch schedule the Pod group.

[0057] The computing tasks submitted and sent by the user in real time are queued in the task queue according to the order in which they arrive at the task queue. According to the principle of giving priority to computing tasks that arrive in the queue first and have higher task priorities, resources are allocated first to the Pod group corresponding to the computing tasks that arrive in the queue first and have higher priorities, and the Pod group is scheduled to run on one or more nodes where the selected target GPU is located, so that the computing task is executed first. If the computing task arrives in the queue later but has a higher priority, the Pod group for the computing task is also allocated resources from the remaining resources of the node where the target GPU is located, so that the Pod group is scheduled to run in batches on the corresponding target nodes.

[0058] In some embodiments, the model information of the GPU pre-configured in each node in the container cluster and the remaining resource amount of the GPU in each node where the target GPU is located are obtained through real-time monitoring.

[0059] Through the GPU monitoring interface (i.e., device plug-in interface) of each node in the container cluster, real-time monitoring is performed to obtain the pre-configured GPU model information of each node (passing through Kubelet, API server, and custom scheduler in turn so that the custom scheduler obtains the GPU model information). There is no need to pre-classify and identify the GPU, which simplifies the resource management method and reduces the resource occupation caused by pre-classification and identification. The real-time monitoring of the GPU monitoring interface of each node can also obtain the remaining resources of the GPU of each node in real time, passing through Kubelet, API server, and custom scheduler in turn so that the custom scheduler can allocate the required resources to the Pod group according to the remaining resources of the GPU configured in each node.

[0060] In summary, the Pod batch scheduling method and system based on GPU selection in the embodiment of the present invention, through the scheduling logic including GPU model selection and target node determination in the custom scheduling process of the custom scheduler, can dynamically and adaptively select the corresponding model (including specific model) of GPU according to the different types and requirements of computing tasks, and can batch schedule all Pods in the Pod group corresponding to the computing task to the target node of the selected GPU of the same model, which can optimize and significantly improve the execution efficiency of various computing tasks in Kubernetes, especially distributed computing tasks, and improve the utilization efficiency of computing resources. This method effectively overcomes the defects of the traditional Kubernetes scheduler in processing multiple models of GPUs and Pod batch scheduling, such as scheduling computing tasks that rely on specific models of GPUs to ordinary models of GPUs, failing to implement scheduling of multiple models of GPUs, and insufficient Pod resources allocated for executing computing tasks, resulting in the inability to successfully execute the computing tasks.

[0061] Corresponding to the above method, the present invention also provides a Pod batch scheduling system based on GPU selection, the system comprising a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps of the above method.

[0062] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0063] An embodiment of the present invention further provides a computer program product, comprising computer instructions, which implement the steps of the aforementioned method when executed by a processor.

[0064] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0065] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0066] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A Pod batch scheduling method based on GPU selection, characterized in that: The method comprises: A custom scheduler in a container cluster receives in real time a Pod group carrying a unique custom resource identifier, wherein the Pod group carrying the unique custom resource identifier is formed by determining a corresponding Pod group according to a computing task received in real time in the container cluster, and marking the Pod group with a corresponding unique custom resource identifier, wherein the Pod group includes multiple Pods; Based on the correspondence between the type of computing task and the model information of the GPU, the GPU corresponding to the model information of the Pod group carrying the unique custom resource identifier is selected from the model information of the GPU pre-configured in each node in the container cluster as the target GPU of the corresponding computing task; According to the remaining resources of the GPU in each node where the target GPU is located, one or more nodes where the target GPU corresponding to each Pod in the Pod group carrying the unique custom resource identifier is located are determined as target nodes for executing the corresponding computing task, so that the API server in the container cluster batch schedules all Pods in the Pod group carrying the unique custom resource identifier to the corresponding target nodes.

2. The method according to claim 1, characterized in that Each Pod includes the name and resource requirements of the corresponding Pod.

3. The method according to claim 2, characterized in that Determining one or more nodes where the target GPU is located corresponding to each Pod in the Pod group carrying the unique custom resource identifier according to the remaining resources of the GPU in each node where the target GPU is located, includes: Determine whether the remaining resources of the GPU in each node where the target GPU is located meet the resource requirements of each Pod in the Pod group carrying the unique custom resource identifier; If the remaining resources of the GPUs in each node where the target GPU is located cannot meet the resource requirements of each Pod in the Pod group carrying the unique custom resource identifier, all Pods in the Pod group carrying the unique custom resource identifier are in a waiting resource allocation state until the remaining resources of the GPUs in each node where the target GPU is located meet the resource requirements of each Pod in the Pod group carrying the unique custom resource identifier; If the remaining resources of the GPU in each node where the target GPU is located meet the resource requirements of each Pod in the Pod group carrying the unique custom resource identifier, then one or more nodes where the target GPU is located that can meet the corresponding Pod resource requirements are allocated to each Pod in the Pod group carrying the unique custom resource identifier.

4. The method according to claim 3, characterized in that When the model information of the target GPUs corresponding to multiple computing tasks is the same, when the waiting time of a Pod group in a state of waiting for resource allocation exceeds a preset time, the resource allocation priority of the Pod group in a state of waiting for resource allocation is increased or a reserved resource amount is set for the Pod group in a state of waiting for resource allocation in the GPUs of one or more nodes where the target GPU is located.

5. The method according to claim 1, characterized in that According to the order in which the computing tasks arrive at the task queue and the priority of the computing tasks, the target node of the Pod group corresponding to the computing tasks is determined to batch schedule the Pod group.

6. The method according to claim 1, characterized in that Based on the correspondence between the type of computing task and the model information of the GPU, in the step of selecting the GPU with the model information corresponding to the Pod group carrying the unique custom resource identifier from the model information of the GPU pre-configured in each node in the container cluster, if the computing task depends on the GPU with specific model information, the GPU of the specific model is selected; if the computing task does not depend on the GPU of a specific model, the comprehensive performance score of each GPU is obtained based on the weighted sum of the performance parameters of GPUs with different model information and the total amount of remaining resources, and the GPU with a higher comprehensive performance score is selected for the computing task with a higher priority.

7. The method according to claim 1, characterized in that The model information of the GPU pre-configured in each node in the container cluster and the remaining resource amount of the GPU in each node where the target GPU is located are obtained through real-time monitoring.

8. A Pod batch scheduling system based on GPU selection, comprising a processor, a memory, and computer instructions stored in the memory, characterized in that: The processor is used to execute the computer instructions, and when the computer instructions are executed, the system implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images