Resource scheduling method and device, storage medium and electronic device

Abstract

The present invention discloses a resource scheduling method and device, a storage medium and an electronic device. The method comprises: obtaining identification information of an image processing unit (GPU) of a server, wherein the identification information is used to identify a scheduling mode of the GPU; obtaining resource information corresponding to the identification information; and determining a scheduling mode of the GPU according to the resource information, thereby achieving the purpose of determining a scheduling mode according to resource information reported by the server, that is, first determining a scheduling mode of the GPU according to the resource information, and further determining a scheduling process of the GPU according to the scheduling mode, while supporting both shared and exclusive use modes, so that users can directly use any mode without having to set whether to open sharing through a switch, thereby solving the technical problem of low efficiency of GPU resource scheduling mode in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a resource scheduling method and device, a storage medium and an electronic device. Background Art

[0002] With the development of artificial intelligence technology, deep learning is becoming more and more important. The implementation of deep learning requires the support of multiple technologies, such as servers, GPUs (Graphics Processing Units), clusters, and cluster management and scheduling software.

[0003] Kubernetes has complete cluster management capabilities, multi-level security protection and access mechanisms, multi-tenant application support capabilities, powerful fault detection and self-repair capabilities, and multi-granular resource quota management capabilities. Kubernetes has added GPU resource scheduling since version 1.6, so it is widely used in the field of artificial intelligence as a scheduling and monitoring support platform for deep learning.

[0004] The existing technical solutions for GPU scheduling through the Kubernetes system have the following problems: users need to control whether to enable GPU resource sharing through switches, which is not flexible in usage. Users need to configure the upper limit of the number of container tasks for sharing GPU resources for each GPU device of the server, which is cumbersome to configure, especially when there are many GPU devices connected to the Kubernetes cluster. The GPU sharing scheduling strategy is only supported by configuring the upper limit of the number of tasks for sharing GPU resources allowed by a single GPU device. The scheduling algorithm is single and has poor scalability, resulting in low efficiency in GPU resource scheduling.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present invention provide a resource scheduling method and device, a storage medium and an electronic device, so as to at least solve the technical problem of low efficiency of GPU resource scheduling in the prior art.

[0007] According to one aspect of an embodiment of the present invention, a resource scheduling method is provided, including: obtaining identification information of a GPU of a server, wherein the identification information is used to identify a scheduling method of the GPU; obtaining resource information corresponding to the identification information; and determining a scheduling method of the GPU based on the resource information.

[0008] Optionally, before obtaining the identification information of the GPU of the server, the method includes: marking the target identification information of the GPU according to the GPU resource scheduling method, wherein the scheduling method includes a shared mode and an exclusive mode, and wherein the target identification information includes the identification information.

[0009] Optionally, obtaining resource information corresponding to the identification information includes: when the GPU resource scheduling mode of the server is a shared mode, obtaining video memory information of the server GPU device, and the resource information includes the video memory information of the server GPU device; when the GPU resource scheduling mode of the server is an exclusive mode, obtaining server GPU device number information reported by the GPU, and the resource information includes the server GPU device number information.

[0010] Optionally, determining the scheduling mode of the GPU according to the resource information includes: when the GPU resource scheduling mode of the server is a shared mode, determining a first server GPU to be scheduled, wherein the first server GPU satisfies the available total video memory to meet the task requirements; and when the GPU resource scheduling mode of the server is an exclusive mode, determining a second server GPU to be scheduled, wherein the second server GPU satisfies the available device number memory to meet the task requirements.

[0011] Optionally, the method further includes: when the GPU resource scheduling mode of the server is a shared mode, determining a third server GPU to be scheduled, wherein the third server GPU satisfies the requirement that idle resources meet the task conditions and the GPU device has the least remaining resources; and when the GPU resource scheduling mode of the server is an exclusive mode, determining a fourth server GPU to be scheduled, wherein the fourth server GPU satisfies the requirement that idle GPU devices meet the task conditions and the node has the least remaining available GPU devices.

[0012] Optionally, after determining the first server GPU to be scheduled, the method includes:

[0013] Assign multiple GPU container tasks to the same GPU device on the same node, and give priority to assigning GPU devices to container tasks with the earliest scheduling process completion timestamp.

[0014] Optionally, after determining the second server GPU to be scheduled, the method further includes: if the task before the second server GPU is scheduled is not completed, the current task enters the scheduling queue, and the scheduling is retried until the second server GPU is determined.

[0015] According to another aspect of an embodiment of the present invention, a resource scheduling device is also provided, including: a first acquisition unit, used to obtain identification information of a GPU of a server, wherein the identification information is used to identify a scheduling method of the GPU; a second acquisition unit, used to obtain resource information corresponding to the identification information; and a determination unit, used to determine the scheduling method of the GPU based on the resource information.

[0016] Optionally, the above-mentioned device includes: a marking unit, which is used to mark the target identification information of the GPU according to the GPU resource scheduling method before obtaining the identification information of the GPU of the server, wherein the scheduling method includes a shared mode and an exclusive mode, and wherein the target identification information includes the identification information.

[0017] Optionally, the second acquisition unit includes: a first acquisition module, used to obtain the video memory information of the server GPU device when the GPU resource scheduling mode of the server is a shared mode, and the resource information includes the video memory information of the server GPU device; a second acquisition module, used to obtain the server GPU device number information reported by the GPU when the GPU resource scheduling mode of the server is an exclusive mode, and the resource information includes the server GPU device number information.

[0018] Optionally, the determination unit includes: a first determination module, used to determine the first server GPU to be scheduled when the GPU resource scheduling mode of the server is a shared mode, wherein the first server GPU satisfies the available total video memory to meet the task requirements; a second determination module, used to determine the second server GPU to be scheduled when the GPU resource scheduling mode of the server is an exclusive mode, wherein the second server GPU satisfies the available device number memory to meet the task requirements.

[0019] Optionally, the above-mentioned device includes: a third determination module, used to determine the third server GPU to be scheduled when the GPU resource scheduling mode of the server is a shared mode, wherein the third server GPU satisfies the GPU device whose idle resources meet the task conditions and has the least remaining resources; a fourth determination module, used to determine the fourth server GPU to be scheduled when the GPU resource scheduling mode of the server is an exclusive mode, wherein the fourth server GPU satisfies the node whose idle GPU devices meet the task conditions and has the least remaining available GPU devices.

[0020] Optionally, the device includes: a scheduling unit, which is used to allocate multiple GPU container tasks to the same GPU device of the same node after determining the first server GPU to be scheduled, and give priority to allocating GPU devices to container tasks with the earliest scheduling process completion timestamp.

[0021] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned resource scheduling method when running.

[0022] According to another aspect of an embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the resource scheduling method through the computer program.

[0023] In an embodiment of the present invention, by obtaining identification information of an image processing unit GPU of a server, wherein the identification information is used to identify a scheduling mode of the GPU; obtaining resource information corresponding to the identification information; and determining a scheduling mode of the GPU according to the resource information, the purpose of determining a scheduling mode according to resource information reported by the server is achieved, that is, the scheduling mode of the GPU is first determined according to the resource information, and the scheduling process of the GPU is further determined according to the scheduling mode. Both shared and exclusive use modes are supported, and users can directly use any mode without having to set whether to enable sharing through a switch, thereby solving the technical problem of low efficiency of GPU resource scheduling mode in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0025] Figure 1 is a schematic diagram of an application environment of an optional resource scheduling method according to an embodiment of the present invention;

[0026] Figure 2 is a flow chart of an optional resource scheduling method according to an embodiment of the present invention;

[0027] Figure 3 A flowchart of an optional method for implementing GPU device sharing scheduling and use on a Kubernetes platform according to an embodiment of the present invention;

[0028] Figure 4 An optional implementation of GPU device sharing scheduling and use device diagram on a Kubernetes platform according to an embodiment of the present invention;

[0029] Figure 5 An optional GPU container task extension scheduling process according to an embodiment of the present invention;

[0030] Figure 6A schematic diagram of the structure of an optional resource scheduling device according to an embodiment of the present invention;

[0031] Figure 7 A schematic structural diagram of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] According to one aspect of an embodiment of the present invention, a resource scheduling method is provided. Optionally, as an optional implementation, the resource scheduling method can be applied to, but is not limited to, Figure 1 In the environment shown. A container cluster management system is running on the server 106, and the terminal device 102 initiates a request to obtain GPU resources to the container cluster management system to obtain GPU resources from the server. Among them, the container cluster management system in the server 106 obtains the identification information of the image processing unit GPU of the server, wherein the identification information is used to identify the scheduling mode of the GPU; obtains the resource information corresponding to the identification information; determines the scheduling mode of the GPU according to the resource information, and achieves the purpose of determining the scheduling mode according to the resource information reported by the server, that is, first determining the scheduling mode of the GPU according to the resource information, and further determining the scheduling process of the GPU according to the scheduling mode, and supports both shared and exclusive use modes at the same time. The user can directly use any mode without having to set whether to open sharing through a switch, thereby solving the technical problem of low efficiency of the GPU resource scheduling mode in the prior art.

[0035] Optionally, in this embodiment, the terminal device 102 may be a terminal device configured with a target client, which may include but is not limited to at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, etc. The target client may be a video client, an instant messaging client, a browser client, an education client, etc. The network may include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network, and a wide area network, and the wireless network includes: Bluetooth, WIFI, and other networks that implement wireless communication. The server may be a single server, or a server cluster consisting of multiple servers, or a cloud server. The above is only an example, and this embodiment does not impose any limitation on this.

[0036] Optionally, as an optional implementation, as Figure 2 As shown, the above resource scheduling method includes:

[0037] Step S202: Obtain identification information of a GPU of the server, wherein the identification information is used to identify a scheduling mode of the GPU.

[0038] Step S203: Acquire resource information corresponding to the identification information.

[0039] Step S206, determining a GPU scheduling method according to the resource information.

[0040] Optionally, in this embodiment, the above resource scheduling method may include but is not limited to being applied to a container cluster management system (Kubernetes) platform to implement GPU resource scheduling through the platform.

[0041] That is to say, the container cluster management system obtains the identification information of the server's image processing unit GPU, where the identification information is used to identify the scheduling method of the GPU; obtains resource information corresponding to the identification information; and determines the scheduling method of the GPU based on the resource information, thereby achieving the purpose of determining the scheduling method based on the resource information reported by the server, thereby solving the technical problem of low efficiency of the GPU resource scheduling method in the prior art.

[0042] Among them, in the application, the scheduling mode of GPU resources is usually divided into a shared mode and an exclusive mode. In the embodiment of the present application, the purpose of the scheduling mode can be determined according to the resource information reported by the server, thereby solving the technical problem of low efficiency of GPU resource scheduling in the prior art.

[0043] Optionally, in this embodiment, before obtaining the identification information of the GPU of the server, it can include: marking the target identification information of the GPU according to the GPU resource scheduling method, wherein the scheduling method includes a shared mode and an exclusive mode, and wherein the target identification information includes identification information.

[0044] Wherein, obtaining resource information corresponding to the identification information may include:

[0045] When the GPU resource scheduling mode of the server is in the sharing mode, obtain the video memory information of the GPU device of the server, and the resource information includes the video memory information of the GPU device of the server;

[0046] When the GPU resource scheduling mode of the server is exclusive mode, obtain the server GPU device number information reported by the GPU, and the resource information includes the server GPU device number information.

[0047] It should be noted that the GPU scheduling method determined according to the resource information may include:

[0048] When the GPU resource scheduling mode of the server is a sharing mode, determining a first server GPU to be scheduled, wherein the first server GPU satisfies the task requirement by satisfying the available total video memory;

[0049] When the GPU resource scheduling mode of the server is the exclusive mode, a second server GPU is determined to be scheduled, wherein the second server GPU satisfies the available device number and memory to meet the task requirement.

[0050] It should also be noted that, when the GPU resource scheduling mode of the server is a shared mode, the third server GPU is determined to be scheduled, wherein the third server GPU satisfies the idle resources that meet the task conditions and is the GPU device with the least remaining resources; when the GPU resource scheduling mode of the server is an exclusive mode, the fourth server GPU is determined to be scheduled, wherein the fourth server GPU satisfies the idle GPU devices that meet the task conditions and is the node with the least remaining available GPU devices.

[0051] Through the embodiments provided by the present application, by obtaining identification information of the image processing unit GPU of the server, wherein the identification information is used to identify the scheduling mode of the GPU; obtaining resource information corresponding to the identification information; and determining the scheduling mode of the GPU according to the resource information, the purpose of determining the scheduling mode according to the resource information reported by the server is achieved, that is, the scheduling mode of the GPU is first determined according to the resource information, and the scheduling process of the GPU is further determined according to the scheduling mode. At the same time, both shared and exclusive use modes are supported. Users can directly use any mode without having to set whether to enable sharing through a switch, thereby solving the technical problem of low efficiency of GPU resource scheduling mode in the prior art.

[0052] Optionally, in this embodiment, after determining the first server GPU to be scheduled, the method may include: allocating multiple GPU container tasks to the same GPU device of the same node, and preferentially allocating the GPU device to the container task with the earliest scheduling process completion timestamp.

[0053] It can be understood that when there are multiple GPU container tasks (shared mode) assigned to the same GPU device of the same node, the GPU sharing plug-in will give priority to allocating GPU devices to the container task with the earliest scheduling process completion timestamp.

[0054] Optionally, in this embodiment, after determining the second server GPU to be scheduled, the method may further include: if the task before the second server GPU is scheduled is not completed, the current task enters the scheduling queue, and the scheduling is retried until the second server GPU is determined.

[0055] In this embodiment, when the GPU container task scheduling stage is not completed (not pre-selected by the Kubernetes scheduler or further screened by the GPU extension scheduler), it enters the scheduling queue, and the Kubernetes scheduler and the GPU extension scheduler will perform periodic scheduling retries until the final schedulable nodes and GPU devices are screened out.

[0056] The above embodiment is described below in conjunction with a specific embodiment, that is, a method for implementing GPU device sharing scheduling and use on the Kubernetes platform. Figure 3 As shown in the figure, it is a flowchart of the method for implementing GPU device sharing scheduling and usage on the Kubernetes platform. The specific instructions are as follows:

[0057] Step S301, start;

[0058] Step S302: The GPU sharing plug-in reports the server GPU device memory information to the Kubernetes platform;

[0059] The GPU sharing plug-in runs on the server managed by Kubernetes.

[0060] Step S303, the GPU exclusive plug-in reports the number of server GPU devices to the Kubernetes platform;

[0061] Among them, the GPU exclusive plug-in runs on the server managed by Kubernetes.

[0062] Step S304, creating a GPU shared / exclusive container task;

[0063] Among them, when creating a GPU shared / exclusive container task, the user can freely specify the task type as needed without switch control; it should be noted that the GPU resource names in the Kubernetes scheduler corresponding to shared and exclusive are different. For example, shared is based on video memory scheduling and allocation, and its resource name is gpu-mem. Exclusive can be freely used to distinguish and mark shared / exclusive usage, and can be named gpu-count. The above naming is an implementation method in this embodiment and is not specifically limited.

[0064] Step S305, determining the task type, if the task type is exclusive mode, executing step S306, if the task type is shared mode, executing step S307;

[0065] Among them, Kubernetes determines whether the container task uses the GPU in shared mode or exclusive mode.

[0066] Step S306, the Kubernetes scheduler preliminarily screens the schedulable nodes, that is, the number of remaining available devices meets the container task requirements;

[0067] Step S307, the Kubernetes scheduler preliminarily screens the schedulable nodes, that is, the remaining available total video memory meets the container task requirements;

[0068] Among them, the Kubernetes scheduler performs preliminary screening of the schedulable nodes (GPU servers). The preliminary screening of the schedulable exclusive container task nodes passes the key necessary condition that the number of remaining available GPU devices on the node meets the container task requirements; the preliminary screening of the schedulable shared container task nodes passes the key necessary condition that the total available GPU memory on the node meets the container task requirements.

[0069] It should be noted that in addition to the above conditions, there are also some other necessary conditions for the Kubernetes scheduler, such as whether the container application node selector label matches the node label, whether other node resources such as CPU and memory meet the container task requirements, etc., following the native implementation logic of Kubernetes.

[0070] Step S308, the GPU extension scheduler is further screened;

[0071] Among them, after the Kubernetes scheduler completes the pre-selection, the GPU extension scheduler completes the further screening of the nodes. Specifically, the screening condition for schedulable shared container task nodes is that the node has at least one GPU device with remaining unallocated video memory that meets the container task video memory requirement; the screening condition for schedulable exclusive container task nodes is that the number of remaining unallocated GPU devices on the node (excluding those used by shared GPU container tasks) meets the container task device number requirement.

[0072] Step S309, the GPU extension scheduler performs optimal scheduling, i.e., determines according to the scheduling algorithm;

[0073] Among them, after further screening is completed, the GPU extension scheduler (scheduling algorithm part) completes the optimal scheduling, selects the final scheduling node and GPU device, marks them in the container task Annotation, and stamps the container task with the timestamp of the completion of the scheduling process. The scheduling algorithm can be binpack, fareshare, etc. (including but not limited to the above two). The algorithm module is connected to the GPU extension scheduler device in the form of a plug-in, and the GPU extension scheduler calls the algorithm module in the interface call mode without the need to perceive the specific algorithm implementation.

[0074] Step S310, determining the task type, if the task type is exclusive mode, executing step S311, if the task type is shared mode, executing step S312;

[0075] Step S311, the Kubernetes scheduler cooperates with the GPU exclusive plug-in to complete GPU device allocation;

[0076] Step S312: The Kubernetes scheduler cooperates with the GPU sharing plug-in to complete GPU device allocation;

[0077] Among them, when the GPU container task scheduling stage is not completed (not pre-selected by the Kubernetes scheduler or further screened by the GPU extension scheduler), it enters the scheduling queue. The Kubernetes scheduler and the GPU extension scheduler will perform periodic scheduling retries until the final schedulable nodes and GPU devices are screened out.

[0078] When multiple GPU container tasks (shared mode) are assigned to the same GPU device on the same node, the GPU sharing plug-in will prioritize the GPU device for the container task with the earliest scheduling process completion timestamp.

[0079] Step S313, end.

[0080] like Figure 4 As shown in the figure, a diagram of GPU device sharing scheduling and usage is implemented on the Kubernetes platform, which is described in detail as follows.

[0081] Among them, Kubelet and Kube-Scheduler are Kubernetes components. Kubelet is the main "node agent" running on each node in the Kubernetes cluster to implement cluster management of nodes; Kube-Scheduler is the Kubernetes scheduling device. Pod is the basis of all business types of Kubernetes and the basic scheduling unit of Kubernetes. It is a combination of one or more containers. These containers share storage, network and namespace, as well as specifications on how to run. In Pod, all containers are uniformly arranged and scheduled and run in a shared context.

[0082] The device of the present invention comprises a GPU Share Scheduler Extender (GPU extended scheduler), a GPU ShareDevice Plugin (GPU shared plug-in) and a GPU Monopoly Device Plugin (GPU exclusive plug-in).

[0083] gpu-mem: GPU memory Kubernetes extended resource name (unit can be MiB, GiB, etc.), suitable for sharing mode. The actual name can be customized to indicate the characteristics of the resource and usage.

[0084] gpu-count: Kubernetes extended resource name for the number of GPU devices (i.e. the number of GPU devices on a node), applicable to exclusive mode. The actual name can be customized to indicate the characteristics of the resource and usage.

[0085] The above-mentioned method for implementing GPU device sharing scheduling and use on the Kubernetes platform, combined with the device and related principles are described as follows.

[0086] like Figure 4 As shown in .ReportDevice()gpu-mem:32552, GPU Share Device Plugin uses the nvml library to query the number of GPU devices and the video memory of each GPU device, and reports the node's total GPU video memory (the sum of the video memory of all GPU devices on the node) as Kubernetes Extended Resource (gpu-mem) to Kubelet (calling method: gRPC); Kubelet further reports to the Kubernetes API Server (calling method: HTTP / HTTPS). For example, if the node has two GPU devices, and each contains 16276MiB, from the user's perspective: the node's GPU video memory resources are 16276*2=32552MiB.

[0087] It should be noted that memory reporting can be considered as a "virtualization" process of the node GPU device, that is, virtualizing the actual GPU device into multiple memory slice devices. The number of memory slice devices that will eventually be virtualized depends on the actual GPU device memory size and the memory unit (GiB, MiB...) used during "virtualization". The GPU virtual memory slice device is also reported to Kubelet, and its identification includes the actual GPU device Uid (UUID format), the actual GPU device subscript (0, 1, 2...), the memory slice number and other information.

[0088] like Figure 4 As shown in .ReportDevice()gpu-count:2, GPU Monopoly Device Plugin reports the actual GPU device information of the node (using the nvml library to query the number of devices) as Kubernetes ExtendedResource(gpu-count) to Kubelet (calling method: gRPC); Kubelet further reports it to the Kubernetes API Server (calling method: HTTP / HTTPS).

[0089] It should be noted that the actual GPU device does not involve virtualization and slicing based on video memory. The device identification includes the actual GPU device Uid (UUID format), the actual GPU device subscript (0, 1, 2...) and other information.

[0090] Among them, Figure 5 The following is a detailed description of the GPU container task expansion scheduling process.

[0091] 1) Kubernetes scheduler pre-selection and GPU extension scheduler further screening

[0092] After performing all filtering actions, the Kubernetes scheduler will call the filter method of GPU Share Scheduler Extender through HTTP / HTTPS. This is because when the Kubernetes scheduler calculates the Extended Resource, it can only determine whether there are idle resources in the total amount of resources that meet the demand, but cannot specifically determine whether the demand is met on a single card; therefore, the GPU Share Scheduler Extender needs to check whether there are available resources on a single card.

[0093] For example, in a Kubernetes cluster consisting of three nodes with two GPU devices, when a user applies for gpu-mem=8138, the Kubernetes scheduling device scans all nodes and finds that the remaining resources of N1 are (16276*2-16276-12207=4069), which does not meet the resource requirements, and the N1 node is filtered out. The remaining resources of N2 and N3 nodes are both 8138 MiB, which meet the screening conditions of the Kubernetes scheduling device from the perspective of overall scheduling; at this time, the Kubernetes scheduling device will entrust the GPU Share Scheduler Extender to perform secondary filtering. In the secondary filtering, the GPU Share Scheduler Extender needs to determine whether a single card meets the scheduling requirements. When checking the N2 node, it is found that although the node has 8138 MiB of available resources, when it falls on each card, GPU0 and GPU1 respectively have only 4069 MiB of available resources, which cannot meet the 8138 MiB requirement of a single card. Although the N3 node also has a total of 8138 MiB of available resources, these available resources all belong to GPU0, which meets the requirement of single-card scheduling. Therefore, accurate condition matching can be achieved through the screening of GPU ShareScheduler Extender.

[0094] In addition, GPU Share Scheduler Extender is also responsible for processing the extended scheduling screening logic of gpu-count. The Kubernetes scheduler will not determine whether the node GPU device has been used by the GPU shared container task when doing pre-selection. This is because the shared and exclusive resource names are different, and the Kubernetes scheduler will schedule the GPU devices in the two usage modes as two resources. When processing container tasks that use the GPU device number in exclusive mode, GPU Share Scheduler Extender will filter out the GPU devices of the nodes used by the GPU shared container tasks (using gpu-mem). Similarly: exclusive container tasks will occupy the entire GPU device, and shared container tasks cannot occupy the GPU devices used by exclusive container tasks (using gpu-count).

[0095] 2) GPU extension scheduler optimization:

[0096] When the Kubernetes scheduler finds a node that meets the conditions, it entrusts the bind method of GPU Share Scheduler Extender (calling method: HTTP / HTTPS) to bind the node and Pod. Here, GPU ShareScheduler Extender needs to do two things:

[0097] a) Find the optimal GPU device id in the node according to different algorithms (such as binpack):

[0098] The optimal meaning here is for different GPU devices on the same node, which is explained below using binpack as an example.

[0099] Sharing: Prioritize the GPU device with the least amount of free resources and the least amount of resources left, and save it as GPU_MEM_IDX in the Pod's Annotation (the key of the Annotation can be customized, assuming the key is GPU_MEM_IDX here); also save the GPU video memory requested by the Pod as GPU_MEM_POD and GPU_ASSUME_TIME in the Pod's Annotation, and bind the Pod to the selected node at this time. (Note: The GPU_MEM_ASSIGNED Pod Annotation is also saved at this time, and it is initialized to "false". It means that the Pod is assigned to a GPU device at the time of scheduling, but the Pod is not actually created on the node. GPU_ASSUME_TIME represents the specified timestamp.)

[0100] Exclusive: Prioritize the node with the least number of available GPU devices but with the idle GPU devices that meet the conditions. Save the selected GPU device ID as GPU_COUNT_IDXS (multiple GPU devices may be occupied at the same time) in the Pod's Annotation (the key of the Annotation can be customized, and it is assumed here that the key is GPU_COUNT_IDXS).

[0101] If it is found during scheduling that no GPU resources on the allocated node meet the conditions, no binding will be performed and the system will exit without error. The Kubernetes scheduling device will reschedule after the assume timeout (i.e., the queuing mechanism).

[0102] The GPU Share Scheduler Extender performs a lock operation during the bind process to ensure that the Pods are bound in sequence in the GPU ShareScheduler Extender. (This is mainly to prevent multiple applications that exceed the GPU device memory from being allocated to the same GPU device, and to ensure the accuracy of the scheduling processing logic).

[0103] b) Call the Kubernetes API to perform node and Pod binding (using binpack as an example):

[0104] Take the sharing mode as an example: when GPU Share Scheduler Extender wants to bind the Pod of gpu-mem:8138 to the selected node N1, it will first compare the available resources of different GPU devices, namely GPU0 (12207), GPU1 (8138), GPU2 (4069), and GPU3 (16276). The remaining resources of GPU2 do not meet the demand and are discarded. Among the other three GPU devices that meet the conditions, GPU1 is the GPU device that meets the free resource requirements but has the least remaining resources, so GPU1 is selected.

[0105] In this embodiment, the GPU container task runs on the node.

[0106] When the event of binding the Pod and the node is received by Kubelet, Kubelet will create a real Pod entity on the node. During this process, Kubelet will call the Allocate method of GPU Share Device Plugin (shared) or GPU MonopolyDevice Plugin (exclusive). The parameter of the Allocate method is the gpu-mem (shared: GPU video memory) or gpu-count (exclusive: number of GPU devices) requested by the Pod.

[0107] In the Allocate method, the corresponding Pod will be run according to the scheduling decision of GPU Share Scheduler Extender. The detailed process is as follows.

[0108] 1) Sharing:

[0109] Kubelet allocates GPU memory (GPU virtual memory slice device: gpu-mem) for container tasks, and obtains the GPU device index of GPU Share Scheduler Extender for Pod bind from PodAnnotation (GPU_MEM_IDX). Then, it filters out the virtual memory slices of the GPU device that matches the GPU_MEM_IDX mark from the allocatable GPU virtual memory slice device set, and allocates the GPU virtual memory slice device that meets the container memory requirements from the filtered set to the container task.

[0110] It should be noted that modifying some Kubelet codes and adding this new processing logic (without affecting other Kubernetes native logic) will only take effect when processing gpu-mem Extended Resource, because the native implementation of Kubelet randomly allocates from the allocatable Extended Resource set, so it is possible to allocate virtual video memory slices that do not match the GPU device marked by GPU_MEM_IDX. In other words, the device allocation has been completed in the GPU Share SchedulerExtender bind method, and there is no need for Kubelet to allocate it again.

[0111] Kubelet calls the Allocate method of the GPU Share Device Plugin (Kubernetes native processing logic, gRPC method) to allocate GPU devices for GPU container tasks.

[0112] The GPU Share Device Plugin lists all GPU shared Pods in the node whose status is Pending and whose GPU_MEM_ASSIGNED is false.

[0113] GPU Share Device Plugin selects the Pods whose GPU_MEM_IDX in the Pod Annotation is exactly the same as the GPU device subscript recorded in the GPU virtual memory shard device identifier allocated by Kubelet, and the memory size requested by the container in the Pod is consistent with the number of allocated GPU virtual memory shards. If there are multiple Pods that meet this condition, the Pod with the earliest GPU_ASSUME_TIME will be selected.

[0114] Set the Pod's Annotation GPU_MEM_ASSIGNED to true, and convert the allocated GPU virtual memory slice device (the actual GPU device Uid) into an environment variable and return it to Kubelet for actual Pod creation.

[0115] Exclusive:

[0116] Kubelet allocates GPU devices (actual GPU devices: gpu-count) to container tasks, and obtains the GPU device index set (exclusive to use multiple GPU devices at the same time) for Pod bind by GPU Share Scheduler Extender from Pod Annotation (GPU_COUNT_IDXS). Then, the GPU devices matching the GPU_COUNT_IDXS mark are selected from the actual GPU device set that can be allocated, and are allocated to container tasks.

[0117] It should be noted that modifying some Kubelet codes and adding this new processing logic (without affecting other Kubernetes native logic) will only take effect when processing gpu-count Extended Resource, because the native implementation of Kubelet randomly allocates from the allocatable Extended Resource set, so it is possible to allocate devices that do not match the GPU device marked by GPU_COUNT_IDXS. In other words, the device allocation has been completed in the GPU Share SchedulerExtender bind method, and there is no need for Kubelet to allocate again.

[0118] Kubelet calls the Allocate method of the GPU Monopoly Device Plugin (Kubernetes native processing logic, gRPC method) to allocate GPU devices for GPU container tasks.

[0119] The GPU Monopoly Device Plugin Allocate method converts the allocated GPU device (gets the actual GPU device UID) into an environment variable and returns it to Kubelet for actual Pod creation. Exclusive use does not perform "virtualization" sharding according to video memory, so no matching is required here.

[0120] To ensure the neatness of the Kubernetes native code, the modifications made to Kubelet in this solution are solidified in git Patch mode. When it is necessary to build a Kubelet program adapted to the device of the present invention, the Patch is submitted to the temporary file required for compilation before compiling Kubernetes (if you need to compile the Kubernetes native Kubelet program, you can choose not to merge). Under normal circumstances (such as daily management of the Kubernetes code base), the code maintains the community native content.

[0121] In this embodiment, based on GPU memory allocation, multiple containers and Pods in Kubernetes (GPU memory required for containers) share the same GPU device. Multiple containers include multiple containers in the same Pod and multiple containers in different Pods. GPU memory includes virtualizing the actual GPU device into a GPU device virtual memory shard device, and maintaining the actual GPU device Uid, subscript and other information and memory shard number information in the shard device identifier.

[0122] A scheduling device-algorithm device provides scheduling algorithms with multiple shared and exclusive usage modes (including compatible scheduling when shared and exclusive use exist at the same time), wherein the algorithm device is integrated into the scheduling device in the form of a plug-in or other more flexible methods.

[0123] The shared / exclusive mode GPU device allocation is directly completed based on the scheduling device without the intervention of Kubelet; the exclusive mode includes adding the GPU device index information to the GPU device identifier reported by the GPU exclusive plug-in.

[0124] Supports queuing of container tasks using GPU devices. After the scheduled container tasks are completed, it supports automatically scheduling the container tasks waiting in the queue to the nodes and GPU devices.

[0125] Based on Kubernetes Annotation, record GPU device allocation information (device index, video memory, etc.) and allocation status (whether allocation is completed, various timestamps, etc.).

[0126] Kubelet is modified to adapt to the allocation of GPU memory and devices.

[0127] It supports both shared and exclusive usage, and users can directly use either method without having to control it through switches or other means.

[0128] The embodiments provided in this application have the following advantages:

[0129] 1) Based on GPU memory allocation, multiple containers in Kubernetes (set the GPU memory required for the container) can share the same GPU device. The GPU plug-in can automatically obtain the number of GPU devices and memory information of the server, and automatically register the number of GPU devices and memory information of the server with the Kubernetes cluster. Users do not need to manually configure the upper limit of the number of container tasks that share GPU resources for each GPU device of the server. Control based on memory granularity is also closer to actual usage, which can help users more intuitively evaluate the scenario requirements of multiple applications sharing the same GPU device.

[0130] 2) The scheduling algorithm can support multiple sharing modes at the same time, and the scheduling strategy is more intelligent.

[0131] 3) Supports queuing of container tasks using GPU devices. When the scheduled container tasks are completed, it supports automatically scheduling the container tasks waiting in the queue to the GPU, and the user does not need to manually submit the container tasks again.

[0132] 4) It supports both shared and exclusive use. Users can directly use either method without having to set whether to enable sharing through a switch.

[0133] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0134] According to another aspect of the embodiments of the present invention, a resource scheduling device for implementing the above resource scheduling method is also provided. Figure 6 As shown, the resource scheduling device includes: a first acquisition unit 61, a second acquisition unit 63 and a determination unit 65.

[0135] The first acquisition unit 61 is used to acquire identification information of a GPU of a server, wherein the identification information is used to identify a scheduling mode of the GPU.

[0136] The second acquisition unit 63 is used to acquire resource information corresponding to the identification information.

[0137] The determination unit 65 is used to determine the GPU scheduling method according to the resource information.

[0138] Through the embodiment provided by the present application, the first acquisition unit 61 acquires the identification information of the GPU of the server, wherein the identification information is used to identify the scheduling mode of the GPU; the second acquisition unit 63 acquires the resource information corresponding to the identification information; the determination unit 65 determines the scheduling mode of the GPU according to the resource information. The purpose of determining the scheduling mode according to the resource information reported by the server is achieved, that is, the scheduling mode of the GPU is first determined according to the resource information, and the scheduling process of the GPU is further determined according to the scheduling mode. At the same time, both shared and exclusive use modes are supported. Users can directly use any mode without having to set whether to open sharing through a switch, thereby solving the technical problem of low efficiency of GPU resource scheduling mode in the prior art.

[0139] Optionally, the above device may include:

[0140] The marking unit is used to mark the target identification information of the GPU according to the GPU resource scheduling mode before obtaining the identification information of the GPU of the server, wherein the scheduling mode includes a sharing mode and an exclusive mode, wherein the target identification information includes identification information.

[0141] Optionally, the second obtaining unit 63 may include:

[0142] The first acquisition module is used to acquire the video memory information of the GPU device of the server when the GPU resource scheduling mode of the server is a sharing mode, and the resource information includes the video memory information of the GPU device of the server;

[0143] The second acquisition module is used to obtain the server GPU device number information reported by the GPU when the GPU resource scheduling mode of the server is the exclusive mode, and the resource information includes the server GPU device number information.

[0144] Optionally, the determining unit 65 may include:

[0145] A first determination module is used to determine a first server GPU to be scheduled when the GPU resource scheduling mode of the server is a sharing mode, wherein the first server GPU satisfies the task requirement that the available total video memory meets;

[0146] The second determination module is used to determine the second server GPU to be scheduled when the GPU resource scheduling mode of the server is the exclusive mode, wherein the second server GPU meets the available device number memory to meet the task requirement.

[0147] A third determination module is used to determine a third server GPU to be scheduled when the GPU resource scheduling mode of the server is a sharing mode, wherein the third server GPU satisfies the GPU device whose idle resources meet the task conditions and has the least remaining resources;

[0148] The fourth determination module is used to determine the fourth server GPU to be scheduled when the GPU resource scheduling mode of the server is the exclusive mode, wherein the fourth server GPU satisfies the idle GPU device that meets the task condition and has the least remaining available GPU devices.

[0149] Optionally, the above device may include:

[0150] The scheduling unit is used to allocate multiple GPU container tasks to the same GPU device of the same node after determining the first server GPU to be scheduled, and give priority to allocating GPU devices to the container tasks with the earliest scheduling process completion timestamp.

[0151] Optionally, the above device is further used to perform the following operation: after determining the second server GPU to be scheduled, if the task before the second server GPU is scheduled is not completed, the current task enters the scheduling queue and the scheduling is retried until the second server GPU is determined.

[0152] According to another aspect of the embodiment of the present invention, an electronic device for implementing the above resource scheduling method is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a server as an example. Figure 7 As shown, the electronic device includes a memory 702 and a processor 704. The memory 702 stores a computer program, and the processor 704 is configured to execute the steps in any of the above method embodiments through the computer program.

[0153] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0154] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0155] S1, obtaining identification information of a GPU of a server, wherein the identification information is used to identify a scheduling mode of the GPU;

[0156] S2, obtaining resource information corresponding to the identification information;

[0157] S3, determining a GPU scheduling method according to the resource information.

[0158] Alternatively, a person skilled in the art may understand that: Figure 7 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, and other terminal devices. Figure 7 The electronic device and the electronic equipment described above are not limited in structure. Figure 7 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 7 Different configurations are shown.

[0159] Among them, the memory 702 can be used to store software programs and modules, such as the program instructions / modules corresponding to the resource scheduling method and device in the embodiment of the present invention. The processor 704 executes various functional applications and data processing by running the software programs and modules stored in the memory 702, that is, realizing the above-mentioned resource scheduling method. The memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 702 may further include a memory remotely located relative to the processor 704, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 702 can be specifically, but not limited to, used for identification information, resource information, and other information. As an example, such as Figure 7 As shown, the memory 702 may include but is not limited to the acquisition unit 61, the second acquisition unit 63 and the determination unit 65 in the resource scheduling device. In addition, it may also include but is not limited to other module units in the resource scheduling device, which will not be repeated in this example.

[0160] Optionally, the transmission device 706 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 706 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 706 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0161] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. Among them, the nodes may form a peer-to-peer (P2P, Peer To Peer) network, and any form of computing device, such as a server, terminal and other electronic devices, may become a node in the blockchain system by joining the peer-to-peer network.

[0162] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the resource scheduling method provided in the above-mentioned determination of GPU resource scheduling or various optional implementations of the determination of GPU resource scheduling. The computer program is configured to execute the steps of any of the above-mentioned method embodiments when it is run.

[0163] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0164] S1, obtaining identification information of a GPU of a server, wherein the identification information is used to identify a scheduling mode of the GPU;

[0165] S2, obtaining resource information corresponding to the identification information;

[0166] S3, determining a GPU scheduling method according to the resource information.

[0167] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0168] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0169] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

[0170] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0171] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0172] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0174] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A resource scheduling method, characterized in that: include: Obtaining identification information of a GPU of a server, wherein the identification information is used to identify a scheduling mode of the GPU; Obtain resource information corresponding to the identification information; Determining a scheduling method of the GPU according to the resource information; The acquiring of resource information corresponding to the identification information includes: when the GPU resource scheduling mode of the server is a shared mode, acquiring video memory information of the GPU device of the server, wherein the resource information includes the video memory information of the GPU device of the server; when the GPU resource scheduling mode of the server is an exclusive mode, acquiring information on the number of GPU devices of the server reported by the GPU, wherein the resource information includes information on the number of GPU devices of the server; The determining the scheduling mode of the GPU according to the resource information includes: when the GPU resource scheduling mode of the server is a shared mode, determining a first server GPU to be scheduled, wherein the first server GPU satisfies the available total video memory to meet the task requirements; when the GPU resource scheduling mode of the server is an exclusive mode, determining a second server GPU to be scheduled, wherein the second server GPU meets the task requirements of the available device; When the GPU resource scheduling mode of the server is a shared mode, a third server GPU is determined to be scheduled, wherein the third server GPU satisfies the requirement that idle resources meet the task conditions and the GPU device has the least remaining resources; when the GPU resource scheduling mode of the server is an exclusive mode, a fourth server GPU is determined to be scheduled, wherein the fourth server GPU satisfies the requirement that idle GPU devices meet the task conditions and the node has the least remaining available GPU devices.

2. The method according to claim 1, characterized in that Before obtaining the identification information of the GPU of the server, the method includes: The target identification information of the GPU is marked according to the GPU resource scheduling mode, wherein the scheduling mode includes a shared mode and an exclusive mode, and wherein the target identification information includes the identification information.

3. The method according to claim 1, characterized in that After determining the first server GPU to be scheduled, the method includes: Assign multiple GPU container tasks to the same GPU device on the same node, and give priority to assigning GPU devices to container tasks with the earliest scheduling process completion timestamp.

4. The method according to claim 1, characterized in that: After determining the second server GPU to be scheduled, the method further includes: In the case that the task before the second server GPU is scheduled is not completed, the current task enters the scheduling queue and is retried until the second server GPU is determined.

5. A resource scheduling device, characterized in that: include: A first acquisition unit, configured to acquire identification information of a GPU of a server, wherein the identification information is used to identify a scheduling mode of the GPU; A second acquisition unit, used to acquire resource information corresponding to the identification information; A determining unit, configured to determine a scheduling mode of the GPU according to the resource information; The second acquisition unit includes: a first acquisition module, used to acquire the video memory information of the GPU device of the server when the GPU resource scheduling mode of the server is a shared mode, and the resource information includes the video memory information of the GPU device of the server; a second acquisition module, used to acquire the number of GPU devices of the server reported by the GPU when the GPU resource scheduling mode of the server is an exclusive mode, and the resource information includes the number of GPU devices of the server; The determination unit includes: a first determination module, used to determine a first server GPU to be scheduled when the GPU resource scheduling mode of the server is a shared mode, wherein the first server GPU satisfies the available total video memory to meet the task requirement; a second determination module, used to determine a second server GPU to be scheduled when the GPU resource scheduling mode of the server is an exclusive mode, wherein the second server GPU meets the task requirement of the available device; The third determination module is used to determine the third server GPU to be scheduled when the GPU resource scheduling mode of the server is a shared mode, wherein the third server GPU satisfies the requirement that idle resources meet the task conditions and the GPU device has the least remaining resources; the fourth determination module is used to determine the fourth server GPU to be scheduled when the GPU resource scheduling mode of the server is an exclusive mode, wherein the fourth server GPU satisfies the requirement that idle GPU devices meet the task conditions and the node has the least remaining available GPU devices.

6. The device according to claim 5, characterized in that The device comprises: A marking unit is used to mark the target identification information of the GPU according to the GPU resource scheduling mode before obtaining the identification information of the GPU of the server, wherein the scheduling mode includes a shared mode and an exclusive mode, and wherein the target identification information includes the identification information.

7. The device according to claim 5, characterized in that The device comprises: The scheduling unit is used to allocate multiple GPU container tasks to the same GPU device of the same node after determining the first server GPU to be scheduled, and preferentially allocate GPU devices to container tasks with the earliest scheduling process completion timestamp.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 4 when executed.

9. An electronic device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 4 through the computer program.

Images