Automatic scheduling method for virtual machine GPU (Graphics Processing Unit) under cluster

Abstract

The invention provides a method for automatically scheduling a GPU (Graphics Processing Unit) of a virtual machine under a cluster, which comprises the following steps of: establishing a GPU resource pool and a vGPU (Virtual Graphics Processing Unit) resource pool by utilizing a GPU management device, virtualizing a physical GPU into a vGPU and marking a type label; the GPU resource monitoring device is used for carrying out statistics on the GPU resource use condition and the vGPU resource use condition of the server and reporting the statistics results to the GPU management device so as to screen out a GPU resource pool and a vGPU resource pool; the GPU management device selects a target server for the to-be-scheduled virtual machine from the screened GPU resource pool and vGPU resource pool according to a configured scheduling strategy and issues the target server to a corresponding GPU resource monitoring device, so that the to-be-scheduled virtual machine is scheduled to a GPU display card in the target server; the management difficulty is reduced, the situation that the virtual machine fails to start up due to the fact that the GPU resources cannot be found is reduced, and meanwhile the utilization rate of the GPU resources is increased.

Description

Technical Field

[0001] The present invention relates to the technical field of computer networks, and in particular to a method for automatically scheduling GPUs of virtual machines in a cluster. Background Art

[0002] GPU (Graphic Processing Unit), the full name of which is computer image processor, has a huge advantage in deep learning, scientific computing, image rendering and other fields due to its parallel computing capability. In particular, the rapid development of machine learning and deep learning in recent years has accelerated the pace of moving heterogeneous computing scenarios to the cloud. Effective management and utilization of GPU resources are crucial to providing a high-performance computing environment, and managing GPU resources has become an issue that developers are increasingly concerned about solving.

[0003] Currently, GPU virtualization technology can be roughly divided into three solutions: pure software simulation of virtual graphics card, physical GPU transparent transmission, and GPU virtualization; each has its own advantages and disadvantages. Device simulation is to use software to simulate hardware, physical graphics card transparent transmission allocates the GPU independently to the virtual machine for use, and GPU virtualization modifies the virtual machine monitor (VMM, Virtual Machine Monitor) to manage the GPU.

[0004] GPU pass-through is also called graphics card pass-through. The pass-through device uses virtualization technology (VT-d, Virtualization Technology for Direct) to pass the GPU core directly to a single virtual machine for use. It has the advantages of good performance and complete functions; but its disadvantage is that it cannot be shared.

[0005] Virtual GPU (vGPU) allows multiple virtual machine instances to share the same physical GPU and provides each virtual machine with a certain proportion of GPU resources. By using vGPU, GPU resources can be reasonably allocated and managed in a virtualized environment.

[0006] GPU virtualization is to divide a physical GPU into multiple virtual GPUs for use by different virtual machines (GPU partitioning). NVIDIA and AMD use different technologies for GPU virtualization.

[0007] NVIDIA vGPU solution (soft partitioning) is a GPU resource sharing method based on virtualization technology. It does not partition at the hardware level, but completes the partitioning through the virtualization layer and the driver in the virtual machine. Through vGPU technology, a physical GPU can be virtualized into multiple virtual GPUs, each of which can independently run an operating system and application. This technology can share GPU resources among multiple users and improve resource utilization.

[0008] AMD MxGPU (hard partitioning) is a GPU resource sharing method based on hardware virtualization technology. Through MxGPU technology, a physical GPU can be partitioned into multiple virtual GPUs, each of which can independently run an operating system and applications. Unlike vGPU, MxGPU technology uses hardware virtualization technology to achieve resource isolation and exclusive access, ensuring the independence and security of each virtual GPU.

[0009] Although there are many ways for virtual machines to use GPUs, managing them on a virtual machine basis requires professional operation and maintenance personnel to manage and allocate GPU resources, which is difficult and can easily lead to serious waste of host machine resources, repeated allocation, and even virtual machine startup failure. Summary of the invention

[0010] The present invention provides a method for automatically scheduling GPUs of virtual machines in a cluster, which aims to reduce the difficulty of management, reduce the situation where virtual machines fail to start up due to not being able to find GPU resources, and improve the utilization rate of GPU resources.

[0011] In order to achieve the above-mentioned object, the present invention provides a method for automatic scheduling of GPUs of virtual machines in a cluster, which is applied to a cloud infrastructure platform. The cloud infrastructure platform includes a cluster consisting of multiple servers, a GPU management device, and multiple GPU resource monitoring devices, where one GPU resource monitoring device corresponds to one server;

[0012] Automatic scheduling methods include:

[0013] Step 1: Based on the GPU resources of each server in the cluster, a GPU management device is used to form multiple GPU resource pools and multiple vGPU resource pools, and physical GPUs are virtualized into vGPUs according to usage scenarios and each vGPU is labeled with a type;

[0014] Step 2: for each server in the cluster, using a GPU resource monitoring device corresponding to the server to collect statistics on the GPU resource usage and vGPU resource usage of the server, obtain a first statistical result and a second statistical result of each server, and report them to the GPU management device;

[0015] Step 3: The GPU management device selects all GPU resource pools and vGPU resource pools that meet the requirements of the virtual machines to be scheduled from multiple GPU resource pools according to the first statistical result, the second statistical result, and the type label of each server;

[0016] Step 4: The GPU management device selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements according to the configured scheduling policy, and sends the selection result to the GPU resource monitoring device corresponding to the target server;

[0017] Step 5: The GPU resource monitoring device schedules the virtual machine to be scheduled to the GPU graphics card in the target server according to the type label and the scoring rule.

[0018] Furthermore, the automatic scheduling method also includes:

[0019] Divide the servers in the cluster into a control node and multiple computing nodes;

[0020] The GPU management device is deployed on the control node and is used to manage all GPU resource pools, all vGPU resource pools, and type tags, and select a target server for the virtual machine to be scheduled;

[0021] The GPU resource monitoring device is deployed on the computing node. The GPU resource monitoring device corresponds to the computing node one by one. It is used to collect statistics on the GPU resource usage and vGPU resource usage of each server, report the first statistical result and the second statistical result of each server to the GPU management device, and schedule the virtual machine to be scheduled to the GPU graphics card in the target server.

[0022] Furthermore, physical GPUs are virtualized into vGPUs according to usage scenarios and each vGPU is labeled with a type, including:

[0023] Virtualize physical GPUs according to usage scenarios to obtain various types of vGPUs, including cloud desktop rendering, graphics rendering, and high-performance computing;

[0024] The vGPU of cloud desktop rendering type is labeled as type 1;

[0025] The vGPU of the image rendering type is labeled as the second type;

[0026] The high-performance computing type vGPU is labeled with the third type.

[0027] Specifically, the server's GPU resource usage and vGPU resource usage include: GPU graphics card utilization, GPU memory utilization, temperature, number of vGPU idle nodes, number of running tasks, and power consumption.

[0028] Specifically, the GPU management device selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements according to the configured scheduling policy, including:

[0029] The GPU management device selects the configured affinity rule as the scheduling strategy, and selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements;

[0030] or,

[0031] The GPU management device selects the configured anti-affinity rule as the scheduling strategy, and selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements.

[0032] Specifically, the affinity rule is: when GPU resources are allowed to be selected, the virtual machines to be scheduled are required to run on the same specified resources, including GPU resource pool affinity, server affinity, and GPU graphics card affinity.

[0033] Specifically, the anti-affinity rule is: when GPU resources are allowed to be selected, it is stipulated that the virtual machine to be scheduled does not run on the same specified resource, including GPU resource pool anti-affinity, server anti-affinity, and GPU graphics card anti-affinity.

[0034] Specifically, selecting a target server for a virtual machine to be scheduled includes:

[0035] According to the server scoring expression Mark1 = HGu*HW0+HGu*HW1+HGn*HW2+HGvn*HW3 and the resource usage in the GPU resource pool and the vGPU resource pool, the servers in the GPU resource pool and the vGPU resource pool are scored to obtain the server with the highest score, and the server with the highest score is used as the target server;

[0036] Among them, HGu represents the average GPU graphics card utilization in the same server, HGm represents the average GPU memory utilization in the same server, HGn represents the number of GPU graphics cards installed in the same server, HGvn represents the total number of unallocated vGPUs in the same server, and HW0, HW1, HW2, and HW3 all represent weights.

[0037] More specifically, step 5 includes:

[0038] The GPU resource monitoring device determines all GPU graphics cards in the target server that meet the requirements of the virtual machine to be scheduled according to the type label;

[0039] Design the graphics card scoring expression Mark2=G according to the scoring rules p *W0+G t *W1+G d *W2 scores the load, temperature, and power consumption of all GPU graphics cards that meet the requirements of the virtual machines to be scheduled, obtains the GPU graphics card with the lowest score as the target graphics card, and schedules the virtual machines to be scheduled to the target graphics card;

[0040] Among them, G p Indicates the GPU graphics card load, G t Indicates the temperature of the GPU graphics card, G d Indicates the power consumption of the GPU graphics card. W0, W1, and W2 all represent weights.

[0041] The above scheme of the present invention has the following beneficial effects:

[0042] The present invention is applied to a cloud infrastructure platform, which includes a cluster consisting of multiple servers, a GPU management device, and multiple GPU resource monitoring devices; the GPU management device is used to form multiple GPU resource pools and multiple vGPU resource pools, and a physical GPU is virtualized into a vGPU and labeled with a type; the GPU resource monitoring device is used to count the GPU resource usage and the vGPU resource usage of the server, and a first statistical result and a second statistical result are obtained and reported to the GPU management device, so as to screen out all GPU resource pools and vGPU resource pools that meet the requirements of virtual machines to be scheduled from the multiple GPU resource pools; the GPU management device selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements according to the configured scheduling strategy. , and sends the selection result to the GPU resource monitoring device corresponding to the target server; the GPU resource monitoring device schedules the virtual machine to be scheduled to the GPU graphics card in the target server according to the type label and scoring rule; compared with the prior art, the present invention takes a cluster composed of multiple servers as the dimension, and dynamically schedules the virtual machine to be scheduled to the optimal server according to the resource usage of all GPU resource pools and vGPU resource pools in the cluster, and does not require operation and maintenance personnel to understand the GPU and vGPU layout in the entire cloud infrastructure platform, thereby reducing the problem of inefficient performance caused by incorrect vGPU and GPU layout of virtual machines by operation and maintenance personnel, reducing the difficulty of management, reducing the situation where virtual machines fail to start up due to not being able to find GPU resources, and greatly improving the utilization rate of GPU resources.

[0043] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention;

[0045] Figure 2 is an architecture diagram of a cloud infrastructure platform in an embodiment of the present invention;

[0046] Figure 3 The diagram is a schematic diagram of a resource allocation process in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0049] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a locking connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0051] In view of the existing problems, the present invention provides a method for automatically scheduling GPUs of virtual machines in a cluster.

[0052] like Figure 1-2 As shown, an embodiment of the present invention provides a method for automatic GPU scheduling of virtual machines in a cluster, which is applied to a cloud infrastructure platform. The cloud infrastructure platform includes a cluster consisting of multiple servers, a GPU management device, and multiple GPU resource monitoring devices, where one GPU resource monitoring device corresponds to one server;

[0053] Automatic scheduling methods include:

[0054] Step 1: The GPU management device forms multiple GPU resource pools and multiple vGPU resource pools based on the GPU resources of each server in the cluster, and virtualizes the physical GPU into vGPU according to the usage scenario and labels each vGPU with a type label;

[0055] Step 2: for each server in the cluster, using a GPU resource monitoring device corresponding to the server to collect statistics on the GPU resource usage and vGPU resource usage of the server, obtain a first statistical result and a second statistical result of each server, and report them to the GPU management device;

[0056] Step 3: The GPU management device selects all GPU resource pools and vGPU resource pools that meet the requirements of the virtual machines to be scheduled from multiple GPU resource pools according to the first statistical result, the second statistical result, and the type label of each server;

[0057] Step 4: The GPU management device selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements according to the configured scheduling policy, and sends the selection result to the GPU resource monitoring device corresponding to the target server;

[0058] Step 5: The GPU resource monitoring device schedules the virtual machine to be scheduled to the GPU graphics card in the target server according to the type label and the scoring rule.

[0059] Specifically, the cloud infrastructure platform also includes:

[0060] Cluster manager to enable centralized management of multiple target servers;

[0061] Enable the cluster function in the cluster manager and make sure all servers have joined the cluster;

[0062] Enable the GPU automatic scheduling function of the cluster in the cluster manager to enable it to automatically manage and schedule GPU resources.

[0063] In an embodiment of the present invention, a cluster is a unified computer composed of multiple servers with the same functions, which implement the same business and provide computing, network, storage and other resources to the outside world, and has the characteristics of scalability and high availability. The GPU resource pool and the vGPU resource pool are obtained by pooling the GPU resources of the server according to the virtualization type. The servers in the GPU resource pool and the vGPU resource pool can be the resource group of all servers in the cluster or the resource group of some servers in the cluster. For the vGPU resource pool, the physical GPU resources of different manufacturers are organized into a relatively independent vGPU resource pool. For the GPU resource pool, the physical GPUs of different manufacturers can be organized into a GPU resource pool. The advantage of GPU resource pooling is on-demand allocation, unlimited resources, and a virtual resource pool that can be expanded / reduced as needed.

[0064] Specifically, the automatic scheduling method also includes:

[0065] Divide the servers in the cluster into a control node and multiple computing nodes;

[0066] The GPU management device is deployed on the control node and is used to manage all GPU resource pools, all vGPU resource pools, and type tags, and select a target server for the virtual machine to be scheduled;

[0067] The GPU resource monitoring device is deployed on the computing node. The GPU resource monitoring device corresponds to the computing node one by one. It is used to collect statistics on the GPU resource usage and vGPU resource usage of each server, report the first statistical result and the second statistical result of each server to the GPU management device, and schedule the virtual machine to be scheduled to the GPU graphics card in the target server.

[0068] In the embodiment of the present invention, the servers in the cluster are divided into one control node and multiple computing nodes according to their functions. One control node corresponds to one server, and one computing node corresponds to one server.

[0069] It should be noted that the GPU management device is a process or program installed on the control node, which is responsible for managing the GPU resource pool and vGPU resource pool of the entire cluster, managing the type labels of GPU resources, and mainly classifying and managing them according to the applicable types of GPU resources. Virtual machines of different business types are selected and scheduled to the corresponding GPU resource pool, and the target server is selected for the virtual machine to be scheduled to maximize the performance of the GPU; the GPU resource monitoring device is a process or program installed on the computing node, which is used to realize the statistics of the GPU resource usage and vGPU resource usage of the server, report the first statistical results and the second statistical results of each server to the GPU management device, and schedule the virtual machine to be scheduled to the GPU graphics card in the target server.

[0070] Specifically, physical GPUs are virtualized into vGPUs according to usage scenarios and each vGPU is labeled with a type, including:

[0071] Virtualize physical GPUs according to usage scenarios to obtain various types of vGPUs, including cloud desktop rendering, graphics rendering, and high-performance computing;

[0072] The vGPU of cloud desktop rendering type is labeled as type 1;

[0073] The vGPU of the image rendering type is labeled as the second type;

[0074] The high-performance computing type vGPU is labeled with the third type.

[0075] In the embodiment of the present invention, the first type tag is "A", the second type tag is "Q", and the third type tag is "C".

[0076] Most preferably, the GPU resource usage and vGPU resource usage of the server include: GPU graphics card utilization, GPU memory utilization, temperature, number of vGPU idle times, number of running tasks, and power consumption.

[0077] Most preferably, the GPU management device selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements according to the configured scheduling policy, including:

[0078] The GPU management device selects the configured affinity rule as the scheduling strategy, and selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements;

[0079] or

[0080] The GPU management device selects the configured anti-affinity rule as the scheduling strategy, and selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements.

[0081] In an embodiment of the present invention, different vGPU resource pools can be established in a cluster, and different scheduling strategies, affinity and anti-affinity can be set for the vGPU resource pools; if the affinity rule is selected, the virtual machine to be scheduled will be preferentially scheduled to the specified vGPU resource pool or the specified server or the specified GPU graphics card; if the anti-affinity rule is selected, the set GPU resource pool, server, and GPU graphics card will be automatically filtered out during scheduling, and after the GPU intelligent scheduling function is turned on for the virtual machine to be scheduled, it can be automatically scheduled to the corresponding vGPU resource pool according to the usage scenario.

[0082] Specifically, the affinity rules are: when GPU resources are allowed to be selected, it is stipulated that the virtual machines to be scheduled should run on the same specified resources, including GPU resource pool affinity, server affinity, and GPU graphics card affinity;

[0083] The anti-affinity rule is: when GPU resources are allowed to be selected, it is stipulated that the virtual machine to be scheduled should not run on the same specified resource, including GPU resource pool anti-affinity, server anti-affinity, and GPU graphics card anti-affinity.

[0084] In an embodiment of the present invention, the premise for enabling the GPU automatic scheduling function of the virtual machine to be scheduled is that the cluster has enabled the GPU intelligent scheduling function and has been formed into vGPU resource pools of different types. When the virtual machine to be scheduled is created, the GPU type label, the number of graphics cards, and the policy rules are selected according to the business type; when the virtual machine to be scheduled is started, the GPU resources are requested from the GPU management device according to the configured rules; after the GPU management device receives the message from the virtual machine to be scheduled requesting GPU resources, it preliminarily screens out all GPU resource pools that meet the requirements according to the GPU type label. If the requirements are not met, the GPU resource pool without the type label is returned, indicating that the startup of the virtual machine to be scheduled has failed, and prompting whether to restart Select a new GPU type tag; after obtaining a GPU resource pool that meets the requirements, select a scheduling strategy, such as enabling affinity rules, to query whether the resources scheduled to the specified vGPU resource pool or the specified server or the specified GPU graphics card are met. If not, it returns that the selected scheduling strategy does not meet the resource requirements. Select to enable anti-affinity rules to filter out the specified vGPU resource pool or the specified server or the specified GPU graphics card resources, and query whether the resources are met in the remaining GPU resource pools. If not, it returns that the selected scheduling strategy does not meet the resources. Please select other strategies or cancel the strategy; after the scheduling strategy is passed, select the best server in the GPU resource pool as the target server.

[0085] Most preferably, selecting a target server for the virtual machine to be scheduled includes:

[0086] According to the server scoring expression Mark1 = HGu*HW0+HGm*HW1+HGn*HW2+HGvn*HW3 and the resource usage in the GPU resource pool and the vGPU resource pool, the servers in the GPU resource pool and the vGPU resource pool are scored to obtain the server with the highest score, and the server with the highest score is used as the target server;

[0087] Among them, HGu represents the average GPU graphics card utilization in the same server, HGm represents the average GPU memory utilization in the same server, HGn represents the number of GPU graphics cards installed in the same server, HGvn represents the total number of unallocated vGPUs in the same server, and HW0, HW1, HW2, and HW3 all represent weights.

[0088] Most preferably, step 5 comprises:

[0089] The GPU resource monitoring device determines all GPU graphics cards in the target server that meet the requirements of the virtual machine to be scheduled according to the type label;

[0090] Design the graphics card scoring expression Mark2=G according to the scoring rules p *W0+G t *W1+Gd *W2 scores the load, temperature and power consumption of all GPU graphics cards that meet the requirements of the virtual machines to be scheduled, obtains the GPU graphics card with the lowest score as the target graphics card, and schedules the virtual machines to be scheduled to the target graphics card. The purpose is to dynamically load balance all GPU resources on the target server and distribute them to different GPU graphics cards;

[0091] Among them, G p Indicates the GPU graphics card load, G t Indicates the temperature of the GPU graphics card, G d Indicates the power consumption of the GPU graphics card. W0, W1, and W2 all represent weights.

[0092] In the embodiment of the present invention, the temperature of the GPU graphics card is mainly measured by calculating the current change rate of 1 hour, that is, (current point temperature - previous hour temperature) / 100;

[0093] The power consumption of GPU graphics cards is mainly measured by calculating the rate of change from the current point in one hour, that is, (power consumption at the current point - power consumption in the previous hour) / maximum power consumption.

[0094] The embodiment of the present invention is applied to a cloud infrastructure platform, which includes a cluster consisting of multiple servers, a GPU management device, and multiple GPU resource monitoring devices; the GPU management device is used to form multiple GPU resource pools and multiple vGPU resource pools, and the physical GPU is virtualized into a vGPU and labeled with a type; the GPU resource monitoring device is used to count the GPU resource usage and the vGPU resource usage of the server, and a first statistical result and a second statistical result are obtained and reported to the GPU management device, so as to screen out all GPU resource pools and vGPU resource pools that meet the requirements of the virtual machines to be scheduled from the multiple GPU resource pools; the GPU management device selects a target server for the virtual machines to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements according to the configured scheduling policy. , and sends the selection result to the GPU resource monitoring device corresponding to the target server; the GPU resource monitoring device schedules the virtual machine to be scheduled to the GPU graphics card in the target server according to the type label and scoring rule; compared with the prior art, the embodiment of the present invention takes a cluster composed of multiple servers as the dimension, and dynamically schedules the virtual machine to be scheduled to the optimal server according to the resource usage of all GPU resource pools and vGPU resource pools in the cluster, and does not require operation and maintenance personnel to understand the GPU and vGPU layout in the entire cloud infrastructure platform, thereby reducing the problem of inefficient performance caused by incorrect vGPU and GPU layout of virtual machines by operation and maintenance personnel, reducing the difficulty of management, reducing the situation where virtual machines fail to start up due to not being able to find GPU resources, and greatly improving the utilization rate of GPU resources.

[0095] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for automatically scheduling GPUs of virtual machines in a cluster, characterized in that: Applied to a cloud infrastructure platform, the cloud infrastructure platform includes a cluster consisting of multiple servers, a GPU management device, and multiple GPU resource monitoring devices, one GPU resource monitoring device corresponding to one server; The automatic scheduling method comprises: Step 1: Based on the GPU resources of each server in the cluster, the GPU management device is used to form multiple GPU resource pools and multiple vGPU resource pools, and the physical GPU is virtualized into a vGPU according to the usage scenario and a type label is added to each vGPU; Step 2: for each server in the cluster, using a GPU resource monitoring device corresponding to the server to collect statistics on the GPU resource usage and vGPU resource usage of the server, obtain a first statistical result and a second statistical result of each server, and report them to the GPU management device; Step 3, the GPU management device screens out all GPU resource pools and vGPU resource pools that meet the requirements of the virtual machines to be scheduled from the multiple GPU resource pools according to the first statistical result, the second statistical result and the type label of each server; Step 4: The GPU management device selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements according to the configured scheduling policy, and sends the selection result to the GPU resource monitoring device corresponding to the target server; Step 5: The GPU resource monitoring device schedules the virtual machine to be scheduled to the GPU graphics card in the target server according to the type label and the scoring rule.

2. The method for automatically scheduling GPUs of virtual machines in a cluster according to claim 1, characterized in that: The automatic scheduling method also includes: Dividing the servers in the cluster into a control node and multiple computing nodes; The GPU management device is deployed on the control node, and is used to manage all GPU resource pools, all vGPU resource pools, and the type tags, and select a target server for the virtual machine to be scheduled; The GPU resource monitoring device is deployed on the computing node. The GPU resource monitoring device corresponds to the computing node one-to-one and is used to collect statistics on the GPU resource usage and vGPU resource usage of each server, report the first statistical result and the second statistical result of each server to the GPU management device, and schedule the virtual machine to be scheduled to the GPU graphics card in the target server.

3. The method for automatically scheduling GPUs of virtual machines in a cluster according to claim 2, characterized in that: Virtualize the physical GPU into a vGPU according to the usage scenario and label each vGPU with a type, including: Virtualizing the physical GPU according to the usage scenario to obtain multiple types of vGPU, including cloud desktop rendering, graphics rendering, and high-performance computing; The vGPU of cloud desktop rendering type is labeled as type 1; The vGPU of the image rendering type is labeled as the second type; The high-performance computing type vGPU is labeled with the third type.

4. The method for automatically scheduling GPUs of virtual machines in a cluster according to claim 3, characterized in that: The GPU resource usage and vGPU resource usage of the server include: GPU graphics card utilization, GPU memory utilization, temperature, number of vGPU idle times, number of running tasks, and power consumption.

5. The method for automatically scheduling GPUs of virtual machines in a cluster according to claim 4, characterized in that: The GPU management device selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements according to the configured scheduling policy, including: The GPU management device selects the configured affinity rule as a scheduling strategy, and selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements; or, The GPU management device selects the configured anti-affinity rule as a scheduling strategy, and selects a target server for the virtual machine to be scheduled from all GPU resource pools and vGPU resource pools that meet the requirements.

6. The method for automatically scheduling GPUs of virtual machines in a cluster according to claim 5, characterized in that: The affinity rule is: when the GPU resource is allowed to be selected, it is stipulated that the virtual machine to be scheduled runs on the same specified resource, including GPU resource pool affinity, server affinity, and GPU graphics card affinity.

7. The method for automatically scheduling GPUs of virtual machines in a cluster according to claim 5, characterized in that: The anti-affinity rule is: when the GPU resource is allowed to be selected, it is stipulated that the virtual machine to be scheduled does not run on the same specified resource, including GPU resource pool anti-affinity, server anti-affinity, and GPU graphics card anti-affinity.

8. The method for automatically scheduling GPUs of virtual machines in a cluster according to claim 5, characterized in that: Selecting a target server for the virtual machine to be scheduled includes: The GPU resource pool and the vGPU resource pool are evaluated based on the server score expression Mark1 = HGu*HW0+HGm*HW1+HGn*HW2+HGvn*HW3, the resource usage in the GPU resource pool and the vGPU resource pool. v The servers in the GPU resource pool are scored to obtain the server with the highest score, and the server with the highest score is used as the target server; Among them, HGu represents the average GPU graphics card utilization in the same server, HGm represents the average GPU memory utilization in the same server, HGn represents the number of GPU graphics cards installed in the same server, HGvn represents the total number of unallocated vGPUs in the same server, and HW0, HW1, HW2, and HW3 all represent weights.

9. The method for automatic GPU scheduling of virtual machines in a cluster according to claim 8, characterized in that: The step 5 comprises: The GPU resource monitoring device determines all GPU graphics cards in the target server that meet the requirements of the virtual machine to be scheduled according to the type label; Design the graphics card scoring expression Mark2=G according to the scoring rules p *W0+G t *W1+G d *W2 scores the load, temperature and power consumption of all GPU graphics cards that meet the requirements of the virtual machines to be scheduled, obtains the GPU graphics card with the lowest score as the target graphics card, and schedules the virtual machines to be scheduled to the target graphics card; Among them, G p Indicates the GPU graphics card load, G t Indicates the temperature of the GPU graphics card, G d Indicates the power consumption of the GPU graphics card. W0, W1, and W2 all represent weights.

Images