Cloud platform AI computing power scheduling method, device and equipment

Abstract

The invention provides a cloud platform AI computing power scheduling method, device and equipment, and the method comprises the steps: after receiving a virtual machine migration instruction, extracting a target model, a target slot position and a target number from VGPU information of a source virtual machine, the target model being a model of a GPU providing a VGPU for the source virtual machine, the target slot position being a slot position where the GPU providing the VGPU for the source virtual machine is located, and the target number being a target number of the GPU providing the VGPU for the source virtual machine; the target number is the number of VGPUs provided by the GPU of the target slot position to the source virtual machine; an alternative computing node is determined from other computing nodes, and the GPU model of the alternative computing node in each target slot is the target model; whether a first computing node exists in the alternative computing nodes or not is detected, and the number of idle VGPUs of the GPUs of the first computing node in each target slot position is larger than or equal to the corresponding target number; and if the first computing nodes exist, determining a target computing node from the first computing nodes. According to the method and the device, the optimal target computing node of the thermal migration in the SR-IOV scene can be determined.

Description

Technical Field

[0001] The present application relates to the field of virtualization technology, and specifically to a cloud platform AI computing power scheduling method, device and equipment. Background Art

[0002] With the widespread application and promotion of artificial intelligence and large models, the demand for AI computing resources is increasing, showing an exponential growth trend. At present, the main demand for servers is AI servers in the intelligent computing center. There will be multiple GPU cards on the AI ​​servers to provide AI computing power, and the AI ​​services of the data center are deployed on the GPU cards in the resource pool of these AI servers. At present, the AI ​​servers of the server mainly use GPUs to perform AI computing tasks, and the CPU is responsible for data input and AI business program execution. In the intelligent computing center, GPUs are the main AI computing power and are scheduled and managed through cloud platforms. In order to improve the management efficiency and utilization of computing resources, servers currently need to use virtualization technology and cloud platforms to schedule and manage computing resources.

[0003] In related technologies, GPU cards use SR-IOV virtualization technology to virtualize PF (Physical Function) into multiple VF (Virtual Function) devices for virtual machines to use, thereby improving GPU computing power utilization. However, cold migration technology is currently mostly used in SR-IOV scenarios. Cold migration will be interrupted for a long time, which has a great impact on the business in scenarios that require continuous work such as training and reasoning, and will also cause waste and imbalance of GPU computing power resources. The computing power scheduling algorithm used in hot migration technology in other scenarios does not fully consider the characteristics of SR-IOV scenarios, and cannot determine the optimal target computing node for hot migration in SR-IOV scenarios. Summary of the invention

[0004] The present application provides a cloud platform AI computing power scheduling method, device and equipment, which can solve the technical problem in the prior art that it is impossible to determine the optimal target computing node for hot migration in the SR-IOV scenario.

[0005] In a first aspect, an embodiment of the present application provides a cloud platform AI computing power scheduling method, which is applied to a control node of a cloud platform. The cloud platform AI computing power scheduling method includes: After receiving the virtual machine migration instruction, extract the target model, target slot and target quantity from the VGPU information of the source virtual machine, where the target model is the model of the GPU that provides the VGPU to the source virtual machine, the target slot is the slot where the GPU that provides the VGPU to the source virtual machine is located, and the target quantity is the number of VGPUs provided by the GPU in the target slot to the source virtual machine, and the target slot corresponds to the target quantity one by one; Determine a candidate computing node from other computing nodes, wherein the GPU model of each target slot of the candidate computing node is the target model; Detect whether there is a first computing node among the candidate computing nodes, wherein the number of idle VGPUs of the GPU of each target slot of the first computing node is greater than or equal to the corresponding target number; If the first computing node exists, the target computing node is determined from the first computing node.

[0006] Furthermore, in one embodiment, after detecting whether the first computing node exists in the candidate computing nodes, the method further includes: If the first computing node does not exist, then detecting whether there is a second computing node among the candidate computing nodes, wherein the sum of the number of idle VGPUs of all target models of GPUs of the second computing node is greater than or equal to the sum of all target numbers; If there is a second computing node, determine the target computing node from the second computing node, and send a first release instruction to the target computing node, wherein the first release instruction is used to control the target computing node to release the VGPU required for the new virtual machine through the GPU of the target model to convert the target computing node into the first computing node.

[0007] Furthermore, in one embodiment, after detecting whether there is a second computing node among the candidate computing nodes, the method further includes: If the second computing node does not exist, the target computing node is determined from the candidate computing nodes, and a second release instruction is sent to the target computing node, wherein the second release instruction is used to control the target computing node to release the VGPU required for the new virtual machine through the GPU and CPU of the target model to convert the target computing node into the first computing node.

[0008] Further, in one embodiment, determining the target computing node from the candidate computing nodes includes: A target computing node is determined from candidate computing nodes whose CPUs support HBM.

[0009] Furthermore, in one embodiment, when there are multiple optional computing nodes for the target computing node, the computing node with the lowest GPU occupancy rate is determined as the target computing node.

[0010] Furthermore, in one embodiment, when there are multiple computing nodes with the lowest GPU occupancy rates, the computing node with the lowest CPU occupancy rate is determined as the target computing node.

[0011] Furthermore, in one embodiment, the cloud platform AI computing power scheduling method further includes: After receiving the virtual machine migration instruction, sending a first synchronization instruction to the source computing node, wherein the first synchronization instruction is used to control the source computing node to synchronize the VGPU video memory of the source virtual machine to the memory of the source computing node; After selecting the target computing node, a second synchronization instruction is sent to the target computing node, wherein the second synchronization instruction is used to control the target computing node to synchronize the VGPU video memory of the source virtual machine to the VGPU video memory of the new virtual machine from the memory of the source computing node.

[0012] Further, in one embodiment, the source computing node synchronizes the VGPU video memory of the source virtual machine to the memory of the source computing node through DMA technology; The target compute node uses GPU DRMA technology to synchronize the VGPU video memory of the source virtual machine from the memory of the source compute node to the VGPU video memory of the new virtual machine.

[0013] In a second aspect, an embodiment of the present application further provides a cloud platform AI computing power scheduling device, which is applied to a control node of a cloud platform, and the cloud platform AI computing power scheduling device includes: An information extraction module, configured to extract a target model, a target slot, and a target quantity from the VGPU information of the source virtual machine after receiving a virtual machine migration instruction, wherein the target model is the model of a GPU that provides a VGPU to the source virtual machine, the target slot is the slot where the GPU that provides a VGPU to the source virtual machine is located, and the target quantity is the number of VGPUs provided by the GPU in the target slot to the source virtual machine, and the target slot corresponds to the target quantity one by one; An alternative determination module, used to determine an alternative computing node from other computing nodes, wherein the GPU model of the alternative computing node in each target slot is the target model; A first detection module is used to detect whether there is a first computing node among the candidate computing nodes, wherein the number of idle VGPUs of the GPU of each target slot of the first computing node is greater than or equal to the corresponding target number; The target determination module is used to determine the target computing node from the first computing node if the first computing node exists.

[0014] In a third aspect, an embodiment of the present application also provides a cloud platform AI computing power scheduling device, which includes a processor, a memory, and a cloud platform AI computing power scheduling program stored in the memory and executable by the processor, wherein when the cloud platform AI computing power scheduling program is executed by the processor, the steps of the above-mentioned cloud platform AI computing power scheduling method are implemented.

[0015] In this application, after receiving the virtual machine migration instruction, the control node extracts the target model, target slot and target number from the VGPU information of the source virtual machine, and based on the extracted information, first finds the candidate computing nodes whose GPU model and slot meet the target model and target slot requirements from other computing nodes, and when there is a first computing node whose number of idle VGPUs in the target slot meets the target number requirements among the candidate computing nodes, the target computing node is determined from the first computing node. Through this application, the characteristics of the SR-IOV scenario are fully considered, and the optimal target computing node for hot migration in the SR-IOV scenario can be determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of a method for scheduling AI computing power on a cloud platform in one embodiment of the present application; Figure 2 A schematic diagram of a computing node in an embodiment of the present application; Figure 3 This is a schematic diagram of VGPU video memory synchronization in an embodiment of the present application; Figure 4 This is a functional module diagram of a cloud platform AI computing power scheduling device in one embodiment of the present application; Figure 5 This is a schematic diagram of the hardware structure of the cloud platform AI computing power scheduling device involved in the embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0019] In a first aspect, an embodiment of the present application provides a cloud platform AI computing power scheduling method, which is applied to the control node of the cloud platform.

[0020] Figure 1 A flow chart of a cloud platform AI computing power scheduling method in one embodiment of the present application is shown.

[0021] Reference Figure 1 In one embodiment, the cloud platform AI computing power scheduling method includes the following steps: S1. After receiving the virtual machine migration instruction, extract the target model, target slot and target quantity from the VGPU information of the source virtual machine, wherein the target model is the model of the GPU that provides the VGPU to the source virtual machine, the target slot is the slot where the GPU that provides the VGPU to the source virtual machine is located, and the target quantity is the number of VGPUs provided by the GPU in the target slot to the source virtual machine, and the target slot corresponds to the target quantity one by one.

[0022] Specifically, the cloud platform includes a control node and multiple computing nodes. Multiple virtual machines run in the computing node. The physical resources of the computing node include CPU and multiple GPUs. The VF obtained by using SR-IOV virtualization technology as PF with GPU is called VGPU. The virtual machine configures VGPU as a peripheral. Different GPUs in the same computing node are set in different slots, and different computing nodes in the cloud platform have the same slot list. The control node is the entrance and management center for logging into the cloud platform. It is responsible for the creation, migration, monitoring and other operations of virtual machines. The control node can obtain the GPU information, CPU information, etc. of all computing nodes as needed, as well as the VGPU information, VCPU information, XML configuration files, etc. of the virtual machines in the computing nodes.

[0023] For example, the slot information may be a BDF (Bus, Device, Function) number.

[0024] The virtual machine migration instruction is manually triggered by the user when maintenance is required, such as replacing the hardware on the computing node (such as the server), maintaining the computing node, adding new computing node resources, etc. After the control node receives the virtual machine migration instruction, it needs to decide which computing node to use as the target computing node. After the hot migration is completed, the target computing node will continue to execute the AI ​​business of the source virtual machine through the new virtual machine. The VGPU configuration of the new virtual machine needs to be consistent with the source virtual machine.

[0025] A virtual machine can be configured with one or more VGPUs. When multiple VGPUs are configured, these VGPUs can come from one or more GPUs. The number of VGPUs provided by different GPUs may be different, but the models of these GPUs need to be the same. Therefore, in this embodiment, the target model is unique, the target slot and the target quantity may not be unique, and each target slot corresponds to a target quantity.

[0026] Figure 2 A schematic diagram of a computing node in an embodiment of the present application is shown.

[0027] Reference Figure 2For example, the computing node runs virtual machine 1 and virtual machine 2, slot 1 is equipped with CPU1, slot 2 is equipped with CPU2, and GPU1 and GPU2 are both model A. Virtual machine 1 is configured with VGPU1 of GPU1, VGPU1 and VGPU2 of GPU2, and virtual machine 2 is configured with VGPU2 of GPU1. When virtual machine 1 is the source virtual machine, the target model is model A, the target slots are slot 1 and slot 2, the target quantity corresponding to slot 1 is 1, and the target quantity corresponding to slot 2 is 2. Accordingly, the GPUs in slots 1 and 2 on the target computing node also need to be model A, and the GPU in slot 1 provides 1 VGPU to the new virtual machine, and the GPU in slot 2 provides 2 VGPUs to the new virtual machine.

[0028] S2. Determine a candidate computing node from other computing nodes, wherein the GPU model of the candidate computing node in each target slot is the target model.

[0029] In this embodiment, the GPU model and slot of the candidate computing node meet the target model and target slot requirements, and there is no need to change the hardware configuration. When there is no candidate computing node in other computing nodes, the virtual machine migration instruction cannot be successfully responded to.

[0030] For example, when Figure 2 When VM 1 in is the source VM, the GPUs in slots 1 and 2 on the candidate compute node are model A.

[0031] S3. Detect whether there is a first computing node among the candidate computing nodes, wherein the number of idle VGPUs of the GPU of each target slot of the first computing node is greater than or equal to the corresponding target number.

[0032] In this embodiment, the number of idle VGPUs in the target slot of the first computing node meets the target number requirement, and the VGPUs required for creating a new virtual machine can be provided without performing a resource release operation before hot migration.

[0033] For example, when Figure 2 When virtual machine 1 in is the source virtual machine, the GPUs in slots 1 and 2 on the first computing node are model A, the number of idle VGPUs in slot 1 is greater than or equal to 1, and the number of idle VGPUs in slot 2 is greater than or equal to 2.

[0034] S4. If the first computing node exists, determine the target computing node from the first computing node.

[0035] In this embodiment, the candidate computing nodes are the complete optional range of the target computing nodes, the first computing node is the primary optional range of the target computing node, and the target computing node is determined preferentially from the first computing node, which can avoid unnecessary resource release operations before hot migration as much as possible.

[0036] It can be understood that when the first computing node is unique, it is determined as the target computing node. When the first computing node is not unique, the target computing node can be determined by referring to other factors, such as GPU occupancy, CPU occupancy, memory occupancy, disk occupancy, etc.

[0037] Therefore, in this embodiment, after receiving the virtual machine migration instruction, the control node extracts the target model, target slot and target quantity from the VGPU information of the source virtual machine, and first finds the candidate computing node from other computing nodes based on the extracted information, and when there is a first computing node among the candidate computing nodes, determines the target computing node from the first computing node. Through this embodiment, the characteristics of the SR-IOV scenario are fully considered, and the optimal target computing node for hot migration in the SR-IOV scenario can be determined.

[0038] It should be noted that the native virtualization layer software on the computing nodes of the current cloud platform does not support hot migration in SR-IOV scenarios. The virtualization layer software needs to be optimized to support hot migration of virtual machines configured with PCIE pass-through devices.

[0039] For example, the operation of modifying the libvirt virtualization software on the computing node includes: 1. Modify the libvirt source code to remove the check of the PCIE passthrough device configured for the source virtual machine when executing the hot migration instruction on the source computing node. Remove the restriction in the original code that stops executing the hot migration instruction if the source virtual machine is detected to have a PCIE passthrough device configuration, and continue to send the virtual machine hot migration instruction to the libvirt of the target computing node.

[0040] 2. Modify the libvirt source code. On the target computing node, first obtain the virtual machine configuration of the source computing node. If it contains the virtual machine configuration of the PCIE pass-through device, call the OS interface to obtain the BDF number of the PCIE device on the current computing node, and compare the BDF number of the PCIE pass-through device in the source computing node configuration with the BDF number on the obtained target computing node. If the target computing node contains the BDF number of the PCIE pass-through device in all virtual machine configuration files, it proves that the hardware configuration of the target computing node supports PCIE device pass-through. Then send a hot migration instruction to qemu to complete the hot migration of the virtual machine. Otherwise, stop sending hot migration instructions to qemu, return that the computing node hardware does not support PCIE device pass-through hot migration, and exit the program.

[0041] Furthermore, in one embodiment, after detecting whether the first computing node exists in the candidate computing nodes, the method further includes: If the first computing node does not exist, then detecting whether there is a second computing node among the candidate computing nodes, wherein the sum of the number of idle VGPUs of all target models of GPUs of the second computing node is greater than or equal to the sum of all target numbers; If there is a second computing node, determine the target computing node from the second computing node, and send a first release instruction to the target computing node, wherein the first release instruction is used to control the target computing node to release the VGPU required for the new virtual machine through the GPU of the target model to convert the target computing node into the first computing node.

[0042] In this embodiment, considering that when computing power is scheduled within the computing node, the VGPUs of the same model GPU can play an equivalent role and are not restricted by the slots. Since the sum of the number of idle VGPUs of all target models of GPUs of the second computing node is greater than or equal to the sum of all target numbers, the second computing node can, under the control of the first release instruction, migrate the AI ​​services of one part of the VGPU to another part of the VGPU only by replacing the VGPU between the target models of GPUs, thereby releasing the VGPU required for the new virtual machine and converting itself to the first computing node.

[0043] After the resource release operation, the corresponding AI business is still executed by the VGPU of the same model GPU, and the execution effect remains basically unchanged. Therefore, this embodiment uses the second computing node as the secondary optional range of the target computing node. When the first computing node does not exist and the second computing node exists, the target computing node is determined from the second computing node, which can minimize the negative impact of the resource release operation.

[0044] For example, when Figure 2 When virtual machine 1 in is the source virtual machine, there is a computing node, the GPUs in slots 1, 2, and 3 are model A, the GPU in slot 1 has no idle VGPU, the number of idle VGPUs in slot 2 is 1, the number of idle VGPUs in slot 3 is 2, the sum of the number of idle VGPUs of all target model GPUs is 3, the sum of the target numbers is 3, and the two are equal. This computing node is the second computing node. When this computing node is the target computing node, the two idle VGPUs in slot 3 replace one occupied VGPU in slot 1 and slot 2 respectively.

[0045] Specifically, the resource release operation corresponding to the first release instruction includes: determining the VGPU to be released from the target slots whose number of idle VGPUs is less than the target number, the number of which is equal to the difference between the target number and the number of idle VGPUs, checking the virtual machine corresponding to the VGPU to be released, modifying the VGPU to be released to the target VGPU that meets the requirements in the XML configuration file of the virtual machine, and restarting the virtual machine so that the VGPU to be released is updated from the occupied state to the idle state. The requirements that the target VGPU must meet include: the target VGPU is in the idle state, the GPU model to which it belongs is the target model, and the GPU to which it belongs does not contain the VGPU to be released.

[0046] Optionally, a VGPU with a lower occupancy rate is used as a VGPU to be released, and a target VGPU is selected from the GPUs with a lower occupancy rate, thereby further reducing the negative impact of the resource release operation.

[0047] Furthermore, in one embodiment, after detecting whether there is a second computing node among the candidate computing nodes, the method further includes: If the second computing node does not exist, the target computing node is determined from the candidate computing nodes, and a second release instruction is sent to the target computing node, wherein the second release instruction is used to control the target computing node to release the VGPU required for the new virtual machine through the GPU and CPU of the target model to convert the target computing node into the first computing node.

[0048] It can be understood that, since there is no second computing node in the candidate computing nodes, sufficient number of VGPUs cannot be released for use by the new virtual machine only by replacing VGPUs between GPUs of the target model.

[0049] In this embodiment, considering that the CPU can also execute a part of the AI ​​business, when there is no second computing node, the control target computing node of the second release instruction not only migrates the AI ​​business of a part of the VGPU to another part of the VGPU by replacing the VGPU between the target model GPUs, but also migrates the AI ​​business of a part of the VGPU to the CPU, thereby releasing the VGPU required for the new virtual machine and converting the target computing node to the first computing node. Through this embodiment, the computing power resources of the computing node are fully utilized, thereby improving the response success rate of the virtual machine migration instruction.

[0050] For example, when Figure 2When virtual machine 1 in is the source virtual machine, there is a computing node, the GPUs in slots 1, 2, and 3 are model A, there is no idle VGPU in the GPU of slot 1, the number of idle VGPUs in slot 2 is 1, the number of idle VGPUs in slot 3 is 1, the sum of the number of idle VGPUs of all target model GPUs is 2, and the sum of the target numbers is 3, the former is less than the latter, and the computing node is not the second computing node. When the computing node is the target computing node, replace the occupied VGPU in slot 1 or slot 2 with the idle VGPU in slot 3, and migrate the AI ​​service of the other occupied VGPU to the CPU.

[0051] Specifically, compared with the resource release operation corresponding to the first release instruction, the resource release operation corresponding to the second release instruction prioritizes modifying the VGPU to be released to a target VGPU that meets the requirements in the XML configuration file of the virtual machine. When the target VGPU cannot be found, the VGPU to be released can also be directly deleted.

[0052] Further, in one embodiment, determining the target computing node from the candidate computing nodes includes: A target computing node is determined from candidate computing nodes whose CPUs support HBM.

[0053] In this embodiment, considering that a CPU supporting HBM (High Bandwidth Memory) can execute AI services more efficiently than an ordinary CPU, when the CPU of the target computing node supports HBM, the negative impact of the resource release operation can be further reduced.

[0054] Furthermore, in one embodiment, when there are multiple optional computing nodes for the target computing node, the computing node with the lowest GPU occupancy rate is determined as the target computing node.

[0055] This embodiment helps to achieve GPU load balancing of different computing nodes. In addition, when the first computing node does not exist, the negative impact of the resource release operation can be further reduced.

[0056] Furthermore, in one embodiment, when there are multiple computing nodes with the lowest GPU occupancy rates, the computing node with the lowest CPU occupancy rate is determined as the target computing node.

[0057] This embodiment helps to achieve overall load balancing of different computing nodes. In addition, when there is no second computing node, the negative impact of the resource release operation can be further reduced.

[0058] Furthermore, in one embodiment, the cloud platform AI computing power scheduling method further includes: After receiving the virtual machine migration instruction, sending a first synchronization instruction to the source computing node, wherein the first synchronization instruction is used to control the source computing node to synchronize the VGPU video memory of the source virtual machine to the memory of the source computing node; After selecting the target computing node, a second synchronization instruction is sent to the target computing node, wherein the second synchronization instruction is used to control the target computing node to synchronize the VGPU video memory of the source virtual machine to the VGPU video memory of the new virtual machine from the memory of the source computing node.

[0059] In the related technology, the hot migration scheme of virtual machines only calculates the dirty pages of the source virtual machine's memory and migrates the dirty pages to ensure the consistency of the programs and data in the source virtual machine and the new virtual machine's memory, but does not process and monitor the dirty pages of the source virtual machine's VGPU video memory, which will cause the source virtual machine's VGPU video memory to be unable to migrate, causing the new virtual machine's VGPU to reload and execute the original business and data, resulting in a waste of GPU computing resources and reduced AI business execution efficiency. For example, for AI businesses that require multiple rounds of training, the training framework and completed training data can be retained, but half of the training data will be lost.

[0060] Through the implementation example, the VGPU video memory of the source virtual machine can be synchronized to the VGPU video memory of the new virtual machine, thereby reducing the waste of GPU computing resources and improving the efficiency of AI business execution.

[0061] Figure 3 A schematic diagram of VGPU video memory synchronization in an embodiment of the present application is shown.

[0062] Reference Figure 3 ,Further, in one embodiment, the source computing node synchronizes the VGPU video memory of the source virtual machine to the memory of the source computing node through DMA technology; The target compute node uses GPU DRMA technology to synchronize the VGPU video memory of the source virtual machine from the memory of the source compute node to the VGPU video memory of the new virtual machine.

[0063] In this embodiment, the integrity and security of the VGPU video memory data synchronization operation in the source computing node can be guaranteed by DMA (Direct Memory Access) technology, and the VGPU video memory synchronization performance from the source computing node to the target computing node can be improved by GPU DRMA (Remote Direct Memory Access) technology.

[0064] In a second aspect, an embodiment of the present application also provides a cloud platform AI computing power scheduling device, which is applied to the control node of the cloud platform.

[0065] Figure 4A schematic diagram of the functional modules of a cloud platform AI computing power scheduling device in one embodiment of the present application is shown.

[0066] Reference Figure 4 In one embodiment, the cloud platform AI computing power scheduling device includes: The information extraction module 10 is used to extract the target model, target slot and target quantity from the VGPU information of the source virtual machine after receiving the virtual machine migration instruction, wherein the target model is the model of the GPU that provides the VGPU to the source virtual machine, the target slot is the slot where the GPU that provides the VGPU to the source virtual machine is located, and the target quantity is the number of VGPUs provided by the GPU in the target slot to the source virtual machine, and the target slot corresponds to the target quantity one by one; The candidate determination module 20 is used to determine a candidate computing node from other computing nodes, wherein the GPU model of the candidate computing node in each target slot is the target model; A first detection module 30 is used to detect whether there is a first computing node among the candidate computing nodes, wherein the number of idle VGPUs of the GPU of each target slot of the first computing node is greater than or equal to the corresponding target number; The target determination module 40 is configured to determine a target computing node from the first computing nodes if the first computing node exists.

[0067] Further, in one embodiment, the cloud platform AI computing power scheduling device also includes a second detection module, which is used to detect whether there is a second computing node among the candidate computing nodes if the first computing node does not exist, wherein the sum of the number of idle VGPUs of all target models of GPUs of the second computing node is greater than or equal to the sum of all target numbers; The target determination module 40 is also used to determine the target computing node from the second computing node if there is a second computing node, and send a first release instruction to the target computing node, wherein the first release instruction is used to control the target computing node to release the VGPU required for the new virtual machine through the GPU of the target model to convert the target computing node into the first computing node.

[0068] Furthermore, in one embodiment, the target determination module 40 is also used to determine the target computing node from the alternative computing nodes if the second computing node does not exist, and send a second release instruction to the target computing node, wherein the second release instruction is used to control the target computing node to release the VGPU required for the new virtual machine through the GPU and CPU of the target model to convert the target computing node into a first computing node.

[0069] Furthermore, in one embodiment, the target determination module 40 is used to determine the target computing node from the candidate computing nodes whose CPUs support HBM.

[0070] Furthermore, in one embodiment, the target determination module 40 is configured to determine a computing node with the lowest GPU occupancy rate as the target computing node when there are multiple optional computing nodes as the target computing node.

[0071] Furthermore, in one embodiment, the target determination module 40 is configured to determine the computing node with the lowest CPU occupancy rate as the target computing node when there are multiple computing nodes with the lowest GPU occupancy rates.

[0072] Furthermore, in one embodiment, the cloud platform AI computing power scheduling device also includes a video memory synchronization module, which is used to: After receiving the virtual machine migration instruction, sending a first synchronization instruction to the source computing node, wherein the first synchronization instruction is used to control the source computing node to synchronize the VGPU video memory of the source virtual machine to the memory of the source computing node; After selecting the target computing node, a second synchronization instruction is sent to the target computing node, wherein the second synchronization instruction is used to control the target computing node to synchronize the VGPU video memory of the source virtual machine to the VGPU video memory of the new virtual machine from the memory of the source computing node.

[0073] Further, in one embodiment, the source computing node synchronizes the VGPU video memory of the source virtual machine to the memory of the source computing node through DMA technology; The target compute node uses GPU DRMA technology to synchronize the VGPU video memory of the source virtual machine from the memory of the source compute node to the VGPU video memory of the new virtual machine.

[0074] Among them, the functional implementation of each module in the above-mentioned cloud platform AI computing power scheduling device corresponds to the various steps in the above-mentioned cloud platform AI computing power scheduling method embodiment, and its functions and implementation process will not be repeated here one by one.

[0075] On the third aspect, an embodiment of the present application provides a cloud platform AI computing power scheduling device, which can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0076] Figure 5 A schematic diagram of the hardware structure of the cloud platform AI computing power scheduling device involved in the embodiment of the present application is shown.

[0077] Reference Figure 5 In an embodiment of the present application, the cloud platform AI computing power scheduling device may include a processor, a memory, a communication interface, and a communication bus.

[0078] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0079] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces, which are used to interconnect devices within the cloud platform AI computing power scheduling device, as well as interfaces used to interconnect the cloud platform AI computing power scheduling device with other devices (such as other computing devices or user devices). Physical interfaces can be Ethernet interfaces, optical fiber interfaces, ATM interfaces, etc.; user devices can be display screens (Display), keyboards (Keyboard), etc.

[0080] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0081] The processor may be a general-purpose processor, and the general-purpose processor may call the cloud platform AI computing power scheduling program stored in the memory and execute the cloud platform AI computing power scheduling method provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the cloud platform AI computing power scheduling program is called may refer to the various embodiments of the cloud platform AI computing power scheduling method of the present application, and will not be repeated here.

[0082] Those skilled in the art will understand that Figure 5 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0083] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0084] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0085] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0086] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0087] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0088] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0089] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A cloud platform AI computing power scheduling method, characterized in that: Applied to the control node of the cloud platform, the cloud platform AI computing power scheduling method includes: After receiving the virtual machine migration instruction, extract the target model, target slot and target quantity from the VGPU information of the source virtual machine, where the target model is the model of the GPU that provides the VGPU to the source virtual machine, the target slot is the slot where the GPU that provides the VGPU to the source virtual machine is located, and the target quantity is the number of VGPUs provided by the GPU in the target slot to the source virtual machine, and the target slot corresponds to the target quantity one by one; Determine a candidate computing node from other computing nodes, wherein the GPU model of each target slot of the candidate computing node is the target model; Detect whether there is a first computing node among the candidate computing nodes, wherein the number of idle VGPUs of the GPU of each target slot of the first computing node is greater than or equal to the corresponding target number; If the first computing node exists, the target computing node is determined from the first computing node.

2. The cloud platform AI computing power scheduling method according to claim 1, characterized in that: After detecting whether the first computing node exists in the candidate computing nodes, the method further includes: If the first computing node does not exist, then detecting whether there is a second computing node among the candidate computing nodes, wherein the sum of the number of idle VGPUs of all target models of GPUs of the second computing node is greater than or equal to the sum of all target numbers; If there is a second computing node, determine the target computing node from the second computing node, and send a first release instruction to the target computing node, wherein the first release instruction is used to control the target computing node to release the VGPU required for the new virtual machine through the GPU of the target model to convert the target computing node into the first computing node.

3. The cloud platform AI computing power scheduling method according to claim 2, characterized in that: After detecting whether there is a second computing node among the candidate computing nodes, the method further includes: If the second computing node does not exist, the target computing node is determined from the candidate computing nodes, and a second release instruction is sent to the target computing node, wherein the second release instruction is used to control the target computing node to release the VGPU required for the new virtual machine through the GPU and CPU of the target model to convert the target computing node into the first computing node.

4. The cloud platform AI computing power scheduling method according to claim 3, characterized in that: The step of determining the target computing node from the candidate computing nodes includes: A target computing node is determined from candidate computing nodes whose CPUs support HBM.

5. The cloud platform AI computing power scheduling method according to any one of claims 1 to 4, characterized in that: When there are multiple optional computing nodes for the target computing node, the computing node with the lowest GPU occupancy rate is determined as the target computing node.

6. The cloud platform AI computing power scheduling method according to claim 5, characterized in that: When there are multiple computing nodes with the lowest GPU occupancy, the computing node with the lowest CPU occupancy is determined as the target computing node.

7. The cloud platform AI computing power scheduling method according to any one of claims 1 to 4, characterized in that: The cloud platform AI computing power scheduling method also includes: After receiving the virtual machine migration instruction, sending a first synchronization instruction to the source computing node, wherein the first synchronization instruction is used to control the source computing node to synchronize the VGPU video memory of the source virtual machine to the memory of the source computing node; After selecting the target computing node, a second synchronization instruction is sent to the target computing node, wherein the second synchronization instruction is used to control the target computing node to synchronize the VGPU video memory of the source virtual machine to the VGPU video memory of the new virtual machine from the memory of the source computing node.

8. The cloud platform AI computing power scheduling method according to claim 7, characterized in that: The source computing node synchronizes the VGPU memory of the source virtual machine to the memory of the source computing node through DMA technology; The target compute node uses GPU DRMA technology to synchronize the VGPU video memory of the source virtual machine from the memory of the source compute node to the VGPU video memory of the new virtual machine.

9. A cloud platform AI computing power scheduling device, characterized in that: Applied to the control node of the cloud platform, the cloud platform AI computing power scheduling device includes: An information extraction module, configured to extract a target model, a target slot, and a target quantity from the VGPU information of the source virtual machine after receiving a virtual machine migration instruction, wherein the target model is the model of a GPU that provides a VGPU to the source virtual machine, the target slot is the slot where the GPU that provides a VGPU to the source virtual machine is located, and the target quantity is the number of VGPUs provided by the GPU in the target slot to the source virtual machine, and the target slot corresponds to the target quantity one by one; An alternative determination module, used to determine an alternative computing node from other computing nodes, wherein the GPU model of the alternative computing node in each target slot is the target model; A first detection module is used to detect whether there is a first computing node among the candidate computing nodes, wherein the number of idle VGPUs of the GPU of each target slot of the first computing node is greater than or equal to the corresponding target number; The target determination module is used to determine the target computing node from the first computing node if the first computing node exists.

10. A cloud platform AI computing power scheduling device, characterized in that: The cloud platform AI computing power scheduling device includes a processor, a memory, and a cloud platform AI computing power scheduling program stored in the memory and executable by the processor. When the cloud platform AI computing power scheduling program is executed by the processor, the steps of the cloud platform AI computing power scheduling method as described in any one of claims 1 to 8 are implemented.

Images