A cloud platform GPU virtual machine scheduling method and device

Abstract

The present invention discloses a cloud platform GPU virtual machine scheduling method and device, which relates to the field of computer technology. The method obtains the target number of GPU resources required for a new task. If the total number of GPU resources remaining in all channels is greater than the target number of GPU resources, and no single channel meets the target number of GPU resources, all channels are sorted from most to least by the number of remaining GPU resources of each channel, and then a preset number of channels are obtained to obtain an effective channel set. The resources in the effective channel set are dynamically migrated by a preset algorithm to obtain a migration result. If the migration result is to generate a target migration channel, the target number of GPU resources of the new task is all scheduled to the target migration channel. By dynamically migrating idle resources across channels, the fragmentation of single channel resources is avoided, and migration from channels with more remaining resources is prioritized, thereby improving the utilization rate of GPU resources, reducing scheduling complexity, and meeting the needs of high-performance computing tasks.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a cloud platform GPU virtual machine scheduling method and device. Background Art

[0002] Driven by the ongoing wave of digitalization, high-performance computing tasks, such as deep learning model training in artificial intelligence and complex data processing in big data analysis, are booming. These tasks place extremely demanding demands on computing resources, with a particular increasing reliance on GPUs (graphics processing units). GPUs, with their powerful parallel computing capabilities, significantly improve the processing efficiency of complex computing tasks and have become a core hardware resource supporting high-performance computing.

[0003] However, scheduling multiple GPUs in a common virtual machine (VM) computing architecture presents a series of pressing challenges. First, low GPU resource utilization is particularly prominent. Due to the inflexible resource allocation mechanism between VMs and GPUs, idle or overallocated GPU resources often occur. For example, in certain business scenarios, some VMs are allocated GPU resources far exceeding their actual needs, resulting in resource waste, while others are unable to fully utilize their computing power due to insufficient resources. Secondly, while cross-channel direct access provides VMs with direct access to GPUs, this approach incurs significant performance overhead. Frequent cross-channel operations during data transmission consume significant system resources, reducing data transmission efficiency and, in turn, impacting the execution speed of the entire computing task. Furthermore, the high complexity of multi-GPU scheduling is a significant issue. As the number of GPUs increases, scheduling algorithms must comprehensively consider multiple factors, such as task priority, resource load, and GPU performance differences. This complicates their design and implementation, making it difficult to effectively meet the requirements of high-performance computing tasks for efficient and stable GPU resource allocation.

[0004] Current traditional scheduling methods are unable to effectively balance system resources in these complex situations. In scenarios with multiple GPUs, this scheduling approach significantly impacts virtual machine performance, preventing the GPUs from fully utilizing their powerful computing capabilities. This not only restricts the efficiency of high-performance computing tasks but also significantly wastes hardware resources. Summary of the Invention

[0005] The purpose of the present invention is to solve the above problems and to propose a cloud platform GPU virtual machine scheduling method and device.

[0006] In a first aspect of the present invention, a cloud platform GPU virtual machine scheduling method is first proposed, the method comprising:

[0007] Obtain the target number of GPU resources required for the new task, and determine whether the total number of GPU resources remaining in all channels is greater than the target number of GPU resources; each channel has a fixed number of GPU resources;

[0008] If the total number of remaining GPU resources is greater than the target number of GPU resources, determining whether there is a single channel that satisfies the target number of GPU resources;

[0009] If no single channel satisfies the target number of GPU resources, then all channels are sorted from most to least based on the number of remaining GPU resources of each channel to obtain an initial resource channel sorting set;

[0010] Obtain the first preset channels in the initial resource channel sorting set to obtain a valid channel set;

[0011] Dynamically migrating the resources in the effective channel set using a preset algorithm to obtain a migration result; the migration result includes generating a target migration channel or not generating a target migration channel; the target migration channel is a channel that meets the target number of GPU resources;

[0012] If the migration result is to generate a target migration channel, all target GPU resources of the new task are scheduled to the target migration channel.

[0013] Optionally, after determining whether the total number of remaining GPU resources in all channels is greater than the target number of GPU resources, the method further includes: if the total number of remaining GPU resources is less than or equal to the target number of GPU resources, determining that the new task execution has failed, and sending an error message to the user.

[0014] Optionally, after determining whether a single channel satisfies the target number of GPU resources, the method further includes:

[0015] If a single channel meets the target number of GPU resources, all channels that meet the target are obtained. For all channels that meet the target, the usage score of the channel is determined based on the GPU memory occupancy rate, CPU usage rate, and memory usage rate.

[0016] The channel with the highest score is used as the execution channel for the new task.

[0017] Optionally, dynamically migrating resources in the valid channel set using a preset algorithm to obtain a migration result includes:

[0018] For each valid channel in the valid channel set, calculating the number of resources that are missing for the valid channel to execute the new task to obtain a resource missing set;

[0019] For each valid channel in the valid channel set, a resource missing is determined, based on the resource missing, to be a virtual machine that meets the conditions in the valid channel to obtain a generation migration virtual machine set;

[0020] For each generation of migrating virtual machines in the generation migrating virtual machine set, determining a migration cost of the generation of migrating virtual machines according to the video memory occupancy rate and usage rate of the generation of migrating virtual machines;

[0021] Obtaining the generation migration virtual machine with the lowest migration cost as the initial migration virtual machine of the valid channel;

[0022] For each channel, the initial migration virtual machine corresponding to the channel is calculated as the migration severity of the initial migration virtual machine, and the initial migration virtual machine with the lowest migration severity is obtained as the initial target migration virtual machine;

[0023] If the initial target migration virtual machine can be migrated to a valid channel other than the current channel, the initial target migration virtual machine is used as the target migration channel, and the migration result is generating a target migration channel; the migration result is not generating a target migration channel.

[0024] Optionally, calculating the migration severity of the initially migrated virtual machine includes:

[0025] Determine the average memory occupancy and average usage of the GPU card being used by the initial migration virtual machine and the number of resources missing from the channel where the initial migration virtual machine is located to determine the migration severity;

[0026] By formula Obtaining the migration severity of the initial migration virtual machine;

[0027] Where R is the migration severity of the initial migration virtual machine, r1, r2, and r3 are weight coefficients respectively, M is the number of resources missing from the channel where the initial migration virtual machine is located, Q is the average memory occupancy of the GPU card in use, and P is the average usage rate of the GPU card in use.

[0028] In a second aspect of the present invention, a cloud platform GPU virtual machine scheduling device is provided, comprising:

[0029] A total number of GPU resources determination module is used to obtain the target number of GPU resources required for a new task and, based on the target number of GPU resources, determine whether the total number of GPU resources remaining in all channels is greater than the target number of GPU resources; each channel has a fixed number of GPU resources;

[0030] a channel determination module, configured to determine whether a single channel satisfies the target number of GPU resources if the total number of remaining GPU resources is greater than the target number of GPU resources;

[0031] A channel sorting module is configured to sort all channels from most to least by the number of remaining GPU resources of each channel to obtain an initial resource channel sorting set if no single channel meets the target number of GPU resources;

[0032] An effective channel determination module, configured to obtain the first preset channels in the initial resource channel sorting set to obtain an effective channel set;

[0033] a resource migration module, configured to dynamically migrate resources in the valid channel set using a preset algorithm to obtain a migration result; the migration result includes generating a target migration channel or not generating a target migration channel; the target migration channel is a channel that meets the target number of GPU resources;

[0034] The new task scheduling determination module is configured to schedule all target GPU resources of the new task to the target migration channel if the migration result is to generate a target migration channel.

[0035] Optionally, the device further includes:

[0036] The error information sending module is used to determine that the execution of the new task fails if the total number of remaining GPU resources is less than or equal to the target number of GPU resources, and send an error message to the user.

[0037] Optionally, the device further includes:

[0038] A channel score determination module is configured to obtain all channels that meet the target number of GPU resources if a single channel exists, and determine the channel's usage score based on GPU memory occupancy, CPU usage, and memory usage for all channels that meet the target number of GPU resources.

[0039] The execution channel determination module is used to obtain the channel with the highest score as the execution channel of the new task.

[0040] Optionally, the resource migration module includes:

[0041] a resource missing set determining module, configured to calculate, for each valid channel in the valid channel set, the number of resources that the valid channel lacks to satisfy the execution of the new task, to obtain a resource missing set;

[0042] a generation migration virtual machine set determination module, configured to determine, based on resource loss corresponding to each valid channel in the valid channel set, virtual machines that meet the conditions in the valid channel to obtain a generation migration virtual machine set;

[0043] A migration cost determination module is used to determine the migration cost of each generation of migration virtual machines in the generation migration virtual machine set according to the video memory occupancy rate and usage rate of the generation of migration virtual machines;

[0044] An initial migration virtual machine determination module is used to obtain a generation migration virtual machine with the lowest migration cost as the initial migration virtual machine of the valid channel;

[0045] A migration severity determination module is configured to calculate the migration severity of the initial migration virtual machine corresponding to each channel, and obtain the initial migration virtual machine with the lowest migration severity as the initial target migration virtual machine;

[0046] The migration result determination module is used to determine whether the initial target migration virtual machine can be migrated to a valid channel other than the current channel, and the initial target migration virtual machine is used as the target migration channel, and the migration result is generating the target migration channel; the migration result is not generating the target migration channel.

[0047] Optionally, the migration severity determination module includes:

[0048] A parameter determination module is used to determine the average memory occupancy and average usage of the GPU card being used by the initial migration virtual machine and the number of resources missing from the channel where the initial migration virtual machine is located to determine the migration severity;

[0049] Migration severity calculation module, used to calculate the severity of the migration through the formula Obtaining the migration severity of the initial migration virtual machine;

[0050] Where R is the migration severity of the initial migration virtual machine, r1, r2, and r3 are weight coefficients respectively, M is the number of resources missing from the channel where the initial migration virtual machine is located, Q is the average memory occupancy of the GPU card in use, and P is the average usage rate of the GPU card in use.

[0051] Beneficial effects of the present invention:

[0052] The present invention proposes a cloud platform GPU virtual machine scheduling method, which obtains the target number of GPU resources required for a new task, and determines whether the total number of GPU resources remaining in all channels is greater than the target number of GPU resources based on the target number of GPU resources; if the total number of GPU resources remaining is greater than the target number of GPU resources, it is determined whether a single channel satisfies the target number of GPU resources; if no single channel satisfies the target number of GPU resources, all channels are sorted from most to least according to the number of GPU resources remaining in each channel to obtain an initial resource channel sorting set; a preset number of channels in the initial resource channel sorting set are obtained to obtain an effective channel set; resources in the effective channel set are dynamically migrated using a preset algorithm to obtain a migration result; if the migration result is to generate a target migration channel, all the target number of GPU resources of the new task are scheduled to the target migration channel. By dynamically migrating idle resources across channels, the fragmentation of resources of a single channel is avoided, and there is no need to rely on the resource sufficiency of a single channel. By prioritizing migration from channels with more remaining resources, the load of each channel can be balanced, the utilization rate of GPU resources can be improved, the scheduling complexity can be reduced, and the requirements of high-performance computing tasks can be met. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will be further described below with reference to the accompanying drawings.

[0054] Figure 1 A flowchart of a cloud platform GPU virtual machine scheduling method provided by an embodiment of the present invention;

[0055] Figure 2 A flowchart of resource migration provided by an embodiment of the present invention;

[0056] Figure 3 A schematic diagram of the structure of a cloud platform GPU virtual machine scheduling device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0058] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0059] The embodiment of the present invention provides a cloud platform GPU virtual machine scheduling method. Figure 1 , Figure 1 A flowchart of a method for scheduling a GPU virtual machine on a cloud platform provided by an embodiment of the present invention. The method includes the following steps:

[0060] S101, obtaining the target number of GPU resources required for the new task, and determining whether the total number of GPU resources remaining in all channels is greater than the target number of GPU resources;

[0061] S102, if the total number of remaining GPU resources is greater than the target number of GPU resources, determine whether there is a single channel that meets the target number of GPU resources;

[0062] S103, if no single channel meets the target number of GPU resources, sort all channels from most to least according to the number of remaining GPU resources of each channel to obtain an initial resource channel sorting set;

[0063] S104, obtaining the first preset channels in the initial resource channel sorting set to obtain a valid channel set;

[0064] S105, dynamically migrating resources in the effective channel set using a preset algorithm to obtain a migration result;

[0065] S106: If the migration result is to generate a target migration channel, all target GPU resources of the new task are scheduled to the target migration channel.

[0066] There is a fixed number of GPU resources in each channel; the migration result includes generating a target migration channel or not generating a target migration channel; the target migration channel is a channel that meets the target number of GPU resources.

[0067] A cloud platform GPU virtual machine scheduling method provided by an embodiment of the present invention avoids resource fragmentation of a single channel by dynamically migrating idle resources across channels. This method does not rely on the resource sufficiency of a single channel and prioritizes migration from channels with more remaining resources. This method can balance the load of each channel, improve GPU resource utilization, reduce scheduling complexity, and meet the needs of high-performance computing tasks.

[0068] In one implementation, idle resources are dynamically migrated across channels to avoid resource fragmentation in a single channel (e.g., when the total remaining resources are sufficient but a single channel is insufficient), thereby improving overall resource utilization. Task requirements can be met by combining multi-channel resources without relying on the resource adequacy of a single channel, adapting to diverse resource allocation scenarios.

[0069] In one implementation, this solution includes channel division, virtual machine-channel binding, in-channel resource scheduling, inter-channel load balancing, and intelligent scheduling. Channel division: The physical GPU cards on each server are divided into multiple logical channels, and each channel corresponds to an independent subset of GPU resources. Virtual machine-channel binding: According to the GPU resource requirements of the virtual machine, the virtual machine is scheduled to a specific channel to ensure that the GPU resources used by the virtual machine come from the same channel. In-channel resource scheduling: Inside the channel, the GPU resources are directly allocated to the virtual machine to avoid resource switching overhead and improve GPU utilization and virtual machine performance. Inter-channel load balancing: According to the load conditions of system CPU, memory, GPU, etc., the remaining resource weights of each channel on each server are dynamically calculated, and then the virtual machine allocation between channels is adjusted to achieve load balancing of system resources. Intelligent scheduling (preset algorithm): During the scheduling process, priority is given to scheduling virtual machines using multi-GPU to the same channel to avoid cross-channel direct connection problems. If it is found during the scheduling process that the remaining resources of each channel cannot meet the multi-GPU requirements of a single virtual machine, then the intelligent scheduling (preset algorithm) is enabled.

[0070] In one implementation, the intelligent scheduling (preset algorithm) includes calculating the number N of remaining GPU resources on the computing platform t of whether the number N0 of GPU resources that meet the scheduling requirements. If N t < N0, the scheduling fails; if N t > N0, query the number of remaining GPU resources of each channel (channel 1, channel 2,..., channel n), denoted as N1, N2,..., N n , then N t = N1 + N2 +... + N n , and N m < N t (m belongs to (1, 2,..., n)), that is, no single channel meets the scheduling requirements. To avoid cross-channel scheduling, start the virtual machine dynamic migration process, select the 3 channels (the first preset number, determined by technical personnel, and taking 3 as the first preset number as an example) with the most remaining GPU resources on the platform, denoted as N t1 , N t2 , N t3 that satisfy (N t1 , N t2 , N t3 ) = sorted(N1, N2,..., N n ), calculate the resource quantities lacking for the GPU resources of the corresponding three channels to meet the virtual machine scheduling, denoted as M1, M2, M3 respectively. M1 = N t - N t1 , M 2= N t - N t2 , M3 = Nt -N t3 , search for virtual machines that meet the conditions in these three channels respectively. Assume that the optimal channel virtual machine 1 (VM1), virtual machine 2 (VM2), and virtual machine 3 (VM3) are searched in the three channels respectively, and the number of GPU resources of each virtual machine is recorded as N vm1 , N vm2 , N vm3 , meeting the conditions: N vm1 ≥M1, N vm2 ≥M2, N vm3 ≥M3, based on the records of the monitoring system, calculate the average memory occupancy of the GPU cards used by virtual machine 1 (VM1), virtual machine 2 (VM2), and virtual machine 3 (VM3), and record them as Q1, Q2, and Q3. Calculate the average usage of the GPU cards used by virtual machine 1 (VM1), virtual machine 2 (VM2), and virtual machine 3 (VM3), and record them as P1, P2, and P3; calculate the migration severity R1, R2, and R3 of the intelligent scheduling of the above three virtual machines respectively; R1=r1M1+r2Q1+r 3P1; R2=r1M2+r2Q2+r3P2; R3=r1M3+r2Q3+r3P3; where r1, r2, and r3 are the weight values ​​of the number of GPU resources, the weight value of GPU memory occupancy, and the weight value of GPU utilization, respectively. r1, r2, and r3 are constants, usually 1, 1, and 1. The specific values ​​are determined by technical personnel. Finally, the virtual machine with the least migration severity among R1, R2, and R3 is selected, and then the migration command is executed on the virtual machine so that the channel where the virtual machine is located can meet the current scheduling requirements.

[0071] In one implementation, priority is given to migrating from channels with more remaining resources, which can balance the load of each channel, avoid some channels being overly busy and some channels being idle, and extend the life of the hardware; the preset algorithm and sorting mechanism can be dynamically adjusted according to the real-time resource status, quickly respond to the resource requirements of sudden tasks, and reduce scheduling delays.

[0072] In one embodiment, after determining whether the total number of remaining GPU resources in all channels is greater than the target number of GPU resources, the method further includes: if the total number of remaining GPU resources is less than or equal to the target number of GPU resources, determining that the new task execution has failed, and sending an error message to the user.

[0073] In one implementation, when the total global resources are insufficient, the task execution is immediately determined to have failed and an error is fed back, preventing the system from attempting invalid resource allocation or migration operations, reducing computing resource waste and scheduling delays; resource bottlenecks are promptly exposed to users to help them quickly locate problems (such as insufficient resources), facilitate users to adjust task parameters (such as splitting tasks, reducing resource requirements) or apply for capacity expansion, and improve interaction efficiency.

[0074] In one embodiment, after determining whether a single channel satisfies the target number of GPU resources, the method further includes:

[0075] If a single channel meets the target number of GPU resources, all channels that meet the requirement are obtained. For all channels that meet the requirement, the usage score of the channel is determined based on the GPU memory occupancy, CPU usage, and memory usage.

[0076] Get the channel with the highest score as the execution channel for the new task.

[0077] In one implementation, instead of relying solely on the single condition of "sufficient remaining GPU resources," a comprehensive assessment of channel load is performed using multiple indicators, including GPU memory occupancy, CPU utilization, and memory utilization. This avoids scheduling tasks to channels with "sufficient GPU resources but fully loaded other resources," thus preventing task performance degradation or channel crashes due to resource imbalance.

[0078] In one implementation, "usage scoring" is used to screen out channels with the lowest overall system load (such as low video memory usage, low CPU, and memory pressure). This provides a better operating environment for new tasks, reduces resource competition, and improves task execution efficiency and stability. For example, if a channel has sufficient remaining GPU resources but the CPU utilization rate has reached 90%, scheduling new tasks may cause GPU resources to be idle due to CPU bottlenecks. The scoring mechanism will prioritize channels with lower CPU and memory loads to achieve coordinated resource utilization.

[0079] In one implementation, the usage score of the channel is determined by the GPU memory occupancy, CPU usage, and memory usage. The usage score is obtained by the formula Z=1 / (aA+bB+cC), where Z is the usage score, the weights of a, b, and c are 0.4, 0.3, and 0.3 respectively, A is the GPU memory occupancy, B is the CPU usage, and C is the memory usage.

[0080] In one implementation, suppose the cloud platform needs to schedule a new GPU virtual machine that requires four GPU cards. The cloud platform checks to see if there are enough resources in a single channel to meet the requirements for four GPU cards. If multiple channels are found to meet the requirements, the platform determines the channel's usage score based on GPU memory occupancy, CPU usage, and memory usage, and selects the optimal channel to schedule the virtual machine.

[0081] In one implementation, scoring is dynamically calculated based on real-time monitoring data, which can respond to channel load fluctuations in real time (such as the release of resources due to the completion of other tasks), ensuring that each scheduling is based on the latest system status and avoiding resource mismatches caused by static allocation. The automated scoring mechanism replaces the complexity of manually judging channel load, reducing the cost of manual adjustment of task allocation by operation and maintenance personnel.

[0082] In one embodiment, see Figure 2 , Figure 2 A resource migration flow chart is provided. The resources in the effective channel set are dynamically migrated through a preset algorithm to obtain the following migration results:

[0083] S1051, for each valid channel in the valid channel set, calculating the number of resources that the valid channel lacks to execute the new task to obtain a resource missing set;

[0084] S1052: for each valid channel in the valid channel set, determining a virtual machine in the valid channel that meets the conditions based on the resource missing to obtain a virtual machine set for migration;

[0085] S1053 , for each generation of migrating virtual machines in the generation migrating virtual machine set, determining a migration cost of the generation of migrating virtual machines based on the video memory occupancy and usage of the generation of migrating virtual machines;

[0086] S1054, obtaining a generation migration virtual machine with the lowest migration cost as the initial migration virtual machine of the valid channel;

[0087] S1055 , calculating the migration severity of the initial migration virtual machine corresponding to each channel, and obtaining the initial migration virtual machine with the lowest migration severity as the initial target migration virtual machine;

[0088] S1056, if the initial target migration virtual machine can be migrated to a valid channel other than the current channel, the initial target migration virtual machine is used as the target migration channel, and the migration result is generating a target migration channel; the migration result is not generating a target migration channel.

[0089] In one implementation, by identifying and migrating low-utilization virtual machines (VMs), scattered resource fragments are integrated into centrally available resource blocks, avoiding the purchase of new hardware due to local resource shortages and significantly reducing IT infrastructure costs. The migration cost of a generation of migrated VMs is determined based on the video memory occupancy and usage rate of the generation of migrated VMs, and the sum of the video memory occupancy and usage rate is calculated as the migration cost of the generation of migrated VMs.

[0090] In one implementation, migration cost assessment (memory occupancy and utilization) prioritizes VMs with the least business impact. For example, VMs with low memory usage and low CPU utilization are prioritized to avoid interrupting critical services. Migration severity assessment further selects VMs with the least impact on source channel performance, ensuring stable operation of the source channel after migration.

[0091] In one implementation, global resource pooling is achieved through cross-channel migration by breaking the resource isolation between channels. Even if a single channel has insufficient resources, the system can still meet task requirements through dynamic adjustment, enhancing its ability to cope with sudden resource demands. A dual screening mechanism (lowest cost and lowest severity) ensures that each migration decision weighs resource benefits and business risks, avoiding service interruptions or performance fluctuations due to blind migration.

[0092] In one embodiment, calculating the migration severity of the initial migration virtual machine includes:

[0093] Determine the average memory occupancy and average usage of the GPU card being used by the initial migration virtual machine and the number of resources missing from the channel where the initial migration virtual machine is located to determine the migration severity;

[0094] By formula Obtaining the migration severity of the initial migration virtual machine;

[0095] Where R is the migration severity of the initial migration virtual machine, r1, r2, and r3 are weight coefficients respectively, M is the number of resources missing from the channel where the initial migration virtual machine is located, Q is the average memory occupancy of the GPU card in use, and P is the average usage rate of the GPU card in use.

[0096] In one implementation, the average memory occupancy rate and average usage rate of the GPU card and the initial migration virtual machine are combined into a single indicator, thereby improving the rationality of screening the migration virtual machines and reducing subjective errors.

[0097] In one implementation, the mathematical model of this formula transforms the complex resource scheduling problem into a quantifiable optimization problem, maximizing resource reuse while ensuring business stability.

[0098] Based on the same inventive concept, the present invention also provides a cloud platform GPU virtual machine scheduling device. Figure 3 , Figure 3 A schematic diagram of a cloud platform GPU virtual machine scheduling device provided by an embodiment of the present invention includes:

[0099] The total number of GPU resources determination module is used to obtain the target number of GPU resources required for the new task and determine whether the total number of GPU resources remaining in all channels is greater than the target number of GPU resources. Each channel has a fixed number of GPU resources.

[0100] A channel determination module is used to determine whether there is a single channel that meets the target number of GPU resources if the total number of remaining GPU resources is greater than the target number of GPU resources;

[0101] A channel sorting module is used to sort all channels from most to least based on the number of remaining GPU resources in each channel to obtain an initial resource channel sorting set if no single channel meets the target number of GPU resources.

[0102] An effective channel determination module is used to obtain the first preset channels in the initial resource channel sorting set to obtain an effective channel set;

[0103] The resource migration module is used to dynamically migrate resources in the valid channel set through a preset algorithm to obtain a migration result; the migration result includes whether a target migration channel is generated or not; the target migration channel is a channel that meets the target number of GPU resources;

[0104] The new task scheduling determination module is used to schedule all target GPU resources of the new task to the target migration channel if the migration result is to generate a target migration channel.

[0105] A cloud platform GPU virtual machine scheduling device provided by an embodiment of the present invention avoids resource fragmentation of a single channel by dynamically migrating idle resources across channels. It does not need to rely on the resource sufficiency of a single channel and prioritizes migration from channels with more remaining resources. This can balance the load of each channel, improve the utilization of GPU resources, reduce scheduling complexity, and meet the needs of high-performance computing tasks.

[0106] In one embodiment, the apparatus further comprises:

[0107] The error information sending module is used to determine that the new task execution fails if the total number of remaining GPU resources is less than or equal to the target number of GPU resources, and send an error message to the user.

[0108] In one embodiment, the apparatus further comprises:

[0109] A channel score determination module is used to obtain all channels that meet the target GPU resource quantity if a single channel exists. For all channels that meet the target GPU resource quantity, the usage score of the channel is determined based on the GPU memory occupancy, CPU usage, and memory usage.

[0110] The execution channel determination module is used to obtain the channel with the highest score as the execution channel for the new task.

[0111] In one embodiment, the resource migration module includes:

[0112] A resource missing set determination module is used to calculate, for each valid channel in the valid channel set, the number of resources that the valid channel lacks to satisfy the execution of the new task to obtain a resource missing set;

[0113] a generation migration virtual machine set determination module, configured to determine, based on resource loss corresponding to each valid channel in the valid channel set, virtual machines that meet the conditions in the valid channel to obtain a generation migration virtual machine set;

[0114] A migration cost determination module is used to determine the migration cost of each generation of migration virtual machines in the generation migration virtual machine set according to the video memory occupancy rate and usage rate of the generation of migration virtual machines;

[0115] An initial migration virtual machine determination module is used to obtain a generation migration virtual machine with the lowest migration cost as the initial migration virtual machine of the valid channel;

[0116] A migration severity determination module is configured to calculate the migration severity of the initial migration virtual machine corresponding to each channel, and obtain the initial migration virtual machine with the lowest migration severity as the initial target migration virtual machine;

[0117] The migration result determination module is used to determine if the initial target migration virtual machine can be migrated to a valid channel other than the current channel, then the initial target migration virtual machine is used as the target migration channel, and the migration result is generating the target migration channel; the migration result is not generating the target migration channel.

[0118] In one embodiment, the migration severity determination module includes:

[0119] A parameter determination module is used to determine the average memory occupancy and average usage of the GPU card being used by the initial migration virtual machine and the number of resources missing from the channel where the initial migration virtual machine is located to determine the migration severity;

[0120] Migration severity calculation module, used to calculate the severity of the migration through the formula Obtaining the migration severity of the initial migration virtual machine;

[0121] Where R is the migration severity of the initial migration virtual machine, r1, r2, and r3 are weight coefficients respectively, M is the number of resources missing from the channel where the initial migration virtual machine is located, Q is the average memory occupancy of the GPU card in use, and P is the average usage rate of the GPU card in use.

[0122] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A cloud platform GPU virtual machine scheduling method, characterized in that: The method comprises: Obtain the target number of GPU resources required for the new task, and determine whether the total number of GPU resources remaining in all channels is greater than the target number of GPU resources; each channel has a fixed number of GPU resources; If the total number of remaining GPU resources is greater than the target number of GPU resources, determining whether there is a single channel that satisfies the target number of GPU resources; If no single channel satisfies the target number of GPU resources, then all channels are sorted from most to least based on the number of remaining GPU resources of each channel to obtain an initial resource channel sorting set; Obtain the first preset channels in the initial resource channel sorting set to obtain a valid channel set; Dynamically migrating the resources in the effective channel set using a preset algorithm to obtain a migration result; the migration result includes generating a target migration channel or not generating a target migration channel; The target migration channel is a channel that satisfies the target number of GPU resources; If the migration result is to generate a target migration channel, all target GPU resources of the new task are scheduled to the target migration channel; Dynamically migrating the resources in the effective channel set using a preset algorithm to obtain migration results includes: For each valid channel in the valid channel set, calculating the number of resources that are missing for the valid channel to execute the new task to obtain a resource missing set; For each valid channel in the valid channel set, a resource missing is determined, based on the resource missing, to be a virtual machine that meets the conditions in the valid channel to obtain a generation migration virtual machine set; For each generation of migrating virtual machines in the generation migrating virtual machine set, determining a migration cost of the generation of migrating virtual machines according to the video memory occupancy rate and usage rate of the generation of migrating virtual machines; Obtaining the generation migration virtual machine with the lowest migration cost as the initial migration virtual machine of the valid channel; For each channel, the initial migration virtual machine corresponding to the channel is calculated as the migration severity of the initial migration virtual machine, and the initial migration virtual machine with the lowest migration severity is obtained as the initial target migration virtual machine; If the initial target migration virtual machine can be migrated to a valid channel other than the current channel, the initial target migration virtual machine is used as the target migration channel, and the migration result is generating a target migration channel; the migration result is not generating a target migration channel.

2. A cloud platform GPU virtual machine scheduling method according to claim 1, characterized in that: After determining whether the total number of remaining GPU resources in all channels is greater than the target number of GPU resources, the method further includes: if the total number of remaining GPU resources is less than or equal to the target number of GPU resources, determining that the new task execution fails, and sending an error message to the user.

3. The cloud platform GPU virtual machine scheduling method according to claim 1, characterized in that: After determining whether there is a single channel that meets the target number of GPU resources, the method further includes: If a single channel meets the target number of GPU resources, all channels that meet the target are obtained. For all channels that meet the target, the usage score of the channel is determined based on the GPU memory occupancy rate, CPU usage rate, and memory usage rate. The channel with the highest score is used as the execution channel for the new task.

4. The cloud platform GPU virtual machine scheduling method according to claim 1, characterized in that: Calculating the migration severity of the initial migration virtual machine includes: Determine the average memory occupancy and average usage of the GPU card being used by the initial migration virtual machine and the number of resources missing from the channel where the initial migration virtual machine is located to determine the migration severity; By formula Obtaining the migration severity of the initial migration virtual machine; Where R is the migration severity of the initial migration virtual machine, r1, r2, and r3 are weight coefficients respectively, M is the number of resources missing from the channel where the initial migration virtual machine is located, Q is the average memory occupancy of the GPU card in use, and P is the average usage rate of the GPU card in use.

5. A cloud platform GPU virtual machine scheduling device, characterized in that: The device comprises: A total number of GPU resources determination module is used to obtain the target number of GPU resources required for a new task and, based on the target number of GPU resources, determine whether the total number of GPU resources remaining in all channels is greater than the target number of GPU resources; each channel has a fixed number of GPU resources; a channel determination module, configured to determine whether a single channel satisfies the target number of GPU resources if the total number of remaining GPU resources is greater than the target number of GPU resources; A channel sorting module is configured to sort all channels from most to least by the number of remaining GPU resources of each channel to obtain an initial resource channel sorting set if no single channel meets the target number of GPU resources; An effective channel determination module, configured to obtain the first preset channels in the initial resource channel sorting set to obtain an effective channel set; a resource migration module, configured to dynamically migrate resources in the valid channel set using a preset algorithm to obtain a migration result; the migration result includes generating a target migration channel or not generating a target migration channel; the target migration channel is a channel that meets the target number of GPU resources; A new task scheduling determination module is configured to schedule all target GPU resources of the new task to the target migration channel if the migration result is to generate a target migration channel; The resource migration module includes: a resource missing set determining module, configured to calculate, for each valid channel in the valid channel set, the number of resources that the valid channel lacks to satisfy the execution of the new task, to obtain a resource missing set; a generation migration virtual machine set determination module, configured to determine, based on resource loss corresponding to each valid channel in the valid channel set, virtual machines that meet the conditions in the valid channel to obtain a generation migration virtual machine set; A migration cost determination module is used to determine the migration cost of each generation of migration virtual machines in the generation migration virtual machine set according to the video memory occupancy rate and usage rate of the generation of migration virtual machines; An initial migration virtual machine determination module is used to obtain a generation migration virtual machine with the lowest migration cost as the initial migration virtual machine of the valid channel; A migration severity determination module is configured to calculate the migration severity of the initial migration virtual machine corresponding to each channel, and obtain the initial migration virtual machine with the lowest migration severity as the initial target migration virtual machine; The migration result determination module is used to determine whether the initial target migration virtual machine can be migrated to a valid channel other than the current channel, and the initial target migration virtual machine is used as the target migration channel, and the migration result is generating the target migration channel; the migration result is not generating the target migration channel.

6. The cloud platform GPU virtual machine scheduling device according to claim 5, characterized in that: The device further comprises: The error information sending module is used to determine that the execution of the new task fails if the total number of remaining GPU resources is less than or equal to the target number of GPU resources, and send an error message to the user.

7. The cloud platform GPU virtual machine scheduling device according to claim 5, characterized in that: The device further comprises: A channel score determination module is configured to obtain all channels that meet the target number of GPU resources if a single channel exists, and determine the channel's usage score based on GPU memory occupancy, CPU usage, and memory usage for all channels that meet the target number of GPU resources. The execution channel determination module is used to obtain the channel with the highest score as the execution channel of the new task.

8. The cloud platform GPU virtual machine scheduling device according to claim 5, characterized in that: The migration severity determination module includes: A parameter determination module is used to determine the average memory occupancy and average usage of the GPU card being used by the initial migration virtual machine and the number of resources missing from the channel where the initial migration virtual machine is located to determine the migration severity; Migration severity calculation module, used to calculate the severity of the migration through the formula Obtaining the migration severity of the initial migration virtual machine; Where R is the migration severity of the initial migration virtual machine, r1, r2, and r3 are weight coefficients respectively, M is the number of resources missing from the channel where the initial migration virtual machine is located, Q is the average memory occupancy of the GPU card in use, and P is the average usage rate of the GPU card in use.

Images