A method based on GPU resource pool reuse and scheduling

Abstract

The embodiment of the present invention provides a method and system based on GPU resource pool reuse and scheduling, the method comprising: the resource scheduling layer of the GPU resource pool reuse and scheduling system obtains the GPU resource usage data of each business module in the PaaS business layer according to the preset time interval; the resource scheduling layer determines whether there is at least one business module that meets the GPU resource scheduling conditions according to the GPU resource usage data of each business module obtained at each acquisition moment; if there is a business module that meets the GPU resource scheduling conditions, the GPU resources are scheduled according to the priority of the business module and the preset scheduling strategy, so that the business module after the GPU resources are scheduled adopts the new GPU resources for execution. The embodiment of the present invention provides a method based on GPU resource pool reuse and scheduling, which not only improves the utilization efficiency of GPU resources, but also ensures the execution performance of the business module.

Description

[0001] The present invention relates to the technical field of cloud computing resource management and scheduling, and in particular to a method based on GPU resource pool reuse and scheduling. Background Art

[0002] Cloud computing is an efficient distributed computing model. Its basic principle is to distribute computing tasks on a large number of computers for parallel processing, thereby improving computing efficiency. GPU is a hardware device specifically used to accelerate computer graphics processing. Its powerful parallel computing capabilities and efficient memory access methods make it widely used in cloud computing in the fields of graphics rendering and scientific computing. Currently, its application scenarios include cloud rendering, cloud e-sports, metaverse, GPU virtual machines, AI computing, deep learning, etc.

[0003] However, the existing GPU resource management model is usually based on fixed allocation, which is difficult to dynamically adjust according to actual task requirements. When the tasks of some business modules are completed, the occupied GPU resources may be vacant for a long time, but they are not released in time and transferred to other business modules, resulting in resource waste and increased costs. Due to the fixed nature of resource allocation, it is difficult to dynamically match the optimal computing node for users, which may lead to inefficient resource utilization and affect task response speed and user experience. In addition, current resource scheduling is usually based on manual or periodic tasks, lacks real-time monitoring and intelligent adjustment mechanisms, and cannot quickly perceive and deal with resource vacancy or shortage problems. Summary of the invention

[0004] The embodiment of the present invention provides a method based on GPU resource pool reuse and scheduling to solve the technical problem in the prior art that unreasonable GPU resource scheduling leads to low GPU resource utilization.

[0005] The embodiment of the present invention discloses a method based on GPU resource pool reuse and scheduling, including:

[0006] The resource scheduling layer of the GPU resource pool reuse and scheduling system obtains the GPU resource usage data of each business module in the PaaS business layer at a preset time interval;

[0007] The resource scheduling layer determines whether there is at least one business module that meets the GPU resource scheduling conditions based on the GPU resource usage data of each business module obtained at each acquisition moment; if there is a business module that meets the GPU resource scheduling conditions, the GPU resources are scheduled according to the priority of the business module and the preset scheduling strategy, so that the business module after the GPU resources are scheduled adopts the new GPU resources for execution.

[0008] Preferably, if there is a service module that meets the GPU resource scheduling condition, the GPU resources are scheduled according to the priority of the service module and the preset scheduling strategy, so that the service module after the GPU resources are scheduled adopts the new GPU resources for execution, specifically including:

[0009] If there is a business module that meets the GPU resource scheduling conditions, the GPU resources of each business module that meets the GPU resource scheduling conditions are scheduled in order of priority from high to low using the preset scheduling strategy, so that the business module after the GPU resources are scheduled uses the new GPU resources for execution.

[0010] Preferably, the GPU resources of each service module that meets the GPU resource scheduling conditions are scheduled using a preset scheduling strategy so that the service module after the GPU resources are scheduled adopts the new GPU resources for execution, specifically including:

[0011] According to the number of GPUs in the running state corresponding to the business module, obtain the initial mounted number of GPUs corresponding to the current collection time of the business module;

[0012] According to the initial GPU mount quantity corresponding to the current acquisition time of the business module and the initial GPU mount quantity corresponding to the last acquisition time, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution.

[0013] Preferably, according to the number of GPUs in the running state corresponding to the business module, the initial mounted number of GPUs corresponding to the current acquisition moment of the business module is obtained, which specifically includes:

[0014] According to the number of GPUs in the running state corresponding to the business module, the formula (1) is used to obtain the initial mounted number of GPUs corresponding to the business module at the current collection time;

[0015] Wherein, the formula (1) is:

[0016]

[0017] in, Indicates rounding up;

[0018] α is the idle ratio corresponding to the business module;

[0019] M i is the number of GPUs in running state corresponding to the ith business module arranged in descending order of priority among the business modules that meet the GPU resource scheduling conditions;

[0020] K iis the initial GPU mount quantity corresponding to the ith business module arranged in descending order of priority among the business modules that meet the GPU resource scheduling conditions.

[0021] Preferably, according to the initial GPU mount quantity corresponding to the current acquisition moment of the business module and the initial GPU mount quantity corresponding to the last acquisition moment, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution, specifically including:

[0022] If the initial number of GPUs mounted at the current acquisition time of the business module is less than the initial number of GPUs mounted at the previous acquisition time, select X idle GPUs from the GPUs mounted by the business module and release them to the GPU resource pool of the specified area;

[0023] Wherein, X is the difference between the initial GPU mount quantity corresponding to the last collection time of the service module and the initial GPU mount quantity corresponding to the current collection time.

[0024] Preferably, according to the initial GPU mount quantity corresponding to the current acquisition moment of the business module and the initial GPU mount quantity corresponding to the last acquisition moment, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution, which further specifically includes:

[0025] If the initial GPU mount quantity corresponding to the current acquisition time of the business module is greater than the initial GPU mount quantity corresponding to the previous acquisition time, and when G is less than Z, G GPUs are selected from the GPU resource pool of the specified area and mounted to the business module to be provided to the business module for use;

[0026] G is the difference between the initial GPU mount quantity corresponding to the current acquisition time of the service module and the initial GPU mount quantity corresponding to the last acquisition time;

[0027] in,

[0028] N is the number of GPUs in the GPU resource pool of the specified region;

[0029] I is the number of business modules that meet the GPU resource scheduling conditions.

[0030] Preferably, according to the initial GPU mount quantity corresponding to the current acquisition moment of the business module and the initial GPU mount quantity corresponding to the last acquisition moment, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution, which further specifically includes:

[0031] If the initial GPU mount quantity corresponding to the current acquisition moment of the business module is greater than the initial GPU mount quantity corresponding to the previous acquisition moment, and when G>Z, and G-Z≥S, all Z GPUs in the GPU resource pool of the specified area are mounted on the business module, and in the business modules that meet the GPU resource scheduling conditions, all GPUs mounted on other business modules with a priority lower than that of the business module are deprived and mounted on the business module.

[0032] Preferably, according to the initial GPU mount quantity corresponding to the current acquisition moment of the business module and the initial GPU mount quantity corresponding to the last acquisition moment, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution, which further specifically includes:

[0033] If the initial mounted number of GPUs corresponding to the current acquisition moment of the business module is greater than the initial mounted number of GPUs corresponding to the previous acquisition moment, and when G>Z and G-Z<S, all Z GPUs in the GPU resource pool of the specified area are mounted on the business module, and the mounted GPUs in each business module are deprived in order from low to high priority from the business modules that meet the GPU resource scheduling conditions until the cumulative number of deprived GPUs is M, and the deprived M GPUs are mounted on the business module.

[0034] On the other hand, this embodiment further provides a system based on GPU resource pool reuse and scheduling, the system is used to execute the method based on GPU resource pool reuse and scheduling as described in this embodiment, the system includes an IaaS computing power resource layer, a resource scheduling layer and a PaaS business layer;

[0035] The PaaS business layer includes multiple business modules, each of which is pre-mounted with at least one GPU;

[0036] The IaaS computing resource layer pre-registers GPU resource pools in multiple regions;

[0037] The resource scheduling layer is used to obtain the GPU resource usage data of each business module in the PaaS business layer at preset time intervals, and determine whether there is at least one business module that meets the GPU resource scheduling conditions based on the GPU resource usage data of each business module obtained at each acquisition moment; if there is a business module that meets the GPU resource scheduling conditions, the GPU resources are scheduled according to the priority of the business module and the preset scheduling strategy, so that the business module after the GPU resources are scheduled adopts the new GPU resources for execution.

[0038] Preferably, the GPU resource usage data of the service module includes: the video memory usage rate of each GPU mounted by the service module, and the number of GPUs mounted by the service module;

[0039] The GPU resource usage data of the business module is monitored by a monitoring service module corresponding to the business module pre-deployed in the PaaS business layer;

[0040] The PaaS business layer is pre-deployed with a plurality of monitoring service modules corresponding to the business modules in the PaaS business layer one by one;

[0041] The GPU resource scheduling condition is: the idle ratio corresponding to the service module is different from the preset value;

[0042] The idle ratio corresponding to the service module is the ratio of the number of GPUs in the idle state corresponding to the service module to the number of GPUs in the running state;

[0043] Among them, the idle GPU is a GPU with a video memory usage rate of less than 5%;

[0044] A GPU in a running state is a GPU whose video memory usage is greater than or equal to 5%.

[0045] The embodiments of the present invention include the following advantages:

[0046] In the method based on GPU resource pool reuse and scheduling of this embodiment, in the PaaS business layer, the resource scheduling layer collects the GPU resource usage data of each business module at a preset time interval, thereby ensuring the real-time grasp of the resource usage status of each business module, providing data support for GPU resource scheduling, and improving the accuracy and real-time performance of scheduling decisions. By analyzing the GPU resource usage data collected each time, it is determined whether there is a business module that meets the GPU resource scheduling conditions, and the business module that needs GPU resource adjustment can be accurately identified, avoiding the problem of blind scheduling or unreasonable resource allocation, thereby improving the overall resource utilization efficiency of the system.

[0047] In addition, through priority sorting and preset scheduling strategies, it is ensured that high-priority business modules can obtain GPU resources first, avoiding the delay or failure of key tasks due to resource competition, thereby effectively improving the overall service quality of the system. For business modules that meet the GPU resource scheduling conditions, resource scheduling enables them to use updated GPU resources to perform tasks, ensuring the rationality and dynamic adaptability of resource allocation, while reducing the resource occupation of low-load modules and optimizing resource utilization efficiency.

[0048] Furthermore, the method of this embodiment can intelligently identify resource utilization and make optimization adjustments by dynamically obtaining GPU memory usage, operating status, and scheduling judgment combined with idle ratio, reducing manual intervention and realizing automation and efficiency of GPU resource scheduling. After dynamic resource allocation, the business module can complete tasks in a more efficient manner, avoiding delays or failures caused by insufficient resources, while effectively avoiding the problems of idle resources and waste of resources, and comprehensively improving the reliability and resource utilization of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of a method based on GPU resource pool reuse and scheduling provided in an embodiment of the present invention;

[0050] Figure 2 It is a schematic diagram of the structure of a GPU resource pool reuse and scheduling system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] See also Figure 1 This embodiment provides a method based on GPU resource pool reuse and scheduling, including:

[0053] The resource scheduling layer of the GPU resource pool reuse and scheduling system obtains the GPU resource usage data of each business module in the PaaS business layer at a preset time interval;

[0054] In this embodiment, the resource scheduling layer of the GPU resource pool reuse and scheduling system will collect GPU resource usage data of each business module in the PaaS business layer at fixed time intervals. For example, the collected data includes video memory occupancy, computing load rate, task queue length, etc. These data reflect the demand for GPU resources of each business module. By regularly collecting GPU resource usage data of each business module, it is possible to understand the changes in resource requirements of each business module in real time, ensure rapid response to resource requirements, and avoid performance bottlenecks or resource waste caused by static resource allocation.

[0055] The resource scheduling layer determines whether there is at least one business module that meets the GPU resource scheduling conditions based on the GPU resource usage data of each business module obtained at each acquisition moment; if there is a business module that meets the GPU resource scheduling conditions, the GPU resources are scheduled according to the priority of the business module and the preset scheduling strategy, so that the business module after the GPU resources are scheduled adopts the new GPU resources for execution.

[0056] In this embodiment, based on the collected GPU resource usage data, it is analyzed whether there are business modules that meet the conditions for GPU resource scheduling. For example, when the GPU resource usage data of a business module meets the preset GPU resource scheduling conditions, it is sorted according to its priority, and the GPU resources of this business module are adjusted according to the preset scheduling strategy. For example, high-priority modules will be allocated GPU resources first to ensure that their tasks are completed in time; low-priority business modules may be restricted from GPU resource allocation when resources are tight, and the scheduled business modules will use new GPU resources to perform tasks, thereby optimizing the resource utilization efficiency of the entire system. In this embodiment, the scheduling strategy is combined with the priority of the business module, so that the high-priority business module can obtain the required GPU resources first in the resource competition. This avoids the risk of delay or failure of important tasks caused by insufficient resources and improves the service quality of the overall system. After GPU resource scheduling, the business module can use GPU resources that match its needs, and the execution efficiency of tasks is significantly improved, avoiding task delays caused by insufficient resources, and also avoiding redundancy caused by excess GPU resources.

[0057] In this embodiment, if there is a service module that meets the GPU resource scheduling conditions, the GPU resources are scheduled according to the priority of the service module and the preset scheduling strategy, so that the service module after the GPU resources are scheduled adopts the new GPU resources for execution, specifically including:

[0058] If there is a business module that meets the GPU resource scheduling conditions, the GPU resources of each business module that meets the GPU resource scheduling conditions are scheduled in order of priority from high to low using the preset scheduling strategy, so that the business module after the GPU resources are scheduled uses the new GPU resources for execution.

[0059] In this embodiment, all business modules that meet the scheduling conditions are sorted in order from high to low priority. For example, a high-priority business module may be an important business module specified by the user, while a low-priority business module may process ordinary tasks. Then, GPU resources are scheduled for each business module that meets the scheduling conditions in turn according to the preset scheduling strategy. Through the priority control of the business module, the high-priority business module can be given priority, and the low-priority module will not excessively occupy GPU resources. Combined with the scheduling strategy, the random allocation or waste of GPU resources is avoided, and a more scientific and reasonable resource allocation is achieved. For example, suppose there are three business modules using GPU resources: business module A (low priority), business module B (medium priority), and business module C (high priority). After sorting according to priority, they are: business module C, business module B, business module A, and then according to business module C, business module B, business module A, the business modules are adjusted for GPU resources in this order according to the scheduling strategy (that is, after the business module C is scheduled according to the scheduling strategy, the business module B is scheduled according to the scheduling strategy, and after the business module B is scheduled, the business module A is finally scheduled according to the scheduling strategy). By combining this priority and scheduling strategy, resource competition can be handled efficiently, improving overall resource utilization efficiency and service quality.

[0060] In this embodiment, the GPU resources of each service module that meets the GPU resource scheduling conditions are scheduled using a preset scheduling strategy, so that the service module after the GPU resources are scheduled uses the new GPU resources for execution, specifically including:

[0061] According to the number of GPUs in the running state corresponding to the business module, obtain the initial mounted number of GPUs corresponding to the current collection time of the business module;

[0062] Specifically, the method includes: according to the number of GPUs in the running state corresponding to the business module, using formula (1) to obtain the initial mounted number of GPUs corresponding to the business module at the current collection time;

[0063] Wherein, the formula (1) is:

[0064]

[0065] in, Indicates rounding up;

[0066] α is the idle ratio corresponding to the business module;

[0067] M iis the number of GPUs in running state corresponding to the ith business module arranged in descending order of priority among the business modules that meet the GPU resource scheduling conditions;

[0068] K i is the initial GPU mount quantity corresponding to the ith business module arranged in descending order of priority among the business modules that meet the GPU resource scheduling conditions.

[0069] In this embodiment, it can be known from formula (1) that, assuming M i = 0 (indicates that the business module has no GPU mounted), then a GPU will be given to the business module as the initial number of GPUs mounted corresponding to the business module. i ≠0 (indicates that the service module is equipped with a GPU). In this case, the number of GPUs currently mounted is M. i The idle ratio α is used to calculate the number of GPU resources that need to be additionally mounted and rounded up (as the initial number of GPUs mounted corresponding to the business module).

[0070] In this embodiment, the goal of GPU resource scheduling is to make full use of existing GPU resources and reduce resource waste. Prioritize the allocation of GPU resources to high-priority business modules. The new mount quantity is directly applied to the business module to ensure that it can execute normally after the scheduling is completed. In this embodiment, the scheduling strategy can reserve a certain amount of idle GPUs by introducing the α parameter to avoid performance degradation caused by a sudden increase in load. In addition, by allocating a minimum amount of GPU resources (such as M i = 0, only one GPU is allocated), which reduces the waste of GPU resources. The rounding up processing method in formula (1) makes the allocation result an integer, which is directly usable, reducing the subsequent calculation and debugging costs. On the basis of retaining a certain margin, the GPU scheduling strategy is further refined, taking into account both efficiency and stability.

[0071] For example, suppose that the number of GPUs in the corresponding running state corresponding to a certain business module is M i =2, assuming α=0.2, then the initial number of GPU mounts corresponding to this business module = □2×0.2□+2=3.

[0072] According to the initial GPU mount quantity corresponding to the current acquisition time of the business module and the initial GPU mount quantity corresponding to the last acquisition time, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution.

[0073] This embodiment can understand the GPU resource usage trend of the business module by comparing the initial GPU mount quantity at the current collection time and the previous collection time. If the initial GPU mount quantity of a business module does not change much between two collection times, it can avoid allocating too many GPU resources to it, thereby reducing resource waste.

[0074] The GPU resource adjustment at each acquisition moment is based on a dynamic monitoring and feedback mechanism, which means that it can respond to load changes in real time. Therefore, the GPU resources can be adjusted more accurately based on the current load and historical data. For example, if a business module requires more GPU resources due to a sudden change in demand, it can determine whether the resource allocation needs to be adjusted by comparing the historical and current initial GPU mount numbers to avoid performance bottlenecks caused by sudden load changes. By comparing historical data, the resource requirements of business modules can be judged more accurately instead of blindly allocating fixed resources. This method can identify regular demand changes in business modules, help the scheduling layer to allocate resources more intelligently, avoid overly frequent and ineffective adjustments, and improve the efficiency of resource scheduling.

[0075] In a specific implementation, according to the initial GPU mount quantity corresponding to the current acquisition time of the service module and the initial GPU mount quantity corresponding to the last acquisition time, the GPU resources of the service module are adjusted so that the service module uses the adjusted GPU resources for execution, specifically including:

[0076] If the initial number of GPUs mounted at the current acquisition time of the business module is less than the initial number of GPUs mounted at the previous acquisition time, select X idle GPUs from the GPUs mounted by the business module and release them to the GPU resource pool of the specified area;

[0077] Wherein, X is the difference between the initial GPU mount quantity corresponding to the last collection time of the service module and the initial GPU mount quantity corresponding to the current collection time.

[0078] In the actual application of this embodiment, each service module consumes certain GPU resources at different time points. This embodiment tracks the use of GPU by service modules based on the GPU resource usage data collected at different collection times.

[0079] By comparing the initial number of GPU mounts at this collection time and the last collection time, determine whether resources need to be released and adjusted. If the initial number of GPU mounts at this collection time is less than the initial number of GPU mounts at the last collection time, it means that the business module does not need so many GPU resources at this time, so some GPU resources can be released. X idle GPUs will be released to the GPU resource pool in the specified area, and the value of X is the difference between the number of GPU mounts at this and last collection times.

[0080] For example, suppose there is a business module, and the GPU usage at the current time and the last time is as follows:

[0081] Last collection time: The initial number of GPUs mounted in this business module is 5 GPUs.

[0082] At the time of this collection: the initial number of GPUs mounted in this business module is only 3 GPUs.

[0083] Then, the value of X will be: X = 5-3 = 2;

[0084] This means that at this collection time, the business module will release two idle GPUs. The specific operation is: select two GPUs that are currently not in use (i.e., GPUs in an idle state); remove these two GPUs from the business module and release them to the GPU resource pool in the specified area.

[0085] In this embodiment, the idle GPUs are released to the GPU resource pool, avoiding the waste of GPU resources. The GPUs in the GPU resource pool can be used by other business modules, thereby ensuring that the GPU resources are maximized. For example, if other business modules are performing calculations and the demand for GPUs increases, the released two GPUs can be promptly allocated to these business modules, improving the overall resource utilization.

[0086] Resource allocation can be adjusted dynamically according to the demand fluctuations of different business modules. For business modules with reduced GPU usage, resources that are no longer needed can be released in time to avoid over-allocation and waste. If GPU resources are not released in time, GPU resources will be occupied for a long time, resulting in idle resources being unavailable for other business modules. By releasing excess GPU resources based on the difference (X), resource waste can be effectively avoided, making the system more energy-efficient and efficient. For example, at the last collection time of a business module, the initial number of GPUs mounted on the business module was 5 GPUs. At the current collection time, the initial number of GPUs mounted on the business module was only 3 GPUs. If no adjustment is made, 2 GPUs will be wasted. By releasing these 2 GPUs, these GPU resources can be used by other tasks to avoid unnecessary waste.

[0087] It should be noted that, in this embodiment, if the initial GPU mount quantity corresponding to the current acquisition time of the service module is equal to the initial GPU mount quantity corresponding to the previous acquisition time, no operation is performed on the service module.

[0088] If the initial GPU mount quantity at this collection time is the same as that at the last collection time, it means that the demand for GPUs by the business module has not changed. In this case, there is no need to make any resource adjustments or scheduling. For example, if the GPU demand of a business module has been maintained at a certain stable level, there is no need to repeatedly adjust GPU resources. If scheduling operations are performed without actual changes in GPU resources, system resources and processing time will be wasted. Therefore, through this conditional judgment, the system can avoid invalid operations and improve overall scheduling efficiency.

[0089] For example, suppose that at a certain moment, the initial number of GPUs mounted at the time of this collection of a certain business module (4) is equal to the initial number of GPUs mounted at the time of the last collection (also 4). In this case, it will be determined that the GPU resources have not changed, so no adjustment is required. In actual applications, if these four GPUs are already running without any load changes or adjustments, then avoiding adjustments can give priority to other tasks waiting for GPU resources, thereby improving resource allocation efficiency. For other business modules, if they need higher GPU resources, they can be directly scheduled from the resource pool without worrying about the stable modules affecting the dynamic allocation of resources.

[0090] In the practical application of this embodiment, according to the initial GPU mount quantity corresponding to the current acquisition time of the service module and the initial GPU mount quantity corresponding to the last acquisition time, the GPU resources of the service module are adjusted so that the service module uses the adjusted GPU resources for execution, which specifically includes:

[0091] If the initial GPU mount quantity corresponding to the current acquisition time of the business module is greater than the initial GPU mount quantity corresponding to the previous acquisition time, and when G is less than Z, G GPUs are selected from the GPU resource pool of the specified area and mounted to the business module to be provided to the business module for use;

[0092] G is the difference between the initial GPU mount quantity corresponding to the current acquisition time of the service module and the initial GPU mount quantity corresponding to the last acquisition time;

[0093] in,

[0094] N is the number of GPUs in the GPU resource pool of the specified region;

[0095] I is the number of service modules that meet the GPU resource scheduling conditions.

[0096] Specifically, if at the current collection moment, the initial GPU mount quantity corresponding to this service module is greater than the initial GPU mount quantity corresponding to the previous collection moment, it indicates that the GPU demand of this service module has increased, and it is necessary to adjust and increase the GPU resources allocated to this service module. G is the difference between the initial GPU mount quantities at the current collection moment and the previous collection moment, representing the number of GPUs that need to be newly added this time. Z is the number of available GPUs in the GPU resource pool in the specified area.

[0097] If there are enough available GPUs in the current GPU resource pool (i.e., G < Z), then G GPUs will be selected from the GPU resource pool and allocated for use by this service module.

[0098] For example, assume the initial GPU mount quantities at two collection moments are as follows:

[0099] Previous collection moment: The initial mount quantity of service module A is 6 GPUs.

[0100] Current collection moment: The initial mount quantity of service module A is 9 GPUs.

[0101] Then, the difference of G is: 9 - 6 = 3, indicating that the demand of this service module has increased by 3 GPUs this time. If in the GPU resource pool, G (i.e., 3 GPUs) is less than the number of available GPUs Z (assuming there are 10 available GPUs in the resource pool), then 3 GPUs will be selected from the resource pool and allocated to service module A to meet its demand.

[0102] In this embodiment, by dynamically adjusting the GPU resources according to the difference between the initial GPU mount quantities of the service module at the previous collection moment and the current collection moment, it can respond in a timely manner to the increase in the demand of the service module, thus avoiding the situation of insufficient resources. When there are enough GPUs in the resource pool, it can quickly select and allocate them to the service module from the pool, avoiding resource waste and ensuring that each module can obtain the required resources. Through this dynamic adjustment and allocation, the system can make more efficient use of the GPUs in the resource pool, avoiding waste caused by resource idleness or uneven allocation.

[0103] In one implementation manner of this embodiment, according to the initial GPU mount quantity corresponding to the current collection moment of this service module and the initial GPU mount quantity corresponding to the previous collection moment, adjusting the GPU resources of this service module so that this service module executes with the adjusted GPU resources further specifically includes:

[0104] If the initial GPU mount quantity corresponding to the current acquisition moment of the business module is greater than the initial GPU mount quantity corresponding to the previous acquisition moment, and when G>Z, and G-Z≥S, all Z GPUs in the GPU resource pool of the specified area are mounted on the business module, and in the business modules that meet the GPU resource scheduling conditions, all GPUs mounted on other business modules with a priority lower than that of the business module are deprived and mounted on the business module.

[0105] Specifically, if at the current collection time, the initial GPU mount quantity corresponding to the service module is greater than the initial GPU mount quantity corresponding to the previous collection time, it indicates that the resource demand of the service module has increased.

[0106] G is the difference between the number of GPUs required at this collection time and the number of GPUs required at the last collection time, indicating the newly added GPU demand. Z is the number of GPUs available in the GPU resource pool of the specified area. S is the total number of GPUs mounted on all other business modules whose deprivation priority is lower than that of the business module in the business module that meets the GPU resource scheduling conditions.

[0107] If G is greater than Z (the number of available GPUs in the resource pool), and the difference G-Z is greater than or equal to S, then all Z GPUs are mounted on the business module: If the conditions are met, all Z GPUs in the resource pool are allocated to the business module.

[0108] Deprive the GPU of the low-priority module. Among other business modules that meet the GPU resource scheduling conditions, if their priority is lower than that of the current business module, the mounted GPUs will be deprived from these business modules and their GPU resources will be allocated to the current business module.

[0109] For example, assume that the priority is ranked from high to low as follows: business module C, business module B, business module A; assume that at the time of this collection, the initial number of GPUs mounted on business module C is 12 GPUs. At the last collection time, the initial number of GPUs mounted on business module C was 2 GPUs.

[0110] Therefore, G=12-2=10, which means that business module C requires 10 more GPUs than last time.

[0111] The number of GPUs available in the resource pool (Z):

[0112] Assume that the number of available GPUs Z in the resource pool is 5 GPUs, which is far less than the 10 GPUs required by business module C. Then all available GPUs in the resource pool are mounted to business module C first, and the GPUs already mounted by business modules B and A, which have lower priority than business module C, are deprived and then mounted to business module C.

[0113] In a specific implementation, according to the initial GPU mount quantity corresponding to the current acquisition time of the service module and the initial GPU mount quantity corresponding to the last acquisition time, the GPU resources of the service module are adjusted so that the service module uses the adjusted GPU resources for execution, which further specifically includes:

[0114] If the initial mounted number of GPUs corresponding to the current acquisition moment of the business module is greater than the initial mounted number of GPUs corresponding to the previous acquisition moment, and when G>Z and G-Z<S, all Z GPUs in the GPU resource pool of the specified area are mounted on the business module, and the mounted GPUs in each business module are deprived in order from low to high priority from the business modules that meet the GPU resource scheduling conditions until the cumulative number of deprived GPUs is M, and the deprived M GPUs are mounted on the business module.

[0115] For example, assume that the priority is ranked from high to low as follows: business module C, business module B, business module A; assume that at the time of this collection, the initial number of GPUs mounted on business module C is 12 GPUs. At the last collection time, the initial number of GPUs mounted on business module C was 2 GPUs.

[0116] Therefore, G=12-2=10, which means that business module C requires 10 more GPUs than last time.

[0117] The number of available GPUs in the resource pool (Z), assuming that the number of available GPUs Z in the resource pool is 5 GPUs, which is far less than the difference of 10 GPUs required by business module C this time. Then all the available GPUs in the resource pool are mounted on business module C first. Even if all 5 GPUs in the resource pool are mounted on business module C, the needs of business module C cannot be met. At this time, assuming that business module B currently has 4 GPUs mounted, and business module A currently has 3 GPUs mounted, then in order of priority from low to high, first deprive business module A of the 3 GPUs already mounted, and mount them on business module C, and then deprive business module B of the 2 GPUs, and then mount them on business module C. In this way, the GPUs on the business modules can meet the needs. The number of resource pool Z is not enough to meet the needs of business module C. By depriving the GPU resources of low-priority business modules A and B, the needs of business module C are finally met. This mechanism can effectively deal with the situation of insufficient resources in reality and ensure the operation of key tasks. When resources are tight, through dynamic adjustment, the GPU demand of business module C can be fully met, ensuring the smooth completion of its key computing tasks. Although low-priority business modules A and B are deprived of some resources, they have a lower priority and are relatively less important, so the impact on the overall operation of the system is controllable.

[0118] In this embodiment, resource deprivation is performed from low to high priority of the service module, thereby ensuring the operation of the high-priority service module. In the case of limited computing resources, the performance problems of the high-priority service module caused by insufficient resources can be minimized. By comparing the changes in resource requirements at two acquisition moments, it is possible to quickly respond to changes in the number of GPUs loaded by the service module, adjust the resource allocation plan in real time, and ensure the flexibility of resource allocation. The strategy of depriving low-priority service modules of resources ensures that resources are always used for the most currently needed task modules, rather than being maintained on low-priority service modules that do not require a large amount of resources, thereby avoiding the waste of GPU resources. The number of deprived resources (M) is limited by the current demand difference (G-Z), and resource deprivation is performed in order of priority, thereby reducing the impact on low-priority service modules, balancing conflicts between service modules, and avoiding unnecessary resource competition.

[0119] On the other hand, this embodiment further provides a GPU resource pool-based reuse and scheduling system, the system is used to execute the GPU resource pool-based reuse and scheduling method in the embodiment, the system includes an IaaS computing power resource layer, a resource scheduling layer and a PaaS business layer;

[0120] The PaaS business layer includes multiple business modules, each of which is pre-mounted with at least one GPU;

[0121] The IaaS computing resource layer pre-registers GPU resource pools in multiple regions;

[0122] The resource scheduling layer is used to obtain the GPU resource usage data of each business module in the PaaS business layer at preset time intervals, and determine whether there is at least one business module that meets the GPU resource scheduling conditions based on the GPU resource usage data of each business module obtained at each acquisition moment; if there is a business module that meets the GPU resource scheduling conditions, the GPU resources are scheduled according to the priority of the business module and the preset scheduling strategy, so that the business module after the GPU resources are scheduled adopts the new GPU resources for execution.

[0123] Specifically, the IaaS computing resource layer in the system based on GPU resource pool reuse and scheduling provided in this embodiment includes GPU resource pools in multiple regions, which are pre-registered and prepared for upper layer calls. The underlying GPU hardware resource support is provided to ensure the flexibility and scalability of computing power.

[0124] The resource scheduling layer in the system is responsible for collecting GPU resource usage data of each business module and monitoring it at preset time intervals. Based on the GPU resource usage data collected each time, it determines whether there is a business module that meets the scheduling conditions. According to the priority of the business module and the preset scheduling strategy, resource allocation or adjustment operations are performed.

[0125] The PaaS business layer includes multiple business modules, each of which is pre-mounted with at least one GPU for task execution. Different business modules have different priorities and resource requirements, and resources are dynamically allocated based on priority.

[0126] Through dynamic scheduling and reuse of GPU resources, waste caused by static resource allocation is reduced. When the load of low-priority business modules is low, GPU resources are released to high-priority business modules to maximize resource utilization. Prioritize the GPU requirements of high-priority business modules to ensure that critical tasks can obtain sufficient computing power support and improve system reliability and service quality.

[0127] The GPU resource usage data of the service module includes: the video memory usage rate of each GPU mounted by the service module, and the number of GPUs mounted by the service module;

[0128] The GPU resource usage data of the business module is monitored by a monitoring service module corresponding to the business module pre-deployed in the PaaS business layer;

[0129] The PaaS business layer is pre-deployed with a plurality of monitoring service modules corresponding to the business modules in the PaaS business layer one by one;

[0130] In this embodiment, by monitoring the detailed data of GPU memory usage and the number of GPUs, the resource usage of each business module can be accurately understood. Based on the memory usage, the idle state (<5%) and running state (≥5%) of the GPU are defined, providing a clear quantitative standard to facilitate the system to identify and process the state of the GPU.

[0131] The GPU resource scheduling condition is: the idle ratio corresponding to the service module is different from the preset value;

[0132] The idle ratio corresponding to the service module is the ratio of the number of GPUs in the idle state corresponding to the service module to the number of GPUs in the running state;

[0133] The idle ratio is dynamically calculated based on the ratio of idle GPUs to running GPUs, and it can reflect the changes in the resource requirements of business modules in real time. When the idle ratio is different from the preset value, it indicates that the current resource configuration is no longer suitable for business needs, thus triggering the scheduling operation to ensure the rationality of resource allocation.

[0134] Among them, the idle GPU is a GPU with a video memory usage rate of less than 5%;

[0135] A GPU in a running state is a GPU whose video memory usage is greater than or equal to 5%.

[0136] In this embodiment, a system based on GPU resource pool reuse and scheduling monitors the usage rate of video memory, accurately identifies idle GPU resources with low load, recycles and dynamically schedules them to business modules with high demand, and avoids resource waste. It can ensure that resources are always allocated to the business modules that need them most, thereby achieving efficient use of resources. Through the priority of the business module, the system gives priority to meeting the GPU requirements of high-priority business modules. Even if resources are insufficient, resources can be deprived of low-priority business modules to ensure the continuity and performance stability of high-priority tasks and improve the reliability of the system.

[0137] For an example, see Figure 2 In a system based on GPU resource pool reuse and scheduling in this embodiment, the IaaS computing power resource layer includes GPU resource pools distributed in multiple areas, such as "GPU resource pool 1" and "GPU resource pool n". These resource pools are uniformly managed and scheduled through the Kubernetes (K8S) platform to form a logically sharable GPU resource pool. The GPU resource pool provides computing power support for different types of high-performance computing applications (that is, business modules), including: Cloud rendering: used to process high-quality graphics rendering tasks. Cloud gaming: provides real-time rendering services for online games. Metaverse: supports the construction of virtual environments and real-time interaction. Other GPU-intensive tasks.

[0138] In this embodiment, Figure 2 The K8S GPU resource pool is a logical unit in Kubernetes for unified management and scheduling of GPU resources. It provides infrastructure support for efficient use of GPUs by multiple tenants.

[0139] The resource scheduling layer is the core of the system and is responsible for monitoring and managing the allocation and scheduling of GPU resources.

[0140] Collect GPU usage: Through the monitoring service module corresponding to each business module, the GPU usage data is obtained at preset time intervals, including video memory usage, GPU idle rate and load status. Based on the collected GPU usage data, determine whether there are modules with idle or insufficient resources, and then determine whether the scheduling conditions are met. For business modules that meet the scheduling conditions, schedule them from high to low priority, dynamically adjust the number of GPU allocations, ensure full utilization of resources, and support multiple business modules to share the same resource pool.

[0141] The PaaS business layer includes multiple business modules (such as cloud gaming, cloud rendering, and Metaverse), which directly use GPU resources to complete computing tasks. Each business module pre-mounts a certain number of GPUs, and the resource scheduling layer dynamically adjusts the number of allocations. GPUs with a memory usage rate of less than 5% are marked as "idle", thereby enabling the flow of GPUs between different business modules and reducing resource waste.

[0142] In this embodiment, a system based on GPU resource pool reuse and scheduling realizes resource reuse by dynamically monitoring and scheduling to avoid the GPU being idle for a long time. Reduce the waste caused by idle GPU resources and reduce the cost of hardware investment. Dynamically allocate GPU resources to match the optimal computing power node for users and shorten the scheduling response time of tasks. Specifically, in a data center environment, different business modules (such as cloud rendering, cloud gaming, and AI reasoning) share a unified GPU resource pool. The system in this embodiment can dynamically allocate the GPU resource pool according to the priority and scheduling strategy of the business module to avoid resource competition or waste.

[0143] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0144] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0145] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method based on GPU resource pool reuse and scheduling, characterized in that: include: The resource scheduling layer of the GPU resource pool reuse and scheduling system obtains the GPU resource usage data of each business module in the PaaS business layer at a preset time interval; The resource scheduling layer determines whether there is at least one business module that meets the GPU resource scheduling conditions based on the GPU resource usage data of each business module obtained at each acquisition moment; if there is a business module that meets the GPU resource scheduling conditions, the GPU resources are scheduled according to the priority of the business module and the preset scheduling strategy, so that the business module after the GPU resources are scheduled adopts the new GPU resources for execution.

2. The method based on GPU resource pool reuse and scheduling according to claim 1, characterized in that: If there is a business module that meets the GPU resource scheduling conditions, the GPU resources are scheduled according to the priority of the business module and the preset scheduling strategy, so that the business module after the GPU resources are scheduled uses the new GPU resources for execution, including: If there is a business module that meets the GPU resource scheduling conditions, the GPU resources of each business module that meets the GPU resource scheduling conditions are scheduled in order of priority from high to low using the preset scheduling strategy, so that the business module after the GPU resources are scheduled uses the new GPU resources for execution.

3. The method based on GPU resource pool reuse and scheduling according to claim 2, characterized in that: The GPU resources of each service module that meets the GPU resource scheduling conditions are scheduled using a preset scheduling strategy so that the service module after the GPU resources are scheduled uses the new GPU resources for execution, specifically including: According to the number of GPUs in the running state corresponding to the business module, obtain the initial mounted number of GPUs corresponding to the current collection time of the business module; According to the initial GPU mount quantity corresponding to the current acquisition time of the business module and the initial GPU mount quantity corresponding to the last acquisition time, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution.

4. The method based on GPU resource pool reuse and scheduling according to claim 3, characterized in that: According to the number of GPUs in the running state corresponding to the business module, the initial number of GPUs mounted corresponding to the business module at the current collection time is obtained, including: According to the number of GPUs in the running state corresponding to the business module, the formula (1) is used to obtain the initial mounted number of GPUs corresponding to the business module at the current collection time; Wherein, the formula (1) is: in, Indicates rounding up; α is the idle ratio corresponding to the business module; M i is the number of GPUs in running state corresponding to the ith business module arranged in descending order of priority among the business modules that meet the GPU resource scheduling conditions; K i is the initial GPU mount quantity corresponding to the ith business module arranged in descending order of priority among the business modules that meet the GPU resource scheduling conditions.

5. The method based on GPU resource pool reuse and scheduling according to claim 4, characterized in that: According to the initial GPU mount quantity corresponding to the current acquisition time of the business module and the initial GPU mount quantity corresponding to the last acquisition time, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution, specifically including: If the initial number of GPUs mounted at the current acquisition time of the business module is less than the initial number of GPUs mounted at the previous acquisition time, select X idle GPUs from the GPUs mounted by the business module and release them to the GPU resource pool of the specified area; Wherein, X is the difference between the initial GPU mount quantity corresponding to the last collection time of the service module and the initial GPU mount quantity corresponding to the current collection time.

6. The method based on GPU resource pool reuse and scheduling according to claim 5, characterized in that: According to the initial GPU mount quantity corresponding to the current acquisition time of the business module and the initial GPU mount quantity corresponding to the last acquisition time, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution, which also specifically includes: If the initial GPU mount quantity corresponding to the current acquisition time of the business module is greater than the initial GPU mount quantity corresponding to the previous acquisition time, and when G is less than Z, G GPUs are selected from the GPU resource pool of the specified area and mounted to the business module to be provided to the business module for use; G is the difference between the initial GPU mount quantity corresponding to the current acquisition time of the service module and the initial GPU mount quantity corresponding to the last acquisition time; in, N is the number of GPUs in the GPU resource pool of the specified region; I is the number of business modules that meet the GPU resource scheduling conditions.

7. The method based on GPU resource pool reuse and scheduling according to claim 6, characterized in that: According to the initial GPU mount quantity corresponding to the current acquisition time of the business module and the initial GPU mount quantity corresponding to the last acquisition time, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution, which also specifically includes: If the initial GPU mount quantity corresponding to the current acquisition moment of the business module is greater than the initial GPU mount quantity corresponding to the previous acquisition moment, and when G>Z, and G-Z≥S, all Z GPUs in the GPU resource pool of the specified area are mounted on the business module, and in the business modules that meet the GPU resource scheduling conditions, all GPUs mounted on other business modules with a priority lower than that of the business module are deprived and mounted on the business module.

8. The method based on GPU resource pool reuse and scheduling according to claim 7, characterized in that: According to the initial GPU mount quantity corresponding to the current acquisition time of the business module and the initial GPU mount quantity corresponding to the last acquisition time, the GPU resources of the business module are adjusted so that the business module uses the adjusted GPU resources for execution, which also specifically includes: If the initial mounted number of GPUs corresponding to the current acquisition moment of the business module is greater than the initial mounted number of GPUs corresponding to the previous acquisition moment, and when G>Z and G-Z<S, all Z GPUs in the GPU resource pool of the specified area are mounted on the business module, and the mounted GPUs in each business module are deprived in order from low to high priority from the business modules that meet the GPU resource scheduling conditions until the cumulative number of deprived GPUs is M, and the deprived M GPUs are mounted on the business module.

9. A GPU resource pool reuse and scheduling system, characterized in that: The system is used to execute the method based on GPU resource pool reuse and scheduling as described in any one of claims 1 to 8, and the system includes an IaaS computing resource layer, a resource scheduling layer and a PaaS business layer; The PaaS business layer includes multiple business modules, each of which is pre-mounted with at least one GPU; The IaaS computing resource layer pre-registers GPU resource pools in multiple regions; The resource scheduling layer is used to obtain the GPU resource usage data of each business module in the PaaS business layer at preset time intervals, and determine whether there is at least one business module that meets the GPU resource scheduling conditions based on the GPU resource usage data of each business module obtained at each acquisition moment; if there is a business module that meets the GPU resource scheduling conditions, the GPU resources are scheduled according to the priority of the business module and the preset scheduling strategy, so that the business module after the GPU resources are scheduled adopts the new GPU resources for execution.

10. The method based on GPU resource pool reuse and scheduling according to claim 9, characterized in that: The GPU resource usage data of the service module includes: the video memory usage rate of each GPU mounted by the service module, and the number of GPUs mounted by the service module; The GPU resource usage data of the business module is monitored by a monitoring service module corresponding to the business module pre-deployed in the PaaS business layer; The PaaS business layer is pre-deployed with a plurality of monitoring service modules corresponding to the business modules in the PaaS business layer one by one; The GPU resource scheduling condition is: the idle ratio corresponding to the service module is different from the preset value; The idle ratio corresponding to the service module is the ratio of the number of GPUs in the idle state corresponding to the service module to the number of GPUs in the running state; The idle GPU is a GPU with a memory usage rate of less than 5%; A GPU in a running state is a GPU whose video memory usage is greater than or equal to 5%.

Images